Stackify vs. New Relic vs. Scout | APM Tool Comparison

Stackify Retrace primarily supports Java, .NET, PHP, Nodej.js, Ruby, and Python applications. New Relic supports Java, node.js, Python, Go, PHP, .NET, and Ruby. On the other hand, Scout APM supports Ruby, Python, Node.js, PHP, Elixir & Phoenix, in addition to Error Monitoring, Database Monitoring and External Services Monitoring.

Comparison Summary

The core features of Stackify are detailed tracing on SQL queries and application/server log management. It supports Async & .NET Core and detailed code level profiling. In contrast, New Relic provides a setup of real-time instrumentation and analytics.

Scout APM provides detailed insights on performance directly on its dashboard. It identifies slow database queries, N+1 database queries, errors, and memory bloat.

Scout provides better transaction traces by breaking down memory allocation in addition to timing metrics. Scout APM can easily backtrace slow method calls to identify performance bottlenecks and tie them to source code. Database Monitoring provides charts to display data across each query (not just the top five expensive queries).

Another important feature of Scout is, it’s easy to compare performance in a small slice of time to the normal performance. It also provides an improved development Profiler.

Integrations

Stackify Retrace integrates with Jira, Slack, Axosoft, Azure DevOps, and AWS AutoScale.

New Relic provides integrations of all the services on major cloud providers. Also, It can integrate with Slack, Lighthouse(ticketing system), Databases, and Kubernetes.

Scout integrates with popular developer tools such as GitHub, Rollbar, Bugsnag, Sentry, Slack, PagerDuty, and Honeybadger.

Core APM Product Features

Stackify Core Features

Tracing SQL Queries

Stackify helps trace SQL queries, including ORM Queries, Relations, and its Associations. 

Managing Server Logs

You can configure Logging frameworks such as NLog, log4j, Serilog, log4net and logback, and more. Stackify collects different logs to monitor the system:

• Application logs
• Syslog
• Windows Event
• Web server logs (IIS, Apache, Nginx)

Code Level Profiling

It involves monitoring memory usage, CPU for the components or code level. This process, however, adds overhead to the application and may affect the application’s performance. Retrace uses an optimized method to minimize the overhead with efficient performance.  

Some of the metrics for code profiling are,

• CPU Usage
• Does background task affects the performance
• Collecting more data regarding the profiling and analysis

New Relic Core Features

Easy to set up real-time instrumentation and analytics

New Relic Dashboard provides metrics such as response time, throughput, error rate, and transactions.

The analysis provides a clear understanding of the whole system’s performance. An APM tool needs to offer a profound experience and clear picture of the analysis for performance monitoring.

A drawback with profiling in New Relic is the system load spikes when it starts crushing data in the background.

Mapping Application Performance to the end-user experience

New Relic analyzes end-user experience through real-user monitoring, synthetic monitoring, and mobile app performance analysis.

New Relic provides a Deployment Marker to describe the impact of that code changes on your application. It’s important to understand this for better decision-making for the business requirement.

Error analysis in real-time with on-demand diagnostic tools

An APM tool’s essential function is to help the developer quickly understand the potential source of that issue and debug to find a solution when a problem occurs. 

New Relic agents provide the ability to analyze errors using an error stack trace.

It provides a diagnostic tool such as Thread Profiler, which offers functionality to see a periodic sampling in real application traces. 

Scout APM Core Features

In-depth Performance Analysis 

Scout is an easy-to-use APM tool that requires a much simpler and streamlined setup process than New Relic or Stackify. Scout APM provides an algorithmic agent that digs through your application and provides in-depth analysis and insights. For example, it provides analysis on 

• Slow database queries
• N+1 database queries
• Memory bloat
• Errors

Fast and Efficient Transaction traces

Transaction traces analyze time spent on a single web request or execution of a single background job.

• While tracing transactions, Scout provides a complete breakdown of memory allocation and timing metrics which is crucial to analyze the performance bottleneck.
• On tracing SQL Queries to monitor the database performance, Scout indicates the row count returned by SQL queries which helps monitor the database.
• Another essential feature is, backtracking slow requests. There may be various reasons for a slow API request. Scout helps to backtrace it and ease the process of debugging the performance bottleneck.

Database Monitoring

Monitoring a database is crucial for application performance. An expensive query may cause other queries to run slower. So, it’s important to monitor and analyze them before significantly impacting your application.

• New Relic and Stackify show metrics for expensive database queries only, and while it can be good on a higher level, you need to understand how each query performs to optimize it. This is where Scout excels as the UI visualizes metrics for all queries, not just the expensive ones.
• Scout APM focuses on ActiveRecords. At the same time, New Relic monitors beyond ActiveRecords. (Active Records provide an abstraction layer between code and data. So you don’t need to write raw SQL queries. Object-relational mapping(ORM) uses this pattern).
• In Scout, it’s easy to compare a small slice’s performance with an average performance metric. 

Usability and Dashboards

Stackify UI

Stackify Dashboard has five sections: Dashboard, Monitoring, Performance, Errors, and Logs.

Dashboard

In the dashboard, you will see information about Health, Performance, and Errors. The Health section will show Availability, User Satisfaction, HTTP Error, and User Satisfaction calculated by Apdex. Logs or APM profiler will calculate HTTP Error metrics.

Monitoring

Performance

It will report how well your application performs overall as perceived by the users. The chart will describe your different requests such as Fast, Sluggish, Too Slow, and Failed requests. 

Errors

In the Errors Section, you can see New, Regressed (or) All Errors. You can also see an overall error rate reported from exception logging or APM profiler.  

New Relic UI

Dashboard

Web transaction time shows the time it took for each transaction. On the top right, is the Apdex score (User satisfaction).

In the middle, you get details about each transaction and its response time. Error rate shows the overall error rate and its average.  

Transaction traces

New Relic provides cross-application tracing if your app depends on another application you’ve monitored with NewRelic. It includes a list of expensive transactions and their details.

Web Endpoints

It shows all the web endpoints and their resource consumption. It also shows the top five web transactions and their throughput.

When you click on the web endpoint, it details its performance breakdown, throughput, and transaction traces. A Histogram displays the details about transaction response time. 

Database Monitoring

Database Monitoring shows the top database operation by its time consumption, query time, and throughput. You can sort the database queriest by most time consumed.

Scout APM UI

Dashboard

Scout dashboard shows the overview of different categories. They are,

• Response time
• Throughput
• Errors
• Memory consumption
• Apdex Score

Scout digs through the data and generates great insights into application performance. For example, it provides metrics and insights on slow database queries, N+1 database queries, errors, and memory bloat.

Transaction traces

Scout provides a breakdown of memory allocation to time metrics. If there are any slow method calls, Scout backtraces it to troubleshoot, and it displays in the dashboard along with the code. It also breakdown the time spent on custom code. 

Web Endpoint

It displays the overview of the API endpoint in list format. It provides a filterable and sortable view. 

When you click a specific endpoint, it provides memory allocation metrics and a detailed listing of transaction traces(up to 10 per minute).

Database Monitoring

Monitoring a database is crucial for application performance. Scout provides charts that display data across every query.  It shows metrics across time consumed, throughput, and mean query time. Scout’s list of queries includes calling the controller of actions or background jobs.

Integration Options

While monitoring the application performance, debugging the issue without any delay is essential. To achieve that, you may need to integrate APM tools with your organization tools, such as bug tracker, project management tools, and communication tools.

Let’s compare how each tool integrates with a third-party application.

Stackify Integrations

Jira Cloud Integration

Integrate Retrace with Jira to push errors into the bug tracking system. 

Slack Integration

Stackify Retrace integrates with Slack to send updates or alerts to specific channels. It mainly helps to alert if a performance bottleneck or CPU/Memory usage is beyond the threshold.

You can also integrate Slack via webhooks in Retrace.

Axosoft Integration

Axosoft is a Project Management tool that integrates with Retrace to track issues/logs for specific tasks. It comes in handy if Retrace tracks multiple products and keeps updated with information/status about the respective product.

WebHooks

WebHooks is a way for an application to provide information to other applications. It delivers the data to other applications as it happens. Retrace provides WebHooks to configure with the products you use daily. It can be Bug tracker, Project Management, or Communication tools.

Azure DevOps

You can push logging exceptions from Retrace to Azure DevOps services using Azure DevOps integration.

AWS AutoScale

Retrace integration with AWS helps monitor AWS services such as EC2, Elastic BeanStalk, and EKS instances. It also tracks the server status, monitoring service events such as creation, removal, and other auto-scaling events.

New Relic Integration

Jira Cloud Integration

New Relic had Jira integration that helps to track the issues. Here’s the app. But Jira integration is suspended. Read more about the details here.

Slack Integration

Slack integration has become a significant part of APM tools since many organizations are using it. New Relic provides slack integration for system alerts and updates. Application is available in the App Marketplace. You can configure various applications to get alert notifications.

Ticketing System Integration

Integrating with the ticketing system helps to track all the performance issues. New Relic provides integration for a Ticketing application called LightHouse out of the box.

Cloud Service Integration

New Relic integrates well with all the cloud provides available in the industry. On top of that, it includes integration for most of the services available in each cloud provider. 

It also integrates with Databases(MongoDB, Postgres, MySQL, Neo4j), Kubernetes and Istio, etc.

Scout APM Integrations

One of my favorite features of Scout is how well it integrates with some critical third-party applications. It provides integration out of the box for popular tools and applications. 

GitHub

Integration with Github helps trace the transactions at the code level. It allows the developer to track the performance bottleneck. Scout trace the expensive SQL queries along with N+1 queries in code level profiling.

For enabling the Github integration, Scout annotates the deployment with the specific git branch. It helps to analyze the deployment in a better way.

Sentry 

If you’re using Sentry to monitor Exceptions in your application, it’s easy to configure Sentry with Scout to track the Sentry exceptions via Scout Dashboard. You don’t need to jump back and forth to do the job.

Note: Sentry configuration is app-specific in Scout. You need to provide a read-only access token from Sentry to Scout.

Slack

To integrate with Slack with Scout, you need to use Zapier for the configuration. Scout provides WebHooks to configure inside Zapier for Slack communication.

Rollbar

Scout displays errors from the app’s associated Rollbar project alongside performance data within the Scout UI when the Rollbar integration is enabled.

Honeybadger

When Honey Badger (A popular error monitoring tool) integration is enabled, Scout dashboard displays error from Honey Badger to the dashboard directly.

Installation

Stackify Installation

Stackify installation varies based on the programming language, deployment type, and cloud service provider. It installs an agent on the server responsible for monitoring the application and code-level profiling.

You can install it on any operating system such as windows and Linux operating system levels.

A few takeaways with Stackify installation are,

• You have to install and enable the Retrace to make it work. It adds a layer of additional configuration compared with other tools such as New Relic and Scout.
• Guidelines and documentation to install and enable agents are available only for programming languages such as Java and .NET.

New Relic Installation

You can configure the New Relic agent on various programming languages such as C, Go, Java, .NET, Node.js, PHP, Python, and Ruby.  New Relic provides an agent you can install across different levels for different functionalities:

• Code Level Profiling
• Browser Monitoring
• Infrastructure Monitoring
• Mobile Monitoring
• Synthetic Monitoring

Since it provides configuration across the different levels, it has a steep learning curve. Configuration can be overwhelming, and it needs training to do the job. Also, use cases for windows installation will be helpful to configure the New Relic agent on Windows operating systems.

Scout Installation

Scout’s agent installation process is simple and straightforward. It provides agents for Ruby, Elixir, Python, PHP, and Node.js. You can directly install an agent in your application as a dependency.

For example, you can install an agent in your Node.js application with:

@scoutapm/scout-apm

If the Scout agent doesn’t support your application’s programming language, you can further configure the agent for your application performance.