There’s no type of monitoring tool that can get to the heart of a performance problem faster than Application Performance Management (APM).
If you’re making this decision, you may be choosing between New Relic and Scout. There are cases where New Relic is a better fit, clear cases where Scout fits like a glove, and some murky situations as well. Where do you fall?
Below, I’ll summarize the factors to consider when choosing your best APM product.
Pricing
Scout’s monthly fee is about 30% lower than the monthly fee in a New Relic annual contract. In most cases, Scout will also buy your New Relic contract.
Language Support
Scout supports Ruby, Elixir, PHP, Node.js and Python with a Core Agent API that can be used to instrument any language. New Relic supports Ruby, Java, Node.js, PHP, .NET, Python, and Go.
Let’s start digging through the products, starting with the app overview.
App Overview
This is the first page you see when checking the health of your app. New Relic’s version is on the left and Scout is on the right:
The top portion of the page is similar between New Relic and Scout: a breakdown of time spent by category (ex: Ruby, Database, External HTTP services, etc) over time. You can view data across similar timeframes in both Scout and New Relic (New Relic offers three months of data in their Pro package and Scout can do the same in their custom plans).
Both let you compare data to the past. Scout gives a few more options than New Relic, which is restricted to yesterday and last week comparisons.
The biggest difference with Scout’s approach: several algorithms dig through your data and generate insights on performance directly on this page. For example, Scout identifies slow database queries, N+1 database queries, and memory bloat. These are ordered by impact, which help you decide where to get most improvements in the least amount of time.
Scout short-circuits the work a Site Reliability Engineer (SRE) often performs when investigating the health of an app.
Let’s move from the 10,000 foot view to the soul of application monitoring.
Transaction Traces
The soul of app monitoring is the transaction trace: all the metrics and analysis an APM platform delivers originates from data collected in a trace. A transaction trace is a breakdown of time spent in a single web request or execution of a single background job.
A New Relic trace is on the left and Scout on the right:
There are some differences here as well:
- Scout provides a breakdown of memory allocations in addition to timing metrics. New Relic does not.
- New Relic offers cross-application tracing if your app touches another app you’ve monitored with New Relic. Scout does not.
- Scout is more likely to provide backtraces to slow method calls and can also display the code directly in the browser.
- Scout indicates the number of rows returned by SQL queries
- Scout can breakdown the time spent in custom code via ScoutProf. New Relic requires custom instrumentation.
Scout typically collects more traces than New Relic. Both New Relic and Scout have similar, one-week retention periods for traces.
 |
Scout Founder’s Take
For the trace, we focus on action: which code paths are significant bottlenecks? What is the code in those bottlenecks? What are the details around slow SQL queries (ie: number of records returned)? |
Web Endpoints
Both New Relic and Scout let you view summary data on web endpoints in list format. New Relic on the left, Scout on the right:
New Relic opts for several different visualizations on this page. Scout goes with a filterable, sortable view.
Both sort web endpoints by percent time consumed by default, helping you focus on the controller-actions that consume the greatest amount of time.
|
Scout Founder’s Take
My favorite feature on this page is the filter above the list of endpoints. It drastically reduces the time needed to navigate to an endpoint or a group of endpoints I’m investigating. Since this is a high-frequency action, the time savings adds up.
|
Web Endpoint Detail
In the same way the app overview provides a breakdown of time spent, both New Relic and Scout offer endpoint-specific views for this:
There are some differences on this page:
- New Relic offers a histogram display option of response times and Scout does not.
- Scout offers memory allocation metrics (more allocations lead to increased memory usage) while New Relic does not.
- New Relic’s breakdown provides more categories, while Scout focuses the breakdown on database queries.
- Scout typically provides more transaction traces (up to 10 per-minute) and more options for sorting and selecting traces.
Background Jobs
Both New Relic and Scout monitor the core Ruby background job frameworks: Sidekiq, Delayed Job, and Resque. The UI is similar to endpoints, so I haven’t provided a screenshot here.
Database Monitoring
The database is the most common bottleneck for web applications. Since a database is a shared resource, one expensive query may cause many other types of queries to run slower. New Relic and Scout provide additional analytics around database query performance:
Both of the UIs resemble each tool’s web endpoints display, but made specific for database queries. There are some subtle yet important differences between the two:
- Scout’s chart displays data across every query – not just the top five most expensive queries – by combining less expensive queries into an “other queries” metric. This gives a more complete picture of performance.
- New Relic provides monitoring beyond ActiveRecord. Scout focuses on ActiveRecord.
- Scout’s list of queries includes the calling controller-action or background job. Expensive queriers are frequently triggered by a few hotspots in the code: this makes it easier to identify those spots at a glance.
Finally, Scout makes it easy to compare performance in a small slice of time to the normal performance. This isn’t possible with New Relic – you can change the entire timezone, but you can’t compare between a small slice of time and a larger one. Here’s how this looks within Scout:
Development Profiler
When you are actively resolving a performance issue, it’s helpful to get immediate feedback on the results as you code. Otherwise, you need to deploy and wait for fresh data to verify the change had the intended impact.
New Relic used to have a developer mode, but it has since been removed from the agent.
Scout DevTrace gives immediate access to traces in your local development environment by clicking on a speed badge in the lower left of your browser:
|
Scout Founder’s Take
Slow feedback cycles – like a long-running test suite – are painful. DevTrace eliminates a lot of this as you can instantly verify that an optimization – like fixing an N+1 query with `includes` – is working as expected before deploying. |
Weekly Trends Email
A recurring report of your app’s health is a great way for managers to stay in the loop on performance and to uncover slowly building problems. New Relic puts more emphasis on high-level numbers. Scout digs more into the source of problems and trends:
|
Scout Founder’s Take
Weekly emails with lots of metrics become repetitive. We attempt to dig through the data and identify underlying causes – like a specific slow query – to push you towards a fix. |
Alerting
Scout and New Relic have comparable alerting functionality.
Agent
Both the New Relic and Scout agents are battle-tested. In Scout’s open-sourced benchmarks, Scout’s agent was shown to have lower overhead:
| APM Agent | Response Time
(Mean)
|
Response Time
(95th Percentile)
|
Response Time
(Max)
|
Overhead | |
|---|---|---|---|---|---|
| None | 55.6 ms | 106.4 ms | 2,174.1 ms | ||
| New Relic | 80.4 ms | 149.5 ms | 2,263.5 ms | 44.5% | |
| Scout | 56.8 ms | 102.7 ms | 2,168.7 ms | 2.2% |
The impact of agent overhead increases as CPU resources on your app servers becomes more scarce.
Error Analytics
Scout tracks error rates and integrates with third-party services like Rollbar to provide details on exceptions. New Relic can provide this data, including backtraces to exceptions.
Why don’t you just tell me which to use?
New Relic and Scout require little configuration – outside adding a dependency – to try. You can run Scout and New Relic at the same time without causing conflicts.
My suggestion? Try Scout alongside New Relic, then ask: “which tool helps our team solve performance issues faster?”
doug+scoutapp@scripted.com