October 15, 2017
There's no type of monitoring tool that can get to the heart of a performance problem faster than Application Performance Management (APM).
If you're making this decision, you may be choosing between New Relic and Scout. There are cases where New Relic is a better fit, clear cases where Scout fits like a glove, and some murky situations as well. Where do you fall?
Below, I'll summarize the factors to consider when choosing your best APM product.
Scout's monthly fee is about 30% lower than the monthly fee in a New Relic annual contract. In most cases, Scout will also buy your New Relic contract.
Let's start digging through the products, starting with the app overview.
This is the first page you see when checking the health of your app. New Relic's version is on the left and Scout is on the right:
The top portion of the page is similar between New Relic and Scout: a breakdown of time spent by category (ex: Ruby, Database, External HTTP services, etc) over time. You can view data across similar timeframes in both Scout and New Relic (New Relic offers three months of data in their Pro package and Scout can do the same in their custom plans).
Both let you compare data to the past. Scout gives a few more options than New Relic, which is restricted to yesterday and last week comparisons.
The biggest difference with Scout's approach: several algorithms dig through your data and generate insights on performance directly on this page. For example, Scout identifies slow database queries, N+1 database queries, and memory bloat. These are ordered by impact, which help you decide where to get most improvements in the least amount of time.
Scout short-circuits the work a Site Reliability Engineer (SRE) often performs when investigating the health of an app.
Let's move from the 10,000 foot view to the soul of application monitoring.
The soul of app monitoring is the transaction trace: all the metrics and analysis an APM platform delivers originates from data collected in a trace. A transaction trace is a breakdown of time spent in a single web request or execution of a single background job.
A New Relic trace is on the left and Scout on the right:
There are some differences here as well:
Scout typically collects more traces than New Relic. Both New Relic and Scout have similar, one-week retention periods for traces.
Scout Founder's TakeFor the trace, we focus on action: which code paths are significant bottlenecks? What is the code in those bottlenecks? What are the details around slow SQL queries (ie: number of records returned)?
Both New Relic and Scout let you view summary data on web endpoints in list format. New Relic on the left, Scout on the right:
New Relic opts for several different visualizations on this page. Scout goes with a filterable, sortable view.
Both sort web endpoints by percent time consumed by default, helping you focus on the controller-actions that consume the greatest amount of time.
Scout Founder's TakeMy favorite feature on this page is the filter above the list of endpoints. It drastically reduces the time needed to navigate to an endpoint or a group of endpoints I'm investigating. Since this is a high-frequency action, the time savings adds up.
In the same way the app overview provides a breakdown of time spent, both New Relic and Scout offer endpoint-specific views for this:
There are some differences on this page:
Both New Relic and Scout monitor the core Ruby background job frameworks: Sidekiq, Delayed Job, and Resque. The UI is similar to endpoints, so I haven't provided a screenshot here.
The database is the most common bottleneck for web applications. Since a database is a shared resource, one expensive query may cause many other types of queries to run slower. New Relic and Scout provide additional analytics around database query performance:
Both of the UIs resemble each tool's web endpoints display, but made specific for database queries. There are some subtle yet important differences between the two:
Finally, Scout makes it easy to compare performance in a small slice of time to the normal performance. This isn't possible with New Relic - you can change the entire timezone, but you can't compare between a small slice of time and a larger one. Here's how this looks within Scout:
When you are actively resolving a performance issue, it's helpful to get immediate feedback on the results as you code. Otherwise, you need to deploy and wait for fresh data to verify the change had the intended impact.
New Relic used to have a developer mode, but it has since been removed from the agent.
Scout DevTrace gives immediate access to traces in your local development environment by clicking on a speed badge in the lower left of your browser:
Scout Founder's TakeSlow feedback cycles - like a long-running test suite - are painful. DevTrace eliminates a lot of this as you can instantly verify that an optimization - like fixing an N+1 query with `includes` - is working as expected before deploying.
A recurring report of your app's health is a great way for managers to stay in the loop on performance and to uncover slowly building problems. New Relic puts more emphasis on high-level numbers. Scout digs more into the source of problems and trends:
Scout Founder's TakeWeekly emails with lots of metrics become repetitive. We attempt to dig through the data and identify underlying causes - like a specific slow query - to push you towards a fix.
Scout and New Relic have comparable alerting functionality.
Both the New Relic and Scout agents are battle-tested. In Scout's open-sourced benchmarks, Scout's agent was shown to have lower overhead:
|APM Agent||Response Time
|None||55.6 ms||106.4 ms||2,174.1 ms|
|New Relic||80.4 ms||149.5 ms||2,263.5 ms||44.5%|
|Scout||56.8 ms||102.7 ms||2,168.7 ms||2.2%|
The impact of agent overhead increases as CPU resources on your app servers becomes more scarce.
Scout tracks error rates and integrates with third-party services like Rollbar to provide details on exceptions. New Relic can provide this data, including backtraces to exceptions.
New Relic and Scout require little configuration - outside adding a dependency - to try. You can run Scout and New Relic at the same time without causing conflicts.
My suggestion? Try Scout alongside New Relic, then ask: "which tool helps our team solve performance issues faster?"