Taking the guesswork out of scaling

Doug Breaker  on April 24, 2008

Determining a web application’s hardware resources isn’t easy (or cheap). Frankly, it’s often just guesswork. Even when you build benchmarking scripts, they can miss key behaviors and ignore important metrics.

Scaling becomes a lot less stressful when you can quickly compare a history of your application data with server performance.

For example, we did this to get a better understanding of how our Scout server performed during our invitation process. The graph below was generated through Scout and shows the relationship between user accounts and the server load. As we expected, the overall load on the server increased as the number of accounts increased. Scout shows us how this data is correlated – it gives us an idea of how many accounts our current hardware can support.

Scout Accounts vs. Server Load

It’s trivial process to regularly feed Scout your application data (user signups, orders, revenue, etc):

  1. Start with this Rails app Plugin Sampe (this assumes a Ruby on Rails application, but you can do this with any framework/language)
  2. Grab your application data – just use ActiveRecord!
  3. Put the plugin on your server (can protect behind basic auth)
  4. Add the plugin