May 22, 2008
We run background jobs on our Scout servers. Lots of them.
As we’ve grown, they’ve used dramatically more resources. We needed a way to simmer them down. Most of these jobs load the entire Rails environment – that’s a hefty overhead.
We’ve modified how we run background jobs and we’re seeing great results. If you’re interested in what we did, checkout Charles’ post on our Highgroove blog.
And finally, here’s the obligatory semi-promotional graph, generated by Scout, of how the load dramatically decreased after implementation: