January 15, 2009
Last week I uncovered a tricky performance problem on Scout I wanted to share.
In seemingly random fashion, when accessing my account home page, the render time was 5-6 seconds – way too slow. Other times, the render time would be far less than a second.
Here’s what I did to debug:
Plain as day, I saw a query taking 30+ seconds. A simple mysql EXPLAIN on the query revealed a missing index. A new index fixed the issue. This query was outside the Rails application – a background job – so I never saw the query when optimizing the application.
Checkout the impact on the DB server load:
It’s a great lesson that you really need the complete picture of your infrastructure when something goes wrong – just looking at the Rails application often doesn’t cut it.
P.S. You’ll love what’s coming up with Scout – finding these problems gets a lot easier.