Developer Happiness: a Five-Part Series

The New York Times ran a popular Op-Ed on Happiness last year. The Cliff Notes version?

The daily activities most associated with happiness are sex, socializing after work and having dinner with others. The daily activity most injurious to happiness is commuting.

Many software developers are fortunate to have flexible work hours that reduce (or eliminate) commute times. But what about that those times when you’re writing code and can’t have sex, socialize, or meet friends for dinner?

(1 of 5): Make it Trivial to Grab Production Data

You need to be able to troubleshoot data-specific issues in the comfort and serenity of your development environment.

So, you need an easy way to grab production data and bring it local. In the simplest case, this is just a rake task that 1) ssh’s into your production db and mysqldumps your database; 2) gzips it and scps it locally; 3) unzips and imports the dump into your development DB.

That approach doesn’t scale. As your product grows, the all-or-nothing dump won’t be enough. Your database will be too big to dump quickly, and it will take too long to transmit over the wire. This is not the way to developer happiness.

What you need is a way to export a slice of your application’s data. There is no way to generalize this; it’s totally application specific. For Scout, the most sensible partitioning approach is to download one account and all its associated data. The trick is to identify the associated data to be export as it cascades through your application. The basic recipe we use for this is:

1. a local rake task invokes a remote SelectiveExporter class, passing the account ids we wish to export.
2. the SelectiveExporter does a sequence of mysqldump calls, assembling the set of data related to the account to export.
3. we scp the result and import it locally.

The payoff is huge: we’re able to grab data needed to troubleshoot a problem in 15 seconds instead of 30 minutes. That puts a smile on our developer faces.

Subscribe to our RSS feed or follow us on Twitter for the rest of the developer happiness series.