Monitor a Django app with Scout

Doug Breaker  on August 24, 2018

In this post, I'll show how to setup Scout to monitor the performance of SQL queries, external HTTP calls, template rendering, and more in Wagtail, a Django CMS app.

Wagtail is a fast, modern opensource content management system built on Django. Used at NASA, Google, MIT, and more, it's a great option for running your own CMS. When we add the scout-apm package to the app, we'll quickly gain insights on the app's performance.

I'll start with the performance monitoring basics, then move onto advanced settings we've found valuable for monitoring our own apps at Scout.

Setup

Scout is distributed via the scout-apm Python package and configuration takes just a couple of lines of code. Once configured, Scout auto-instruments Django apps, tracking the performance of individual web requests, SQL queries, and more.

If you don't have a Scout account, signup for a 14-day trial. After signup, follow our Django install instructions.

Wagtail uses environment-specific setting files. I'd suggest the following approach to configure Scout:

bakerydemo/settings/base.py

# Scout settings
SCOUT_MONITOR = True
SCOUT_NAME    = WAGTAIL_SITE_NAME
SCOUT_KEY     = "SCOUT_KEY" # Grab your key from the Scout UI

dev and prod environments

Scout aggregates metrics by the application name. It's useful to separate your apps by environment as your dev environment often behaves very differently than production.

I set unique names for dev and production environments.

In bakerydemo/settings/dev.py:

SCOUT_NAME = SCOUT_NAME + " [DEV]"

In bakerydemo/settings/production.py:

SCOUT_NAME = SCOUT_NAME + " [PROD]"

Loading the SCOUT_KEY from environment variables

If you prefer to keep your API keys out of version control, you can set a SCOUT_KEY environment variable instead. The Scout agent will use this key when reporting metrics. Remove SCOUT_KEY from base.py to avoid confusion.

Restart your app.

That's it! Just run python manage.py runserver to start your app server, send some requests to your app, and verify your Wagtail app appears in Scout.

Overview

After restarting your app, you'll see an overview of your app's performance within Scout:

overview

Scout's basic installation automatically tracks SQL, view, and rendering performance.

You can also explore the performance of specific views via the "Web Endpoints" area of Scout:

endpoints

...and dig into the performance of a specific view as well:

search_view

GitHub Integration

Scout can quickly connect a slow method call to the code behind it via our GitHub integration. To enable the GitHub integration, access "GitHub" under "Integrations" in the app settings:

search_view

Authorize Scout via GitHub's OAuth screen. Once authorize, you can view the code behind slow method calls right in your browser:

trace

Deploy Tracking

Changes to our app can trigger performance issues. After configuring deploy tracking, it's easy to correlate these changes to performance. When combined with the GitHub integration, we can also view a diff of changes between deploys.

I'm using Heroku to deploy my Wagtail app, so all I need to do is enable dyno metadata:

heroku labs:enable runtime-dyno-metadata

You'll see deploy markers on charts going forward:

deploy_marker

Scout's GitHub integration enriches deploy tracking, providing a diff summary of changes when you mouseover a deploy marker or the Git SHA next to your app name.

Error Monitoring

Scout doesn't provide detailed error monitoring by itself, but it easily integrates with exception monitoring services like Rollbar. You can enable the Rollbar integration in your app settings:

rollbar_scout

After adding the integration, you'll see application errors right on your Scout dashboard, giving you a single pane-of-glass for both performance and errors:

"rollbar<emscoutoverview"/>

Alerting

Let's have Scout notify us via email if our app performance degrades. Configure an alert on "95th percentile response time":

alert

This alert will be emailed to users of our choice.

Context & Trace Explorer

Many performance issues aren't experienced by everyone: they are triggered in specific situations. For example, a power user accessing a lot of data is more likely to see slower response times. Sometimes this performance profile may be hidden in overview metrics.

Scout has a context API: this lets you add metadata about your requests. For example, Scout automatically tracks the current user in the session. I'll add context with the search query term used and the number of search results found on Wagtail's search:

...then, in Scout's Trace Explorer, I can filter traces in realtime, accessing traces for specific search queries:

trace_explorer

Custom Instrumentation

Last but not least, Scout instruments many popular Python libraries, but not your custom code. For this, you can add custom instrumentation.

Let's add some custom instrumentation around the search results pagination:

You'll see time spent in this category on your overview charts and within traces:

custom_instru

TL;DR

Scout makes monitoring the performance of Django apps affordable and approachable in just a couple of lines of code. In this walkthrough, I've added Scout to Wagtail, an opensource Django CMS. You can find my modifications to the Wagtail Bakerydemo app on GitHub.

Try Scout for Django monitoring today with our 14-day, credit card-less trial.

Also See