The nuts and bolts of our Ruby-based realtime charts solution
May 31, 2012
Scout's realtime charts have been a big hit. Once you start using them for major deploys or performance incidents, going back to ten terminal windows running "top" feels like the dark ages.
So, how did we go about it?
The Scout agent runs a process on your server(s), sending metrics to Pusher's realtime PaaS every second or so. Pusher publishes those metrics in realtime through websockets to any browser that’s connected and authenticated for your metrics.
So the essence of realtime:
- Scout-managed process on your server
- HTTP posts from your server to Pusher
- Websocket connections between Pusher and browser(s)
Diagrams are for project managers, right? Here's how it looks in simplified Ruby code:
The Scout Agent (Ruby)
So far so good – but what about session control and security?
Realtime Session Control
You initiate a realtime session by clicking the realtime button on a Scout chart. When clicked, scoutapm.com will send a directive to the agent (which runs once a minute) telling it to spin up a a realtime streamer process.
Under normal operation, the Scout agent sends data directly to scoutapm.com. Only when viewing a realtime chart is data sent to Pusher.
We need to be careful that only members of the originating Scout account have access to reatime data as it flows through the 3rd party service (Pusher). Here’s how we restrict access:
Why didn't we roll our own?
We don't stream metrics unless you initiate a realtime session – most of the time, Scout isn't using realtime. This means our realtime volume isn’t huge. We're able to use Pusher's startup plan ($49/mo), which is a trivial expense compared to managing our own realtime infrastructure.
We investigated two Ruby EventSource servers for handling web sockets: Cramp and Goliath. It was clear that just using Pusher was a much simpler path for our limited use case.
- Use Pusher. Rolling your own realtime infrastructure isn’t trivial. Using a PaaS like Pusher lets you start focusing on the experience faster.
- Don’t be intimidated by realtime. Is there anything in your product that can be enhanced by realtime communication?