Configuring OpenTelemetry in Python

Overview

For this post, we’ll use a small Flask app that allows users to input a city name and they receive the current weather information for that city. We’ll make an API call to openweathermap.org to get the weather information for the city.

weather.py

import json
import urllib

from flask import Flask, render_template, request

app = Flask(__name__)
api_key = 'YOUR_API_KEY_FROM_OPENWEATHERMAP'
api_url = 'http://api.openweathermap.org/data/2.5/weather?units=imperial&appid=' + api_key

@app.route('/', methods =['POST', 'GET'])
def get_weather():
    if request.method == 'POST':
        city = request.form['city']
    else:
        city = 'Denver'
  
    ret = urllib.request.urlopen(api_url + '&q=' + city).read()
    json_ret = json.loads(ret)
    data = process_result(json_ret)
    return render_template('index.html', data = data)

def process_result(json_result):
	return {
        'city_name': str(json_result['name']),
        'country_code': str(json_result['sys']['country']),
        'lat': str(json_result['coord']['lat']),
        'lon': str(json_result['coord']['lon']),
        'temp': str(json_result['main']['temp']),
        'pressure': str(json_result['main']['pressure']),
        'humidity': str(json_result['main']['humidity']),
    }

if __name__ == '__main__':
    app.run(debug = True)

templates/index.html

<!DOCTYPE html>
<html lang="en" dir="ltr">

<head>
  <meta charset="utf-8">
  <title>Weather Demo</title>
  <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.0/css/bootstrap.min.css">
  <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
  <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.0/js/bootstrap.min.js"></script>
</head>

<body>
  <nav class="row" style="background: #337ab7; color: white;">
    <h1 class="col-md-3 text-center">Weather Demo</h1>
  </nav>
  <br />
  <br />
  <center class="row">
    <form method="post" class="col-md-6 col-md-offset-3">
      <div class="input-group">
        <input type="text" class="form-control" name="city" placeholder="Search">
        <div class="input-group-btn">
          <button class="btn btn-primary" type="submit">
            <i class="glyphicon glyphicon-search"></i>
          </button>
        </div>
        <form>
  </center>
  <div class="row">
    {% if data.temp and data.pressure and data.humidity %}
    <div class="col-md-6 col-md-offset-3">
      <h3>Country code: {{data.country_code}}</h1>
        <h5>City name: {{data.city_name}}</h5>
        <h5>Coordinates: lat {{data.lat}}, lon {{data.lon}}</h5>
        <h5>Temp: {{data.temp}}F</h5>
        <h5>Pressure: {{data.pressure}} </h5>
        <h5>Humidity: {{data.humidity}}</h5>
    </div>
    {% endif %}
  </div>
</body>

</html>

Installing the API and SDK

First we install the API and SDK packages for OpenTelemetry:

pip install opentelemetry-api
pip install opentelemetry-sdk

Instrumentation

The opentelemetry-python-contrib repo contains out of the box instrumentation for a number of popular Python packages. To install these, you can simply insert the appropriate name in the command below:

pip install opentelemetry-instrumentation-{instrumentation}

As of June 2022, these are the available packages:

aiohttp
aiopg
asgi
asyncpg
aws
boto
boto3sqs
botocore
celery
confluent
dbapi
django
elasticsearch
falcon
fastapi
flask
grpc
httpx
jinja2
kafka
logging
mysql
pika
psycopg2
pymemcache
pymongo
pymysql
pyramid
redis
remoulade
requests
sklearn
sqlalchemy
sqlite3
starlette
system
tornado
urllib
urllib3
wsgi

Our example application uses Flask and Urllib, so we’ll install those:

pip install opentelemetry-instrumentation-flask
pip install opentelemetry-instrumentation-urllib

And then add to our code:

from opentelemetry import trace
from opentelemetry.instrumentation.flask import FlaskInstrumentor
from opentelemetry.instrumentation.urllib import URLLibInstrumentor
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import (
    ConsoleSpanExporter,
    BatchSpanProcessor,
)

trace.set_tracer_provider(TracerProvider())
trace.get_tracer_provider().add_span_processor(
    BatchSpanProcessor(ConsoleSpanExporter())
)
tracer = trace.get_tracer(__name__)

app = Flask(__name__)
FlaskInstrumentor().instrument_app(app)
URLLibInstrumentor().instrument()

This will allow us to see the traces as json output to our console. The trace payloads will look like this:

{
    "name": "HTTP GET",
    "context": {
        "trace_id": "0xf4e29ce1a54c98087264d9ccf63532c3",
        "span_id": "0x9cb4ea193590822c",
        "trace_state": "[]"
    },
    "kind": "SpanKind.CLIENT",
    "parent_id": "0x15482de6b18e14e9",
    "start_time": "2022-06-07T22:36:58.573072Z",
    "end_time": "2022-06-07T22:36:58.724174Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "http.method": "GET",
        "http.url": "http://api.openweathermap.org/data/2.5/weather?units=imperial&appid=REDACTED&q=Denver",
        "http.status_code": 200
    },
    "events": [],
    "links": [],
    "resource": {
        "telemetry.sdk.language": "python",
        "telemetry.sdk.name": "opentelemetry",
        "telemetry.sdk.version": "1.11.1",
        "service.name": "unknown_service"
    }
}
{
    "name": "/",
    "context": {
        "trace_id": "0xf4e29ce1a54c98087264d9ccf63532c3",
        "span_id": "0x15482de6b18e14e9",
        "trace_state": "[]"
    },
    "kind": "SpanKind.SERVER",
    "parent_id": null,
    "start_time": "2022-06-07T22:36:58.557230Z",
    "end_time": "2022-06-07T22:36:58.741642Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "http.method": "POST",
        "http.server_name": "127.0.0.1",
        "http.scheme": "http",
        "net.host.port": 5000,
        "http.host": "127.0.0.1:5000",
        "http.target": "/",
        "net.peer.ip": "127.0.0.1",
        "http.user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36",
        "net.peer.port": 55668,
        "http.flavor": "1.1",
        "http.route": "/",
        "http.status_code": 200
    },
    "events": [],
    "links": [],
    "resource": {
        "telemetry.sdk.language": "python",
        "telemetry.sdk.name": "opentelemetry",
        "telemetry.sdk.version": "1.11.1",
        "service.name": "unknown_service"
    }
}

Custom Instrumentation

In a real-world application, you’ll want to measure parts of your code that does not have instrumentation provided out of the box by OpenTelemetry. Let’s add custom instrumentation to measure our process_result method:

def process_result(json_result):
    with tracer.start_as_current_span("process_result"):
        return {
            'city_name': str(json_result['name']),
            'country_code': str(json_result['sys']['country']),
            'lat': str(json_result['coord']['lat']),
            'lon': str(json_result['coord']['lon']),
            'temp': str(json_result['main']['temp']),
            'pressure': str(json_result['main']['pressure']),
            'humidity': str(json_result['main']['humidity']),
        }

You can also use a decorator:

@tracer.start_as_current_span("process_result")
def process_result(json_result):
    return {
        'city_name': str(json_result['name']),
        'country_code': str(json_result['sys']['country']),
        'lat': str(json_result['coord']['lat']),
        'lon': str(json_result['coord']['lon']),
        'temp': str(json_result['main']['temp']),
        'pressure': str(json_result['main']['pressure']),
        'humidity': str(json_result['main']['humidity']),
    }

Attributes

Attaching attributes to spans is extremely important to understand how your application behaves in certain contexts. This can be done like so:

def process_result(json_result):
    with tracer.start_as_current_span("process_result"):
        span = trace.get_current_span()
        span.set_attribute("city", str(json_result['name']))
        return {
            'city_name': str(json_result['name']),
            'country_code': str(json_result['sys']['country']),
            'lat': str(json_result['coord']['lat']),
            'lon': str(json_result['coord']['lon']),
            'temp': str(json_result['main']['temp']),
            'pressure': str(json_result['main']['pressure']),
            'humidity': str(json_result['main']['humidity']),
        }

Resource Configuration

From the OpenTelemetry spec, a Resource:

Resource captures information about the entity for which telemetry is recorded. For example, metrics exposed by a Kubernetes container can be linked to a resource that specifies the cluster, namespace, pod, and container name.

Resource may capture an entire hierarchy of entity identification. It may describe the host in the cloud and specific container or an application running in the process.

In other words, the resource attributes attached to a trace provide the metadata to identify where the trace was generated, and any other useful context that enables the user searching traces to filter and drill down to specific values of the attributes. E.g. Show me traces:

We’ll set the resource name like so:

from opentelemetry.sdk.resources import Resource

resource = Resource(attributes={
    "service.name": "service"
})

trace.set_tracer_provider(TracerProvider(resource=resource))

Baggage

Our example application is stand-alone, however in a real-world application you’ll likely want to enable distributed tracing so you can trace a transaction though all of your services. By default, the W3C Trace Context for baggage propagation. If you want to use B3, change it like so:

from opentelemetry import propagators
from opentelemetry.sdk.trace.propagation.b3_format import B3Format

propagators.set_global_textmap(B3Format())

Putting It All Together

Our final code example now looks like this:

import json
import urllib

from flask import Flask, render_template, request

# OpenTelemetry
from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import (
    ConsoleSpanExporter,
    BatchSpanProcessor,
)
# OpenTelemetry Instrumentation
from opentelemetry.instrumentation.flask import FlaskInstrumentor
from opentelemetry.instrumentation.urllib import URLLibInstrumentor

# Set OpenTelemetry Resource Attributes
resource = Resource(attributes={
    "service.name": "weather-demo"
})

# Configure OpenTelemetry Tracer and Exporter
trace.set_tracer_provider(TracerProvider(resource=resource))
trace.get_tracer_provider().add_span_processor(
    BatchSpanProcessor(ConsoleSpanExporter())
)
tracer = trace.get_tracer(__name__)

app = Flask(__name__)
# Apply OpenTelemetry Instrumenation
FlaskInstrumentor().instrument_app(app)
URLLibInstrumentor().instrument()

api_key = 'YOUR_API_KEY_FROM_OPENWEATHERMAP'
api_url = 'http://api.openweathermap.org/data/2.5/weather?units=imperial&appid=' + api_key

@app.route('/', methods =['POST', 'GET'])
def get_weather():
    if request.method == 'POST':
        city = request.form['city']
    else:
        city = 'Denver'
  
    ret = urllib.request.urlopen(api_url + '&q=' + city).read()
    json_ret = json.loads(ret)
    data = process_result(json_ret)
    return render_template('index.html', data = data)

def process_result(json_result):
    with tracer.start_as_current_span("process_result"):
        span = trace.get_current_span()
        span.set_attribute("city", str(json_result['name']))
        return {
            'city_name': str(json_result['name']),
            'country_code': str(json_result['sys']['country']),
            'lat': str(json_result['coord']['lat']),
            'lon': str(json_result['coord']['lon']),
            'temp': str(json_result['main']['temp']),
            'pressure': str(json_result['main']['pressure']),
            'humidity': str(json_result['main']['humidity']),
        }

if __name__ == '__main__':
    app.run(debug = True)

Wrapping Up

In this post, we’ve just scratched the surface of how to use OpenTelemetry in Python. Be sure to check out these other resources for learning more about OpenTelemetry in Python:

https://open-telemetry.github.io/opentelemetry-python/index.html

https://opentelemetry-python.readthedocs.io/en/latest/index.html

https://opentelemetry.io/docs/instrumentation/python/

If you’re interested in learning more about OpenTelemetry, Observability, and what we’re working on at Scout, sign up for our Observability newsletter or opt-in to beta test our new product at https://scoutapm.com/observability.