Log Management
Scout’s Log Management feature allows you to monitor, search, and analyze your Ruby application logs directly within the Scout UI. By integrating with our existing APM agent, we provide enhanced context and filtering capabilities.
Installation
AAdd the scout_apm_logging gem to your gemfile.
gem 'scout_apm_logging'
BConfigure Scout in your scout_apm.yml configuration file:
common: &defaults
# ... other Scout APM settings
logs_monitor: true
logs_ingest_key: aaaa-1111-aaaa-1111 # Provided in App Logs Page
CDeploy!
It takes approximately five minutes for your data to first appear within the Scout UI.
Configuration Options
The following configuration settings are available. These can be set in the scout_apm.yml configuration file or as environment variables with the SCOUT_ prefix, e.g. SCOUT_LOGS_INGEST_KEY.
Only logs_ingest_key is required. The rest are optional.
| Setting Name | Description | Default | Required |
|---|---|---|---|
| logs_ingest_key | The Ingest Key to use for logs | Yes | |
| logs_monitor | True or false. If true, monitor logs and send them to Scout | false |
No |
| logs_capture_level | The minimum log level to capture and send to Scout | debug |
No |
| log_level | Log level for the agent itself | info |
No |
Advanced Configuration
These settings are internal and are not typically needed for normal operation, but if things like filesystem access cause issues, the following may be useful.
| Setting Name | Description | Default | Required |
|---|---|---|---|
| logs_proxy_log_dir | The directory to store logs in for monitoring | nil |
No |
| logs_capture_call_stack | True or false. If true, capture the call stack for each log message | false |
No |
| logs_capture_log_line | True or false. If true, capture the log line for each log message | false |
No |
| logs_call_stack_search_depth | The number of frames to search in the call stack | 10 |
No |
| logs_call_stack_capture_depth | The number of frames to capture in the call stack | 10 |
No |
| logs_method_missing_warning | True or false. If true, log a warning when method_missing is called |
false |
No |
| logs_method_missing_call_stack | True or false. If true, capture the call stack when method_missing is called |
false |
No |
| logs_config | A hash of configuration options for merging into the collector’s config | {} |
No |
| logs_reporting_endpoint | The endpoint to send logs to | https://logs.scoutapm.com |
No |
| log_stdout | True or false. If true, log to STDOUT | false |
No |
| log_stderr | True or false. If true, log to STDERR | false |
No |
| log_file_path | Either a directory or "STDOUT" |
nil |
No |
| log_class | The underlying class to use for logging. Defaults to Ruby’s Logger class | Logger |
No |
How It Works
For Rails applications, if you are using 7.1+ we will add a custom logger, which has a formatter that utilizes the OpenTelemetry Ruby SDK to send logs to us, to the broadcast logger. For Rails applications before 7.1, we will use this same custom logger but we will also create a proxy logger which we will swap out for all instances of the old logger, and this proxy logger will log to the old logger as well as the custom one.
The logs will be available in the Scout UI for you to search and filter. We will automatically set a custom log formatter to include additional context in the logs sent to us. Your original logs will not be altered.
Default Attributes
The Scout Logs agent enriches Rails logs with a few attributes by default:
- entrypoint: The top-level action (e.g. Controller class, in Rails) that the log was generated from.
- location: The file and line number where the log was generated.
In addition, it will capture all key-value pairs from any Custom Context that you have set. This means logs can be filtered by any Custom Context attributes.
Data Retention
Scout retains your log data for 14 days. If you require longer-term storage, please let us know as we would like to create options for our customers, but at this point you will need to also send them to an alternative location.
