Puma::Plugin::Telemetry

Puma plugin which should be able to handle all your metric needs regarding your webserver:

• ability to publish basic puma statistics (like queue backlog) to both logs and datadog
• ability to add custom target whenever you need it
• ability to monitor puma socket listen queue (!)
• ability to report requests queue time via custom rack middleware - the time request spent between being accepted by Load Balancer and start of its processing by Puma worker

Add this line to your application's Gemfile:

gem "puma-plugin-telemetry"

And then execute:

$ bundle install

Or install it yourself as:

$ gem install puma-plugin-telemetry

In your puma configuration file (i.e. config/puma.rb or config/puma/<env>.rb):

plugin "telemetry"

Puma::Plugin::Telemetry.configure do |config|
  config.enabled = true

  # << here rest of the configuration, examples below
end

Output telemetry as JSON to STDOUT

  config.add_target :io

Given gem provides built in target for Datadog Statsd client, that uses batch operation to publish metrics.

NOTE Be sure to have dogstatsd gem installed.

  config.add_target :dogstatsd, client: Datadog::Statsd.new

You can provide all the tags, namespaces, and other configuration options as always to Datadog::Statsd.new method.

For detailed documentation checkout Puma::Plugin::Telemetry::Config class.

Puma::Plugin::Telemetry.configure do |config|
  config.enabled = true
  config.initial_delay = 10
  config.frequency = 30
  config.puma_telemetry = %w[workers.requests_count queue.backlog queue.capacity]
  config.socket_telemetry!
  config.socket_parser = :inspect
  config.add_target :io, formatter: :json, io: StringIO.new
  config.add_target :dogstatsd, client: Datadog::Statsd.new(tags: { env: ENV["RAILS_ENV"] })
end

Target is a simple object that implements call methods that accepts telemetry hash object. This means it can be super simple proc or some sofisticated class calling some external API.

Just be mindful that if the API takes long to call, it will slow down frequency with which telemetry will get reported.

  # Example logfmt to stdout target
  config.add_target proc { |telemetry| puts telemetry.map { |k, v| "#{k}=#{v.inspect}" }.join(" ") }

This gems comes together with middleware for measuring request queue time, which will be reported in request.env and published to given statsd client.

Example configuration:

# in Gemfile add `require` part
gem "puma-plugin-telemetry", require: ["rack/request_queue_time_middleware"]

# in initializer, i.e. `request_queue_time.rb`
Rails.application.config.middleware.insert_after(
  0,
  RequestQueueTimeMiddleware,
  statsd: Datadog::Statsd.new(namespace: "ruby.puma", tags: { "app" => "accounts" })
)

Rails.application.config.log_tags ||= {}
Rails.application.config.log_tags[:queue_time] = ->(req) { req.env[::RequestQueueTimeMiddleware::ENV_KEY] }

This will provide proper metric in Datadog and in logs as well. Logs can be transformed into log metrics and used for auto scaling purposes.

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install.

All gem releases are manual, in order to create a new release follow:

1. Create new PR (this could be included in feature PR, if it's meant to be released)
   • update VERSION, we use Semantic Versioning
   • update CHANGELOG
   • merge
2. Draft new release via Github Releases
   • use v#{VERSION} as a tag, i.e. v0.1.0
   • add release notes based on the Changelog
   • create
3. Gem will get automatically published to given rubygems server

Bug reports and pull requests are welcome on GitHub at https://github.com/babbel/puma-plugin-telemetry.

The gem is available as open source under the terms of the MIT License.

Everyone interacting in the puma-plugin-telemetry project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.