Marlowe, Your Request Sleuth¶ ↑

code: github.com/KineticCafe/marlowe/
issues: github.com/KineticCafe/marlowe/issues
docs: www.rubydoc.info/github/KineticCafe/marlowe/master
continuous integration

Description¶ ↑

Marlowe is a Rack middleware that extracts or creates a request ID using a pre-defined header, permitting request correlation across multiple services.

When using Rails, Marlowe automatically adds itself to the middleware before Rails::Rack::Logger.

Upgrading from Marlowe 1.x¶ ↑

In Marlowe 1.0, the correlation header was called Correlation-Id; since then, Rails 5 and other tools and frameworks (such as Phoenix) have standardized on the header X-Request-Id. Marlowe 2.0 changes to this header by default.

To keep complete compatibility with Marlowe 1.0, the following should be used:

# Rails
config.marlowe_header = 'Correlation-Id'
config.marlowe_handler = :simple
config.marlowe_return = false

# Rack
use Marlowe::Middleware, header: 'Correlation-Id', handler: :simple, return: false

Configuration¶ ↑

Marlowe has three main configuration options: the request ID header, the request ID handler, and the request ID return. The options may be provided to the Rack use command as a keyword option, or set in a corresponding marlowe_option configuration variable in Rails.

Request ID Header¶ ↑

Specifies the header to be used for the request correlation ID. Defaults to X-Request-Id.

# Rails
config.marlowe_header = 'Correlation-Id'
# OR: config.marlowe_correlation_header = 'Correlation-Id'

# Rack
use Marlowe::Middleware, header: 'Correlation-Id'
# OR: use Marlowe::Middleware, correlation_header: 'Correlation-Id'

Marlowe will convert this to an appropriate HTTP header (in the Rack env parameter, the above header would be represented as env['HTTP_CORRELATION_ID']).

Request ID Handler¶ ↑

Specifies the method for sanitizing or generating the request correlation ID. Values can be :clean (the default, which limits incoming correlation IDs to 255 alphanumeric-or-dash characters), :simple (does not limit incoming correlation IDs), or a proc to transform or generate a correlation ID.

In all cases, if a correlation request ID is not handled, a UUID will be generated.

# Rails
config.marlowe_handler = :simple
config.marlowe_handler = ->(req_id) {
  req_id.try(:reverse) || SecureRandom.uuid
}

# Rack
use Marlowe::Middleware, handler: :simple
use Marlowe::Middleware, handler: ->(req_id) {
  req_id.try(:reverse) || SecureRandom.uuid
}

Request ID Return¶ ↑

If true (the default), the request correlation ID will be returned to the client in the same header that it was provided in.

# Rails
config.marlowe_return = false

# Rack
use Marlowe::Middleware, return: false

Using Marlowe with Rails 5¶ ↑

Rails 5 includes the ActionDispatch::RequestId middleware, reducing the need for Marlowe. Marlowe is more configurable than the Rails 5 default, so set marlowe_replace_action_dispatch_request_id to true to have Marlowe::Middleware will replace ActionDispatch::RequestId:

# Rails only
config.marlowe_replace_action_dispatch_request_id = true

Accessing the Correlation ID¶ ↑

The correlation id can be accessed throughout the application by accessing the RequestStore storage.

RequestStore[:correlation_id]

Logging¶ ↑

For a Rails application, you simply need to change the log formatter to one of the provided ones. Correlated versions of both the SimpleFormatter and Formatter are included.

# config/environments/development.rb
Rails.application.configure do
  config.log_formatter = Marlowe::SimpleFormatter.new
end

To create your own formatter, you'll need to access the RequestStore storage. You can use this pattern if you've rolled your own logger/formatter:

# lib/correlated_formatter.rb
require 'request_store'

class CorrelatedSimpleFormatter < ActiveSupport::Logger::SimpleFormatter
  def call(severity, timestamp, progname, msg)
    "[#{RequestStore.store[:correlation_id]}] #{super}"
  end
end

Lograge¶ ↑

As lograge supplies its own formatter, you will need to do something a little different:

# config/application.rb

class Application < Rails::Application
  config.before_initialize do
    ...
    # use lograge for all request logs
    config.lograge.enabled = true
    config.lograge.custom_options = lambda do |event|
      { correlation_id: RequestStore[:correlation_id] }
    end
  end
end

Clients¶ ↑

Catching and creating the correlation ID is a great all on its own, but to really take advantage of the correlation in a service based architecture you'll need to pass the request ID to the next service in the change.

Here's an example of a Hurley client:

# lib/correlated_client.rb

require 'hurley'
require 'request_store'

class Hurley::CorrelatedClient < Hurley::Client
  def initialize(*args, &block)
    super
    header['X-Request-Id'] = ::RequestStore.store[:correlation_id]
  end
end

If you have long-lived Hurley clients, it is also possible to use the Hurley callback machanism to add the outgoing headers:

client.before_call do |request|
  request.header['X-Request-Id'] = ::RequestStore.store[:correlation_id]
end

class Correlator
  def name
    :correlator
  end

  def call(request)
    request.header['X-Request-Id'] = ::RequestStore.store[:correlation_id]
  end
end

client.before_call(Correlator.new)

Install¶ ↑

Add Marlowe to your Gemfile:

gem 'marlowe', '~> 2.0'

Or manually install:

$ gem install marlowe

Marlowe Semantic Versioning¶ ↑

Marlowe uses a Semantic Versioning scheme with one significant change:

When PATCH is zero (0), it will be omitted from version references.

Additionally, the major version will generally be reserved for plug-in infrastructure changes.

Community and Contributing¶ ↑

Marlowe welcomes your contributions as described in Contributing.md. This project, like all Kinetic Cafe open source projects, is under the Kinetic Cafe Open Source Code of Conduct.

marlowe

Development

Runtime