Rails API Logger

The simplest way to log API requests of your Rails application in your database.

The Rails API logger gem introduces a set of tools to log and debug API requests. It works on two sides:

• Inbound requests: API exposed by your application
• Outbound requests: API invoked by your application

This gem has been extracted from various Renuo projects, where we implemented this technique multiple times successfully.

This gem creates two database tables to log the following information:

• path the path/url invoked
• method the method used to invoke the API (get, post, put, etc...)
• request_body what was included in the request body
• response_body what was included in the response body
• response_code the HTTP response code of the request
• started_at when the request started
• ended_at when the request finished

Add this line to your application's Gemfile:

gem 'rails_api_logger'

And then execute:

bundle install
spring stop # if it's running. otherwise it does not see the new generator 
bundle exec rails generate rails_api_logger:install
bundle exec rails db:migrate

This will generate two tables inbound_request_logs and outbound_request_logs. These tables will contain the logs.

Given an outbound request in the following format:

uri = URI('http://example.com/some_path?query=string')
http = Net::HTTP.start(uri.host, uri.port)
request = Net::HTTP::Get.new(uri)
response = http.request(request)

you can log it by doing the following:

uri = URI('http://example.com/some_path?query=string')
http = Net::HTTP.start(uri.host, uri.port)
request = Net::HTTP::Get.new(uri)

log = OutboundRequestLog.from_request(request)

response = http.request(request)

log.response_body = response.body
log.response_code = response.code
log.save!

You can also use the provided logger class to do that in a simpler and safer manner:

uri = URI('https://example.com/some_path')
request = Net::HTTP::Post.new(uri)
request.body = { answer: 42 }.to_json
request.content_type = 'application/json'

response = RailsApiLogger.new.call(nil, request) do
  Net::HTTP.start(uri.host, uri.port, use_ssl: true) { |http| http.request(request) }
end

This will guarantee that the log is always persisted, even in case of errors.

If you log your outbound requests inside of parent app transactions, your logs will not be persisted if the transaction is rolled-back. You can circumvent that by opening another database connection to the same (or another database if you're into that stuff) when logging.

# config/initializers/outbound_request_log_patch.rb

module OutboundRequestLogTransactionPatch
  extend ActiveSupport::Concern

  included do
    connects_to database: { writing: :primary, reading: :primary }
  end
end

OutboundRequestLog.include(OutboundRequestLogTransactionPatch)

You can also log the request in a separate thread to provoke the checkout of a separate database connection. Have a look at this example here.

If you are exposing some API you might be interested in logging the requests you receive. You can do so by adding this middleware in config/application.rb

config.middleware.insert_before Rails::Rack::Logger, InboundRequestsLoggerMiddleware

this will by default only log requests that have an impact in your system (POST, PUT, and PATCH calls). If you want to log all requests (also GET ones) use

config.middleware.insert_before Rails::Rack::Logger, InboundRequestsLoggerMiddleware, only_state_change: false

If you want to log only requests on a certain path, you can pass a regular expression:

config.middleware.insert_before Rails::Rack::Logger, InboundRequestsLoggerMiddleware, path_regexp: /api/

If you want to skip logging the body of certain requests, you can pass a regular expression:

config.middleware.insert_before Rails::Rack::Logger, InboundRequestsLoggerMiddleware, skip_body_regexp: /api/letters/

In the implementation of your API, you can call any time attach_inbound_request_loggable(model) to attach an already persisted model to the log record.

For example:

def create
  @user = User.new(user_params)
  if @user.save
    attach_inbound_request_loggable(@user)
    render json: { id: @user.id }, status: :created
  else
    render json: @user.errors.details, status: :unprocessable_entity
  end
end

in the User model you can define:

has_many :inbound_request_logs, inverse_of: :loggable, dependent: :destroy, as: :loggable

to be able to access the logs attached to the model.

We provide here some code samples to integrate the models in RailsAdmin.

This configuration will give you some nice views, and searches to work with the logs efficiently.

%w[InboundRequestLog OutboundRequestLog].each do |logging_model|
  config.model logging_model do
    list do
      filters %i[method path response_code request_body response_body created_at]
      scopes [nil, :failed]

      include_fields :method, :path, :response_code, :created_at

      field :request_body, :string do
        visible false
        searchable true
        filterable true
      end

      field :response_body, :string do
        visible false
        searchable true
        filterable true
      end
    end

    show do
      include_fields :loggable, :method, :path, :response_code
      field(:created_at)
      field(:request_body) do
        formatted_value { "<pre>#{JSON.pretty_generate(bindings[:object].request_body)}</pre>".html_safe }
      end
      field(:response_body) do
        formatted_value { "<pre>#{JSON.pretty_generate(bindings[:object].response_body)}</pre>".html_safe }
      end
    end
  end
end

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Bug reports and pull requests are welcome on GitHub at https://github.com/renuo/rails_api_logger. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.

The gem is available as open source under the terms of the MIT License.

Everyone interacting in the RailsApiLogger project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.