Project

prosopite

0.27
There's a lot of open issues
N+1 auto-detection for Rails with zero false positives / false negatives
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
 Dependencies

Development

 Project Readme

Prosopite CI Gem Version

Prosopite is able to auto-detect Rails N+1 queries with zero false positives / false negatives.

N+1 queries detected:
  SELECT `users`.* FROM `users` WHERE `users`.`id` = 20 LIMIT 1
  SELECT `users`.* FROM `users` WHERE `users`.`id` = 21 LIMIT 1
  SELECT `users`.* FROM `users` WHERE `users`.`id` = 22 LIMIT 1
  SELECT `users`.* FROM `users` WHERE `users`.`id` = 23 LIMIT 1
  SELECT `users`.* FROM `users` WHERE `users`.`id` = 24 LIMIT 1
Call stack:
  app/controllers/thank_you_controller.rb:4:in `block in index'
  app/controllers/thank_you_controller.rb:3:in `each'
  app/controllers/thank_you_controller.rb:3:in `index':
  app/controllers/application_controller.rb:8:in `block in <class:ApplicationController>'

The need for prosopite emerged after dealing with various false positives / negatives using the bullet gem.

Compared to Bullet

Prosopite can auto-detect the following extra cases of N+1 queries:

N+1 queries after record creations (usually in tests)

FactoryBot.create_list(:leg, 10)

Leg.last(10).each do |l|
  l.chair
end

Not triggered by ActiveRecord associations

Leg.last(4).each do |l|
  Chair.find(l.chair_id)
end

First/last/pluck of collection associations

Chair.last(20).each do |c|
  c.legs.first
  c.legs.last
  c.legs.pluck(:id)
end

Changing the ActiveRecord class with #becomes

Chair.last(20).map{ |c| c.becomes(ArmChair) }.each do |ac|
  ac.legs.map(&:id)
end

Mongoid models calling ActiveRecord

class Leg::Design
  include Mongoid::Document
  ...
  field :cid, as: :chair_id, type: Integer
  ...
  def chair
    @chair ||= Chair.where(id: chair_id).first!
  end
end

Leg::Design.last(20) do |l|
  l.chair
end

Why a new gem

Creating a new gem makes more sense since bullet's core mechanism is completely different from prosopite's.

How it works

Prosopite monitors all SQL queries using the Active Support instrumentation and looks for the following pattern which is present in all N+1 query cases:

More than one queries have the same call stack and the same query fingerprint.

Installation

Add this line to your application's Gemfile:

gem 'prosopite'

If you're not using MySQL/MariaDB, you should also add:

gem 'pg_query'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install prosopite

Configuration

The preferred type of notifications can be configured with:

  • Prosopite.rails_logger = true: Send warnings to the Rails log
  • Prosopite.prosopite_logger = true: Send warnings to log/prosopite.log
  • Prosopite.stderr_logger = true: Send warnings to STDERR
  • Prosopite.raise = true: Raise warnings as exceptions

Development Environment Usage

Prosopite auto-detection can be enabled on all controllers:

class ApplicationController < ActionController::Base
  unless Rails.env.production?
    around_action :n_plus_one_detection
    
    def n_plus_one_detection
      Prosopite.scan
      yield
    ensure
      Prosopite.finish
    end
  end
end

And the preferred notification channel should be configured:

# config/environments/development.rb

config.after_initialize do
  Prosopite.rails_logger = true
end

Test Environment Usage

Tests with N+1 queries can be configured to fail with:

# config/environments/test.rb

config.after_initialize do
  Prosopite.rails_logger = true
  Prosopite.raise = true
end

And each test can be scanned with:

# spec/spec_helper.rb

config.before(:each) do
  Prosopite.scan
end

config.after(:each) do
  Prosopite.finish
end

WARNING: scan/finish should run before/after each test and NOT before/after the whole suite.

Allow list

Ignore notifications for call stacks containing one or more substrings:

Prosopite.allow_stack_paths = ['substring_in_call_stack']

Ignore notifications matching a specific SQL query:

Prosopite.ignore_queries = [/regex_match/, "SELECT * from EXACT_STRING_MATCH"]

Scanning code outside controllers or tests

All you have to do is to wrap the code with:

Prosopite.scan
<code to scan>
Prosopite.finish

or

Prosopite.scan do
<code to scan>
end

Pausing and resuming scans

Scans can be paused:

Prosopite.scan
# <code to scan>
Prosopite.pause
# <code that has n+1s>
Prosopite.resume
# <code to scan>
Prosopite.finish

An example of when you might use this is if you are testing Active Jobs inline, and don't want to run Prosopite on background job code, just foreground app code. In that case you could write an Active Job callback that pauses the scan while the job is running.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/charkost/prosopite.

License

Prosopite is licensed under the Apache License, Version 2.0. See LICENSE.txt for the full license text.