Isolator
Detect non-atomic interactions within DB transactions.
Examples:
# HTTP calls within transaction
User.transaction do
user = User.new(user_params)
user.save!
# HTTP API call
PaymentsService.charge!(user)
end
#=> raises Isolator::HTTPError
# background job
User.transaction do
user.update!(confirmed_at: Time.now)
UserMailer.successful_confirmation(user).deliver_later
end
#=> raises Isolator::BackgroundJobError
Of course, Isolator can detect implicit transactions too. Consider this pretty common bad practice–enqueueing background job from after_create
callback:
class Comment < ApplicationRecord
# the good way is to use after_create_commit
# (or not use callbacks at all)
after_create :notify_author
private
def notify_author
CommentMailer.comment_created(self).deliver_later
end
end
Comment.create(text: "Mars is watching you!")
#=> raises Isolator::BackgroundJobError
Isolator is supposed to be used in tests and on staging.
Installation
Add this line to your application's Gemfile:
# We suppose that Isolator is used in development and test
# environments.
group :development, :test do
gem "isolator"
end
# Or you can add it to Gemfile with `require: false`
# and require it manually in your code.
#
# This approach is useful when you want to use it in staging env too.
gem "isolator", require: false
Usage
Isolator is a plug-n-play tool, so, it begins to work right after required.
However, there are some potential caveats:
-
Isolator tries to detect the environment automatically and includes only necessary adapters. Thus the order of loading gems matters: make sure that
isolator
is required in the end (NOTE: in Rails, all adapters loaded after application initialization). -
Isolator does not distinguish framework-level adapters. For example,
:active_job
spy doesn't take into account which AJ adapter you use; if you are using a safe one (e.g.Que
) just disable the:active_job
adapter to avoid false negatives. You can do this by adding an initializer:require "active_job/base" Isolator.adapters.active_job.disable!
-
Isolator tries to detect the
test
environment and slightly change its behavior: first, it respect transactional tests; secondly, error raising is turned on by default (see below). -
Experimental multiple databases has been added in v0.7.0. Please, let us know if you encounter any issues.
Configuration
Isolator.configure do |config|
# Specify a custom logger to log offenses
config.logger = nil
# Raise exception on offense
config.raise_exceptions = false # true in test env
# Send notifications to uniform_notifier
config.send_notifications = false
# Customize backtrace filtering (provide a callable)
# By default, just takes the top-5 lines
config.backtrace_filter = ->(backtrace) { backtrace.take(5) }
# Define a custom ignorer class (must implement .prepare)
# uses a row number based list from the .isolator_todo.yml file
config.ignorer = Isolator::Ignorer
# Turn on/off raising exceptions for simultaneous transactions to different databases
config.disallow_cross_database_transactions = false
end
Isolator relies on uniform_notifier to send custom notifications.
NOTE: uniform_notifier
should be installed separately (i.e., added to Gemfile).
Callbacks
Isolator different callbacks so you can inject your own logic or build custom extensions.
# This callback is called when Isolator enters the "danger zone"—a within-transaction context
Isolator.before_isolate do
puts "Entering a database transaction. Be careful!"
end
# This callback is called when Isolator leaves the "danger zone"
Isolator.after_isolate do
puts "Leaving a database transaction. Everything is fine. Feel free to call slow HTTP APIs"
end
# This callback is called every time a new transaction is open (root or nested)
Isolator.on_transaction_open do |event|
puts "New transaction from #{event[:connection_id]}. " \
"Current depth: #{event[:depth]}"
end
# This callback is called every time a transaction is completed
Isolator.on_transaction_close do |event|
puts "Transaction completed from #{event[:connection_id]}. " \
"Current depth: #{event[:depth]}"
end
Transactional tests support
- Rails' baked-in use_transactional_tests
- database_cleaner gem. Make sure that you require isolator after database_cleaner.
Supported ORMs
-
ActiveRecord
>= 6.0 (see older versions of Isolator for previous versions) -
ROM::SQL
(only if Active Support instrumentation extension is loaded)
Adapters
Isolator has a bunch of built-in adapters:
-
:http
– built on top of Sniffer :active_job
:sidekiq
:resque
:resque_scheduler
:sucker_punch
:mailer
-
:webmock
– track mocked HTTP requests (unseen by Sniffer) in tests :action_cable
You can dynamically enable/disable adapters, e.g.:
# Disable HTTP adapter == do not spy on HTTP requests
Isolator.adapters.http.disable!
# Enable back
Isolator.adapters.http.enable!
For active_job
, be sure to first require "active_job/base"
.
Fix Offenses
For the actions that should be executed only after successful transaction commit (which is mostly always so), you can try to use the after_commit
callback from after_commit_everywhere gem (or use native AR callback in models if it's applicable).
Ignore Offenses
Since Isolator adapter is just a wrapper over original code, it may lead to false positives when there is another library patching the same behaviour. In that case you might want to ignore some offenses.
Consider an example: we use Sidekiq along with sidekiq-postpone
–gem that patches Sidekiq::Client#raw_push
and allows you to postpone jobs enqueueing (e.g. to enqueue everything when a transaction is commited–we don't want to raise exceptions in such situation).
To ignore offenses when sidekiq-postpone
is active, you can add an ignore proc
:
Isolator.adapters.sidekiq.ignore_if { Thread.current[:sidekiq_postpone] }
You can add as many ignores as you want, the offense is registered iff all of them return false.
Using with sidekiq/testing
If you require sidekiq/testing in your tests after isolator is required then it will blow away isolator's hooks, so you need to require isolator after requiring sidekiq/testing.
If you're using Rails and want to use isolator in development and staging, then here is a way to do this.
# Gemfile
gem "isolator", require: false # so it delays loading till after sidekiq/testing
# config/initializers/isolator.rb
require "sidekiq/testing" if Rails.env.test?
unless Rails.env.production? # so we get it in staging too
require "isolator"
Isolator.configure do |config|
config.send_notifications = true # ...
end
end
Using with legacy Rails codebases
If you already have a huge Rails project it can be tricky to turn Isolator on because you'll immediately get a lot of failed specs. If you want to fix detected issues one by one, you can list all of them in the special files .isolator_todo.yml
and .isolator_ignore.yml
in the following way:
sidekiq:
- app/models/user.rb:20
- app/models/sales/**/*.rb
You can ignore the same files in multiple adapters using YML aliases in the following way:
http_common: &http_common
- app/models/user.rb:20
http: *http_common
webmock: *http_common
All the exceptions raised in the listed lines will be ignored.
The .isolator_todo.yml
file is intended to point to the code that should be fixed later, and .isolator_ignore.yml
points to the code that for some reasons is not expected to be fixed. (See #40)
Using with legacy Ruby codebases
If you are not using Rails, you'll have to load ignores from file manually, using Isolator::Ignorer.prepare(path:)
, for instance Isolator::Ignorer.prepare(path: "./config/.isolator_todo.yml")
Custom Adapters
An adapter is just a combination of a method wrapper and lifecycle hooks.
Suppose that you have a class Danger
with a method #explode
, which is not safe to be run within a DB transaction. Then you can isolate it (i.e., register with Isolator):
# The first argument is a unique adapter id,
# you can use it later to enable/disable the adapter
#
# The second argument is the method owner and
# the third one is a method name.
Isolator.isolate :danger, Danger, :explode, options
# NOTE: if you want to isolate a class method, use singleton_class instead
Isolator.isolate :danger, Danger.singleton_class, :explode, options
Possible options
are:
-
exception_class
– an exception class to raise in case of offense -
exception_message
– custom exception message (could be specified without a class) -
details_message
– a block to generate additional exception message information:
Isolator.isolate :active_job,
target: ActiveJob::Base,
method_name: :enqueue,
exception_class: Isolator::BackgroundJobError,
details_message: ->(obj) {
"#{obj.class.name}(#{obj.arguments})"
}
Isolator.isolate :promoter,
target: UserPromoter,
method_name: :call,
details_message: ->(obj_, args, kwargs) {
# UserPromoter.call(user, role, by: nil)
user, role = args
by = kwargs[:by]
"#{user.name} promoted to #{role} by #{by&.name || "system"})"
}
Trying to register the same adapter name twice will raise an error. You can guard for it, or remove old adapters before in order to replace them.
unless Isolator.has_adapter?(:promoter)
Isolator.isolate(:promoter, *rest)
end
# Handle code reloading
class Messager
end
Isolator.remove_adapter(:messager)
Isolator.isolate(:messager, target: Messager, **rest)
You can also add some callbacks to be run before and after the transaction:
Isolator.before_isolate do
# right after we enter the transaction
end
Isolator.after_isolate do
# right after the transaction has been committed/rolled back
end
Troubleshooting
Verbose output
In most cases, turning on verbose output for Isolator helps to identify the issue. To do that, you can either specify ISOLATOR_DEBUG=true
environment variable or set Isolator.debug_enabled
manually.
Tests failing after upgrading to Rails 6.0.3 while using Combustion
The reason is that Rails started using a separate connection pool for advisory locks since 6.0.3. Since Combustion usually applies migrations for every test run, this pool becomse visible to test fixtures, which resulted in 2 transactional commits tracked by Isolator, which only expects one. That leads to false negatives.
To fix this disable migrations advisory locks by adding advisory_locks: false
to your database configuration in (spec|test)/internal/config/database.yml
.
Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/palkan/isolator.
License
The gem is available as open source under the terms of the MIT License.