TrxExt

Extends functionality of ActiveRecord's transaction to auto-retry failed SQL transaction in case of deadlock, serialization error or unique constraint error. It also allows you to define on_complete callback that is being executed after SQL transaction is finished(either COMMIT-ed or ROLLBACK-ed). The implementation is not bound to any database, but relies on the rails connection adapters instead. Thus, if your database is supported by rails out of the box, then the gem's features will just work. Currently supported adapters:

• postgresql
• mysql2
• sqlite3
• trilogy

WARNING!

Because the implementation of this gem patches ActiveRecord - carefully test its integration into your project. For example, if your application patches ActiveRecord or if some of your gems patches ActiveRecord - there might be conflicts in the implementation which could potentially lead to the data loss.

Currently, the implementation is tested for rails v7.1+. If you need the support of rails v6.0, v6.1, v7.0 - please use v1.x of this gem, but it works with PostgreSQL only.

• ActiveRecord 7.1+
• Ruby 3

Add this line to your application's Gemfile:

gem 'trx_ext'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install trx_ext

require 'trx_ext'

# Object#trx is a shorthand of ActiveRecord::Base.transaction
trx do
  DummyRecord.first || DummyRecord.create
end

trx do
  DummyRecord.first || DummyRecord.create
  trx do |c|
    c.on_complete { puts "This message will be printed after COMMIT statement." }
  end  
end

trx do
  DummyRecord.first || DummyRecord.create
  trx do |c|
    c.on_complete { puts "This message will be printed after ROLLBACK statement." }    
  end
  raise ActiveRecord::Rollback
end

class DummyRecord
  # Wrap method in transaction
  wrap_in_trx def some_method_with_quieries
    DummyRecord.first || DummyRecord.create    
  end
end

If you are using non-primary connection for your model - you have to explicitly call trx method over that class:

DummyRecord.trx do
  DummyRecord.first || DummyRecord.create
end

In general, you should know about this if you are using multi-databases configuration.

If you want to wrap some method into a transaction using wrap_in_trx outside the ActiveRecord model context, you can pass a model name as a second argument explicitly:

class MyAwesomeLib
  # Wrap method in transaction
  def some_method_with_quieries
    DummyRecord.first || DummyRecord.create    
  end
  wrap_in_trx :some_method_with_quieries, 'DummyRecord'
end

TrxExt.configure do |c|
  # Number of retries before failing when unique constraint error raises. Default is 5
  c.unique_retries = 5
end

When an ActiveRecord SQL query fails due to deadlock error, serialization error or unique constraint error - it is automatically retried. In case of ActiveRecord transaction - the block of code the AR transaction belongs to is re-executed, thus the transaction query is retried.

Don't put into a single transaction more than needed for integrity purposes.

There is "On complete" feature that allows you to define callbacks(blocks of code) that will be executed after transaction is complete. See On complete callbacks section bellow for the docs. See On complete callbacks integrity section bellow to be aware about different situations with them.

• Don't explicitly wrap queries.

  Bad

  trx { User.find_by(username: 'someusername') }

  Good

  User.find_by(username: 'someusername')

• Don't wrap multiple SELECT queries in a single transaction unless it is of vital importance (see epigraph).

  Bad

  trx do
    @author = User.first
    @posts = current_user.posts.load
  end

  BEGIN
  SELECT "users".* FROM "users" ...
  SELECT "posts".* FROM "posts" ...
  COMMIT

  Good

  @author = User.first
  @posts = current_user.posts.load

  -- TrxExt::Retry.with_retry_until_serialized {
  SELECT "users".* FROM "users" ...
  -- }
  -- TrxExt::Retry.with_retry_until_serialized {
  SELECT "posts".* FROM "posts" ...
  -- }

• Beware of AR::Relation lazy loading if it is necessary to have multiple SELECTs in a single transaction.

  Bad

  trx do
    @posts = Post.all
    @users = User.all
  end

  will result in no query.

  Good

  trx do
    @posts = Post.all.load
    @users = User.all.load
  end

  BEGIN
  SELECT "posts".* FROM "posts" ...
  SELECT "users".* FROM "users" ...
  COMMIT

• When performing UPDATE/INSERT queries that depend on record's state - reload that record in the beginning of trx block.

  Bad

  def initialize(user)
    @user = user  
  end
  
  def update_posts
    trx do
      @user.posts.update_all(banned: true) if @user.user_permission.admin?          
    end  
  end    

  BEGIN      
    UPDATE posts SET banned = TRUE WHERE posts.user_id IN (...)
  COMMIT

  Explanation

  It might not be obvious that this code depends on @user - UserPermission#admin? is used to detect whether Post#banned must be updated. However, it is accessed through @user and there is no guarantee that, when calling @user.user_permission, it was not already cached by either previous calls, upper by stack trace, or inside trx block on transaction retry. This is why it is mandatory to call @user.reload - to reset user's cache and the cache of user's relations.

  Good

  def initialize(user)
    @user = user  
  end
  
  def update_posts
    trx do
      @user.reload
      @user.posts.update_all(banned: true) if @user.user_permission.admin?
    end  
  end    

  BEGIN
    SELECT * FROM users WHERE users.id = ...
    SELECT * FROM user_permissions WHERE user_permissions.user_id = ...
    UPDATE posts SET banned = TRUE WHERE posts.id IN (...)
  COMMIT

• It may happen that you need to invoke mailer's method inside trx block and pass there values that are calculated within the transaction block. Normally, you need to extract those values into after-transaction code and invoke mailer after transaction's end. Use on_complete callback to simplify your code:

  Bad

  trx do
    user = User.find_or_initialize_by(email: email)
    if user.save
      # May be invoked more than one time if transaction is retried        
      Mailer.registration_confirmation(user.id).deliver_later 
    end
  end  

  Good (before refactoring)

  user = nil
  result = 
    trx do
      user = User.find_or_initialize_by(email: email)
      user.save
    end
  Mailer.registration_confirmation(user.id).deliver_later if result

  Good (after refactoring)

  trx do |c|
    user = User.find_or_initialize_by(email: email)
    if user.save
      c.on_complete { Mailer.registration_confirmation(user.id).deliver_later }   
    end
  end

• Always keep in mind, that retrying of transactions is just re-execution of ruby's block of code on transaction retry. If you have any variables, that are changing inside the block - ensure that their values are reset in the beginning of block. Don't use methods that will raise error if called more than twice.

  Bad

  resurrected_users_count = 0
  trx do
    User.deleted.find_each do |user|
      if user.created_at > 2.days.ago
        user.active!
        resurrected_users_count += 1      
      end
    end
  end
  puts resurrected_users_count    

  Good

  resurrected_users_count = nil
  trx do
    resurrected_users_count = 0
    User.deleted.find_each do |user|
      if user.created_at > 2.days.ago
        user.active!
        resurrected_users_count += 1      
      end
    end
  end
  puts resurrected_users_count

  Bad

  class UsersController
    def update
      # This may raise AbstractController::DoubleRenderError if either redirect or render invoked twice
      trx do
        if @user.update(user_params)
          redirect_to @user
        else
          render :edit                              
        end
      end
    end
  end

  Bad

  class UsersController
    # This may raise AbstractController::DoubleRenderError if either redirect or render invoked twice
    wrap_in_trx def update
      if @user.update(user_params)
        redirect_to @user
      else
        render :edit                              
      end
    end
  end

  Good

  class UsersController
    def update
      if @user.update(user_params)
        redirect_to @user
      else
        render :edit                              
      end
    end
  end

  Good

  class UsersController
    def update
      trx do |c|
        if @user.update(user_params)
          c.on_complete { redirect_to @user }
        else
          c.on_complete { render :edit }                              
        end
      end
    end
  end

• Carefully implement the code that is related to the non-relational databases like Redis or MongoDB

  Bad

  trx do
    @post.reload
    if @post.tags_arr.include?('special')
      @post.update_columns(special: true)
      @post.mongo_post.update(special: true)
    end  
  end

  Explanation

  Example: @post.tags_arr.include?('special') == true and, as a result, @post.mongo_post.update(special: true) is executed but transaction is failed to be serialized. On second try - @post.tags_arr.include?('special') becomes false but the value of MongoPost#special was already changed

  Good

  trx do
    @post.reload
    if @post.tags_arr.include?('special')
      @post.update_columns(special: true)
    end
    @post.mongo_post.update(special: @post.tags_arr.include?('special'))        
  end

• Don't explicitly use return in the transaction's block of code. It may affect on how the transaction is going to be finished. Currently, it finishes with COMPLETE statement, but in the future versions it may change - according to the warning message, the behaviour may change soon.

  Bad

  def some_method
    trx do 
      return if User.where(email: email).exists?
      
      User.create(email: email)
    end
  end

  Bad

  def some_method
    trx do |c|
      user = User.find_by(email: email)
      return user if user
      
      user = User.create(email: email)
      c.on_complete { Mailer.registration_confirmation(user.id).deliver_later }
    end
  end

  Explanation

  Using return in the Proc(a block of code is a Proc) will return from the stack call instead the return from the block of code. Example:

  def some_method
    puts "Start"
    yield
    puts "End"      
  end
  
  def another_method
    some_method do
      puts "Hi"
      return
    end
  end
  

  Calling #another_method will output Start and Hi string, End string will never get output. Refer to official docs for more info.

  Good

  def some_method
    trx do 
      unless User.where(email: email).exists?
        User.create(email: email)
      end
    end
  end

  Good

  wrap_in_trx def some_method
    return if User.where(email: email).exists?
      
    User.create(email: email)     
  end

  Good

  wrap_in_trx def some_method
    user = User.find_by(email: email)
    return user if user
    
    user = User.create(email: email)
    trx { |c| c.on_complete { Mailer.registration_confirmation(user.id).deliver_later } }
  end

On-complete callbacks are defined with TrxExt::CallbackPool#on_complete method. An instance of TrxExt::CallbackPool is passed in each transaction block. You may add as much on-complete callbacks as you want by calling TrxExt::CallbackPool#on_complete several times - they will be executed in the order you define them(FIFO principle). The on-complete callbacks from nested transactions will be executed from the most deep to the most top transaction. Another words, if top transaction defines <#TrxExt::CallbackPool 0x1> instance and nested transaction defines <#TrxExt::CallbackPool 0x2> instance then, when executing on-complete callbacks - the callbacks of <#TrxExt::CallbackPool 0x2> instance will be executed first(FILO principle).

Example:

ActiveRecord::Base.transaction do |c1|
  User.first
  c1.on_complete { puts "This is 3rd message" }
  ActiveRecord::Base.transaction do |c2|
    User.last
    c2.on_complete { puts "This is 2nd message" }
    ActiveRecord::Base.transaction do |c3|
      c3.on_complete { puts "This is 1st message" }
      User.first(2)
    end
  end
  c1.on_complete { puts "This is 4th message" }
end

If you don't need to define on-complete callbacks - you may skip explicit definition of block's argument.

Example:

ActiveRecord::Base.transaction { User.first }

Keep in mind, that all on-complete callbacks are not a part of the transaction. If you want to make it transactional - you need to wrap it in another transaction.

Example:

ActiveRecord::Base.transaction do |c1|
  User.first
  c1.on_complete do
    ActiveRecord::Base.transaction do
      User.find_or_create_by(email: email)
    end
  end
end

You may define on-complete callbacks inside another on-complete callbacks. You may define another transactions in on-complete callbacks. Just don't get confused in the order they are going to be executed.

Example:

ActiveRecord::Base.transaction do |c1|
  User.first
  c1.on_complete do
    puts "This line will be executed first"
    ActiveRecord::Base.transaction do |c2|
      User.last
      c2.on_complete do
        puts "This line will be executed second"
      end
    end
    puts "This line will be executed third"
  end
end

Also, please avoid usage of the callbacks that belong to one transaction in another transaction explicitly. This complicates the readability of the code.

Example:

ActiveRecord::Base.transaction do |c1|
  User.first
  c1.on_complete do
    ActiveRecord::Base.transaction do
      User.last
      c1.on_complete do
        puts "This will be executed at the time when parent transaction's on-complete callbacks are executed!"
      end
    end
  end
end

• Don't define callbacks blocks as lambdas unless you are 100% sure what you are doing. Lambda has a bit different behaviour comparing to Proc. Refer to ruby documentation.

• When defining a callback - make sure that it does not depend on transaction's integrity. Another words - define it in a way like it is a normal code implementation outside the transaction:

  Bad

  trx do |c|
    user = User.find(id)
    user.referrals.create(referral_attrs)
    c.on_complete do  
      Mailer.new_referral(
        user_id: user.id, total_referrals: user.referrals.count
      ).deliver_later 
    end
  end

  Explanation

  The example above introduces two issues:

  • on_complete callback does not depend on the result of user.referrals.create(referral_attrs). And it should - we only need to send the email only if referral is created. Solution - add the condition for the on_complete callback
  • the number of user's referrals user.referrals.count is calculated inside on_complete, but it should be calculated within the transaction. Solution - calculate referrals count in transaction, extract its value into local variable and use that variable in the on_complete callback

  Good

  trx do |c|
    user = User.find(id)
    referral = user.referrals.create(referral_attrs)
    if referral.persisted?
      total_referrals = user.referrals.count
      c.on_complete do 
        Mailer.new_referral(user_id: user.id, total_referrals: total_referrals).deliver_later
      end
    end
  end

You can make any method atomic by wrapping it into transaction using #wrap_in_trx. Example:

class ApplicationRecord < ActiveRecord::Base
  class << self
    wrap_in_trx :find_or_create_by
    wrap_in_trx :find_or_create_by!    
  end

  wrap_in_trx def some_method
    SomeRecord.first || SomeRecord.create
  end
end

• After checking out the repo, run bundle install to install dependencies.
• Run docker-compose using docker-compose up command - it starts necessary services
• Run next command to create dev and test databases:

bundle exec rails db:create db:migrate
RAILS_ENV=test bundle exec rails db:migrate

Now you can run bin/console for an interactive prompt that will allow you to experiment.

You can run tests for currently installed AR using rspec command. There is bin/test_all_ar_versions executable that allows you to run tests within all supported AR versions.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and the created tag, and push the .gem file to rubygems.org.

Bug reports and pull requests are welcome on GitHub at https://github.com/intale/trx_ext. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.

The gem is available as open source under the terms of the MIT License.

Everyone interacting in the TrxExt project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.