Bundler::Multilock

Extends Bundler to allow arbitrarily many lockfiles (and Gemfiles!) for variations of the Gemfile, while keeping all of the lockfiles in sync.

bundle install, bundle lock, and bundle update will operate only on the default lockfile (Gemfile.lock), afterwhich all other lockfiles will be re-created based on this default lockfile. Additional lockfiles can be based on the same Gemfile, but vary at runtime. You can force a specific lockfile by setting the BUNDLE_LOCKFILE environment variable, or customize it any way you want by setting active: true on one of your lockfiles in your Gemfile.

Alternately (or in addition!), you can define a lockfile to use a completely different Gemfile. This will have the effect that common dependencies between the two Gemfiles will stay locked to the same version in each lockfile.

A lockfile definition can opt in to requiring explicit pinning for any dependency that exists in that variation, but does not exist in the default lockfile. This is especially useful if for some reason a given lockfile will not be committed to version control (such as a variation that will include private gems).

Finally, bundle check will enforce additional checks to compare the final locked versions of dependencies between the various lockfiles to ensure they end up the same. This check might be tripped if Gemfile variations (accidentally!) have conflicting version constraints on a dependency, that are still self-consistent with that single Gemfile variation. bundle install, bundle lock, and bundle update will also verify these additional checks. You can additionally explicitly allow version variations between explicit dependencies (and their sub-dependencies), for cases where the lockfile variation is specifically to transition to a new version of a dependency (like a Rails upgrade).

Bundler::Multilock 2.0 makes a number of breaking changes:

• The lockfile name parameter to lockfile must be a full filename, and not a shorthand. "Gemfile" and ".lock" will not be added to the parameter any more. This reduces confusion with the new lockfile method in Bundler 4.0. BUNDLE_LOCKFILE=active is still supported to select which lockfile is declared with active: true.
• lockfile(default:) is removed; use lockfile(active:) instead.
• lockfile(allow_mismatched_dependencies:) is removed with no replacement.

Bundler::Multilock 2.0 requires Bundler 4.1 or newer. Use version 1.x with older versions of Bundler.

Install the gem and add to the Gemfile by executing:

bundle plugin install bundler-multilock

Add additional lockfiles to your Gemfile like so:

# frozen_string_literal: true

source "https://rubygems.org"

plugin "bundler-multilock", "~> 2.0"
return unless Plugin.loaded?("bundler-multilock")

lockfile "Gemfile.rails-6.1.lock" do
  gem "rails", "~> 6.1"
end

lockfile "Gemfile.rails-7.0.lock" do
  gem "rails", "~> 7.0"
end

And then run bundle install. This will automatically generate additional lockfiles: Gemfile.rails-6.1.lock and Gemfile.rails-7.0.lock. Note that the default lockfile (Gemfile.lock, or set by lockfile without a block) will not contain any gems that are in the specific lockfile blocks. If you have gems you want in the default lockfile, but not in other lockfiles, you can define a block for just that lockfile:

lockfile do
  gem "rake"
end

When running other commands (such as tests), you select the desired lockfile with BUNDLE_LOCKFILE:

BUNDLE_LOCKFILE=Gemfile.rails-7.0.lock bundle exec rspec

You can also dynamically select it in your Gemfile, and pass active: true to (exactly one!) lockfile method.

A lockfile can also use a completely separate Gemfile:

lockfile "other/Gemfile.lock", gemfile: "other/Gemfile"

That Gemfile may also include the same preamble at the top, so that it loads the plugin when it's used directly (for example, with BUNDLE_GEMFILE=other/Gemfile):

plugin "bundler-multilock", "~> 2.0"
return unless Plugin.loaded?("bundler-multilock")

In some cases, you may want to essentially disable bundler-multilock's syncing behavior, while still allowing the Gemfile to select the active lockfile. For example, if you have gems in the default lockfile that cannot be installed under a certain Ruby version, but still want to be able to CI against that Ruby version, you may set it up like this:

lockfile active: RUBY_VERSION >= "2.7" do
  gem "debug", "~> 1.9"
end

lockfile "Gemfile.ruby-2.6.lock", active: RUBY_VERSION < "2.7" do
  gem "debug", "~> 1.8.0"
end

However, you cannot run a regular bundle install while running Ruby 2.6 in this situation, since simply evaluating the default lockfile's block, even for checking that the lockfiles are valid, will raise an exception that debug 1.9.0 require Ruby 2.7. You could set BUNDLE_LOCKFILE=Gemfile.ruby-2.6.lock to have bundler-multilock only consider the one lockfile, but then your CI may need additional logic to not set it for other Ruby versions. Instead, you can set BUNDLE_LOCKFILE=active, which will not override the Gemfile's selection of which lockfile is active, but still behave as if BUNDLE_LOCKFILE is set, bypassing any other syncing logic and not evaluating the default lockfile in our example.

bundle cache packages the gems from all of your lockfiles into vendor/cache (including any version differences between them), so that deployments and CI can install without network access. bundle install does the same when vendor/cache already exists. Gems that none of the lockfiles use anymore are pruned, the same as Bundler does for a single lockfile. This applies even when an alternate lockfile is flagged with active: true.

Run bundle install first, so that the gems from every lockfile are installed; bundle cache only copies them from there. Bundler's own settings apply to every lockfile: cache_all (on by default in Bundler 4) controls whether path and git gems are packaged, and cache_all_platforms (or --all-platforms) controls whether gems for other platforms are. Neither is set for you.

Setting BUNDLE_LOCKFILE limits bundle cache to that one lockfile, and it will prune gems that only other lockfiles use. If you do need to cache lockfiles individually, set bundle config set --local keep_outdated_cache true to keep what's already there.

bundle clean (and bundle install with the clean setting) works the same way: it keeps the installed gems from all of your lockfiles, and removes the ones that none of them use. With BUNDLE_LOCKFILE set, it keeps only that lockfile's gems. Bundler's prune setting also covers the git gems from all of your lockfiles.

Appraisal is a gem that might serve a similar purpose, but with a very different implementation. Appraisal is not a Bundler plugin, and instead works as a separate utility automatically generating additional gemfiles, based on the primary gemfile. It has no concern for lockfiles at all, and any steps to ensure the gemfiles themselves stay in sync are manual. Bundler::Multilock, in contrast, is not a separate file, does not generate additional gemfiles, automatically keeps the additional lockfiles synchronized anytime you run bundle install or bundle update, and ensures that all common dependencies stay in sync in the lockfile themselves between the lockfiles. It also supports relatively orthogonal scenarios, such as keeping dependencies in lockfiles for multiple gems in sync.

First, remove appraisal from your Gemfile or gemspec, then install Bundler::Multilock as above. Then move the contents of your Appraisals file into your Gemfile, just below the the newly added lines from Bundler::Multilock. Change the method from appraise to lockfile. Assuming you've committed all of your lockfiles, move them from gemfiles/*.gemfile.lock to Gemfile.*.lock next to the main Gemfile. Then run bundle install to ensure everything is happy. Be sure to update any tooling that uses BUNDLE_GEMFILE to use BUNDLE_LOCKFILE (with the appropriate changes to its value), and remove any appraisal install steps, since they're now redundant. Bundler::Multilock doesn't have a separate executable to repeat a command for each lockfile, so you'll need to handle that yourself if you're using something like appraisal bundle exec rspec.

Bootboot is a similar Bundler plugin that allows multiple lockfiles. It generally has a much narrower scope than Bundler::Multilock, though. Bootboot only allows two lockfiles, with a fixed name for the alternate lockfile, while Bundler::Multilock allows an arbitrary number with arbitrary names. Bootboots synchronization method for the alternate lockfile is relatively naive - essentially doing the equivalent of bundle update --all on the other lockfile, instead of propagating only the updated dependencies they have in common. Bundler::Multilock also has robust mechanisms to ensure that dependencies don't end up being mismatched outside of the normal workflow. Bootboot uses the DEPENDENCIES_NEXT environment variable to select the lockfile, while Bundler::Multilock uses BUNDLER_LOCKFILE to select a specific lockfile, as well as allowing the Gemfile itself to dynamically select a specific lockfile using any means it would like. Bundler::Multilock also features keeping lockfiles in sync amongst otherwise unrelated Gemfiles, such as if you have vendored gems in your repository. One feature that differs significantly is Bootboot supports varying Ruby versions between lockfiles, and ensuring that only compatible gems are installed as appropriate. Bundler::Multilock does not yet handle this, and requires the Ruby version to match among all lockfiles.

First, remove Bootboot's plugin bootstrapping from your Gemfile, then install Bundler::Multilock as above. Alter your Gemfile as appropriate, similar to this:

# original Gemfile

if ENV['DEPENDENCIES_NEXT']
  gem "rails", "~> 5.2.0"
else
  gem "rails", "~> 5.1.0"
end

# updated Gemfile

lockfile "Gemfile_next.lock" do
  gem "rails", "~> 5.2.0"
end

lockfile do
  gem "rails", "~> 5.1.0"
end

Be sure to update any tooling that uses DEPENDENCIES_NEXT to use BUNDLE_LOCKFILE (with the appropriate changes to its value).