Mutineer Mongoid

mutineer-mongoid is a fork of Mutineer adding parallel mutation testing for Rails applications using Mongoid and RSpec. It retains upstream Minitest, RSpec, and SQLite functionality; Mongoid support is an addition, not a replacement for those backends. This fork is based on v1.6.0. It adds Mongoid 9 / RSpec execution with --rails --daemon --jobs N and keeps the mutineer executable and Ruby namespace. The original MIT license and author attribution are preserved. Development is maintained by Justin Allen Marsh.

Version 2.0.0 renames the gem, repository, and action to mutineer-mongoid. Replace mutineer-gaia in your Gemfile, run bundle install, and update action references to justinallenmarsh/mutineer-mongoid@v2.0.0. The command remains bundle exec mutineer; configuration and behavior are unchanged. The existing v1.7.0 tag and its mutineer-gaia-1.7.0.gem release asset remain available. GAIA is the application where this integration was developed and tested; the fork contains no GAIA application code and can be used by other applications.

The new action release uses v2; v1 remains on the original release. GitHub does not redirect action calls after a repository rename: update the repository path in every workflow, including workflows pinned to v1.7.0 or a SHA.

Ruby 3.4+ and Linux or macOS are required for the Mutineer process. The --test-command backend can run suites under an older application Ruby. --matrix works with Minitest and RSpec on the serial in-process backend; it cannot be combined with --daemon or --test-command. The supported Mongoid isolation scope and other limitations are listed below.

Use this gem in place of mutineer or mutineer-gaia; install only one of them because they provide the same mutineer executable and Ruby namespace:

gem "mutineer-mongoid", "~> 2.0", require: false

bundle exec mutineer run app/models/example.rb --rails --daemon --jobs 2

Install from RubyGems using Bundler, or run gem install mutineer-mongoid for the standalone command. The identical gem and checksum are attached to the GitHub release.

The GitHub action builds and installs the source at its pinned action ref:

- uses: actions/checkout@v7
- uses: ruby/setup-ruby@v1
  with:
    ruby-version: "3.4"
    bundler-cache: true
- uses: justinallenmarsh/mutineer-mongoid@v2.0.0
  with:
    sources: app/models/example.rb
    test: spec/models/example_spec.rb
    framework: rspec
    rails: "true"
    daemon: "true"
    jobs: "2"
    output: mutation-report.json

Start MongoDB and configure the application's test client before this step. For example, a GitHub-hosted Linux job can provide mongo:8.2 through a service container exposing port 27017. Use use-bundler: "true" when the application's Gemfile includes this fork. The default standalone mode also supports a daemon booting a separate application bundle. Pin an immutable commit SHA instead of a tag when your dependency policy requires it.

Mongoid daemon runs capture the boot-time test database's documents, collection options, and indexes in memory. Each coverage check and mutant gets a fresh, randomly named database. The parent drops it after reaping the child, including on timeout or error. The source database is read only. Coverage caches include the seed snapshot. MongoDB connection failures inside examples or suite hooks are reported as errors, not caught mutations. Mongoid clients disconnect before forks and reconnect in children. Credentials must allow creation and deletion of the disposable databases. A hard kill of the parent can leave one behind; these have names matching mutineer_<8 hex source hash>_<16 hex run token>_<child pid>.

Supported scope: one default Mongoid client, ordinary collections, and RAILS_ENV=test. Multiple clients, views, time-series collections, and mixed ActiveRecord/Mongoid applications fail explicitly. Raw Mongo::Client instances outside Mongoid, cached database handles created during boot, and specs that reconfigure Mongoid are outside this isolation contract. Redis, object storage, and other external resources keep the application's existing isolation rules. --matrix remains available on the serial in-process backend. Daemon execution still reloads whole source files, with upstream's callback and concern limitations.

To check the MongoDB integration independently of an application:

BUNDLE_GEMFILE=test/fixtures/mongoid_app/Gemfile bundle install
# MongoDB must be running; set MUTINEER_MONGO_URL if it requires credentials.
bundle exec rake test:mongoid

The integration tests check concurrent writes, seed data, unique indexes, cleanup on timeout and error, source-data preservation, and serial/parallel report parity with both killed and surviving mutations. Mongoid is only in the fixture bundle; the gem still has zero runtime dependencies.

The following reference retains upstream behavior except where the compatibility table and Mongoid notes above describe this fork's additions.

A clean-room mutation-testing tool for Ruby. Mutineer mutates your source one change at a time, runs your test suite (Minitest or RSpec) against each mutant, and reports the ones your tests failed to catch — the gaps where your suite isn't actually testing anything.

"Clean-room" means Mutineer was written from scratch. It contains no code from Mutant or any other mutation-testing tool. Mutineer's own code is licensed under MIT. See Mutineer and Mutant for how the two tools differ.

• Prism + stdlib only — zero runtime dependencies (Ruby ≥ 3.4).
• One mutation per mutant, validity-checked by re-parsing.
• Fork-isolated, parallel execution (Linux + macOS).
• Coverage-guided — each mutant runs only the test files that cover its line.
• Stops at the first failing test — in-process runs (not --daemon or --test-command) stop a mutant's test run at the first failure, unless --matrix asks for every covering test. The source's paired test files run first, then the rest from fastest to slowest in doubling steps (under 1 s, 1 to 3 s, 3 to 7 s, and so on); files in one step run in path order.

📖 mutineer.github.io → — overview, operators, and usage.

mutineer run <source...> --test <test> [--test <test>...] [options]

Mutate lib/calculator.rb, checking it against its test, and fail CI if the mutation score drops below 90%:

mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90

Run mutineer --list-operators to see them. Default (Tier 1): arithmetic, comparison, boolean_connector, boolean_literal, statement_removal. Available but off by default (Tier 2, enable via --operators): return_nil, literal_mutation, condition_negation, string_literal, regex, collection_method, safe_navigation, range, negation_removal, chain_link, operand_removal, array_literal, condition_true, condition_false, operator_assignment.

Rails code needs its environment booted before the suite runs, so point Mutineer at your app with --rails and run it inside the project's bundle:

RAILS_ENV=test bundle exec mutineer run \
  app/models/order.rb --test test/models/order_test.rb --rails

--rails boots config/environment once in the parent process (every mutant then forks and inherits it), defaults --strategy to redefine without --daemon (surgical — it avoids reloading files into the app tree; --daemon keeps reload), and reconnects ActiveRecord in each fork so the database connection is fork-safe. Use --boot FILE to boot a different entry point. Boot mode requires at least one --test file and is coverage-guided — each mutant runs only the test files that exercise its line (coverage is captured by forking the booted app, then cached).

Some code runs while the app boots or a class loads, for example a class body that calls a method to build a constant, a to_prepare initializer, or a --require file that calls a source method. That run happens before the mutant is applied, and the forked test does not repeat it. A mutant on such a line is ran_at_load, not survived or no_coverage. It is left out of the score and does not fail --threshold; a kill still counts. This holds under both strategies: reload runs the mutated file's class body again, but not an initializer or another file's code, so a survivor on such a line is not trusted there either. Run those mutants with --test-command, which boots a fresh process per mutant, to get a verdict.

Add Mutineer to your Gemfile's test group:

gem "mutineer-mongoid", "~> 2.0", group: :test, require: false

--rails boots your app once but runs mutants serially — parallel --jobs under Rails is unsafe, because every worker shares one test database and clobbers the others' fixtures. --daemon fixes both: it boots the app once in a persistent helper and forks per mutant, and gives each parallel worker its own database, so --jobs N is safe and its verdicts are proven identical to a serial run.

RAILS_ENV=test bundle exec mutineer run \
  app/models/order.rb --test test/models/order_test.rb \
  --rails --daemon --jobs 4

• One boot, forked per mutant — restores the shared-boot speed.
• Coverage-guided — each mutant runs only its covering tests (like --rails); a mutant on an uncovered line is no_coverage, so the score stays comparable to the in-process --rails score.
• Safe --jobs N — each worker routes to its own copy of the test database, so parallel verdicts equal serial (no fixture cross-talk). On first use, a worker's database is a copy of the test database after the app boots, so rows written by initializers or --require files are there, as in-process. When the copy's schema differs from db/schema.rb (another schema version or schema_sha1 checksum, or no stored checksum, as in an empty or out-of-date test database, or one set up with a plain load "db/schema.rb"), the worker loads db/schema.rb over it, which drops the copied rows of the tables it defines.
• One backend at a time — --daemon can't be combined with --test-command (choose one), and it needs an app to boot (--rails or --boot).

ActiveRecord daemon status: SQLite with Minitest (hermetic, CI-proven). This fork also supports Mongoid 9 with RSpec as described above. Per-worker provisioning for Postgres and other adapters is not supported: on those, --daemon scores every mutant as error. Use --daemon with a SQLite test database, or drop --daemon to run serially on other adapters.

Mutineer's own process needs Ruby ≥ 3.4 (it parses with stdlib Prism), and the --rails path above boots your app inside Mutineer's process — so it can't run against an app pinned to an older Ruby (ruby "3.1.6" in the Gemfile), where the bundle rejects 3.4.

--test-command decouples the two: Mutineer stays on ≥ 3.4, but your suite runs as a subprocess in your app's own runtime (whatever Ruby its bundle resolves to). Run Mutineer with a 3.4+ Ruby and hand it the command that runs your tests:

RAILS_ENV=test mutineer run app/models/order.rb \
  --test test/models/order_test.rb \
  --test-command "bundle exec rails test %{files}"

• %{files} is required; it expands to the --test paths as separate arguments (a path with a space stays one argument — there is no shell).
• Environment: vars like RAILS_ENV / DATABASE_URL set on the Mutineer command are inherited. Mutineer unsets BUNDLE_*, GEM_*, RUBY*, RBENV_VERSION, ASDF_RUBY_VERSION, and RBENV_DIR in the child (so Mutineer's own Ruby cannot pin the suite), and drops version-manager version bins (e.g. ~/.rbenv/versions/3.4.x/bin) from PATH, then prepends rbenv/asdf shims when a pin was scrubbed. Do not rely on RBENV_VERSION=… on the Mutineer command for the suite; use .ruby-version or a wrapper. Don't put KEY=val prefixes inside --test-command (no shell; that would be treated as the program name).

Automatic scrub targets rbenv and asdf (shims + version bins). chruby has no shims: Mutineer still strips …/rubies/…/bin so it cannot leave Mutineer's Ruby pinned, but you need a wrapper that sources chruby and selects the app version. If the smoke check still reports a Ruby version mismatch, wrap the suite. Example rbenv wrapper (bin/mutineer-test in the app):

#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
unset GEM_HOME GEM_PATH RUBYLIB RUBYOPT BUNDLE_GEMFILE BUNDLE_BIN_PATH BUNDLER_VERSION
export RBENV_VERSION="$(cat .ruby-version 2>/dev/null || true)"
export RAILS_ENV="${RAILS_ENV:-test}"
export PATH="${HOME}/.rbenv/shims:${PATH}"
exec bundle exec rails test "$@"

# Mutineer on 3.4+; suite on the app's Ruby via the wrapper
mutineer run app/models/order.rb \
  --test test/models/order_test.rb \
  --test-command "bin/mutineer-test %{files}"

Mutineer also surfaces a targeted smoke-check message when Bundler prints RubyVersionMismatch, instead of only blaming DB/migrations.

Tradeoffs — this path is correct but not free:

• Slower: your app re-boots for every mutant (no shared boot yet).
• No coverage narrowing: every mutant runs the full --test set, so the score is an upper bound and not comparable to an in-process (--rails) score — uncovered mutants count as survivors, and an infrastructure failure is scored as a kill. Mutineer prints this caveat on every run and aborts up front (a "smoke check") if your unmutated suite isn't green.
• Reload strategy only (--strategy redefine is rejected on this path) and serial (--jobs is forced to 1). For apps on Ruby ≥ 3.4, --daemon gives safe parallelism instead (see the daemon backend).

The mutation score is killed / (killed + survived). A mutant whose tests run past --timeout is a timeout. It is neither killed nor survived, so it is left out of the score, like no_coverage, uncapturable, unplaceable, ran_at_load, errored, skipped and ignored mutants. A timeout is not counted as a kill, because a hang the mutant caused and a suite that is just slow look the same. The human report shows the count in its Timeout: row, and the JSON report in summary.timeout.

Under --threshold, a timeout is a mutant with no verdict. The run exits 1 when more than one mutant produced no verdict and they exceed 10% of those attempted, or when nothing could be scored and something broke. If a slow suite times out mutants that its tests would catch, raise --timeout. A limit set below what the suite needs does not raise the score: it turns those mutants into timeouts, and once they pass 10% of the attempted mutants the run fails.

Some mutants are equivalent (behaviour-identical) and survive forever — keeping a file off 100%. Suppress them so the score and --threshold gate stay meaningful:

• Inline: some_line # mutineer:disable-line (or scope it: # mutineer:disable-line comparison). Put a reason after --: # mutineer:disable-line comparison -- the test checks only 20.
• Config: a .mutineer.yml ignore: list of mutant ids. Each survivor's id is printed in the JSON report, so copy it straight into ignore:.

Suppressed mutants are excluded from the score (so 100% becomes reachable).

A mutant id is 12 hex characters. It hashes the file path (relative to the project root), the method's qualified name, the operator, the mutated code, and the mutant's position among identical mutants in that method. When one file has two methods with the same qualified name (for example two top-level def index in two DSL blocks), the second and later ones also hash their position among those methods, so their ids differ. The first one's id does not change. An edit outside the method does not change the id. Moving or renaming the file, renaming the method or its class, or adding an identical mutant earlier in the method does. Adding a method with the same name earlier in the same file also does.

• The project root is the directory mutineer runs from (in the Action, the working-directory). Run from the same root to get the same ids.
• A source outside the project root uses its absolute path, so its ids differ between machines.

Migrating from ids without the file path. Before 1.3, ids did not include the file path, so two files could share an id (#126). Old-format ids keep working in this fork, with a migration warning:

• ignore: An old entry still suppresses its mutants. The run prints the new ids for each old entry, each with its file and method. When it names one mutant, replace the entry with that id. When it names several (in different files, or same-named methods in one file), the old entry over-matched: it also hid mutants you did not mean to ignore. The warning says so. Keep only the ids for the mutant you meant to ignore, not all of them. The list covers only the sources and operators in that run, so run over every source with every operator set you use (for example your Tier-2 --operators) for the full list.
• --baseline An old baseline still matches: a survivor matches a stored one with the same old id in the same file. A stored file that is an absolute path outside the project root (a baseline written on another machine) matches on the old id alone. The run tells you to regenerate it. Regenerate it with --format json, but only after every gate that reads it runs 1.3 or later (the Action's version: pin, your CI Gemfile.lock). An older version treats every new-format survivor as new.

The JSON report's summary.id_format is 2 for the new format. summary.legacy_id_matches.ignore counts the old-format ignore entries a run matched, and summary.legacy_id_matches.baseline counts the survivors matched only through an old baseline id.

Ids are relative to the directory you run mutineer from. mutineer finds .mutineer.yml by walking up. When the file it loads is in a parent directory (other than your home directory), it warns that the ignore ids will not match and tells you which directory to run from.

--matrix reports which tests kill each mutant, and from that, which tests add nothing to the suite.

mutineer run lib/calculator.rb --test test/calculator_test.rb --matrix

For each mutant, Mutineer runs every test in its covering files instead of stopping at the first failure, and records which tests failed. Coverage is recorded per test file, so that set also holds tests that never reach the mutated line; they pass and count as having run. A mutant's row is complete when its whole covering set ran and the suite returned normally (under Minitest, an Interrupt that cut the run short does not count as returning normally). The report names two kinds of test:

• Blind: the test ran in at least one complete row and killed no mutant in any row. It would still pass with this run's mutations of the code, so it guards none of them. It can still check behavior outside this run's mutants (code in other files, or edits no operator makes), so look at what it asserts before you delete it, or give it an assertion that can fail.
• Redundant: every mutant the test kills, another test kills too. That makes it a candidate for deletion, one test at a time: two redundant tests can be the only killers of one mutant, so re-run after each deletion.

The answers cover this run's mutants only. A test of code outside the sources you passed kills nothing here, so mutate the code a test exercises before you call the test blind.

The matrix changes no verdict, score or exit code: each mutant gets the verdict a run without --matrix gives. That run skips every later test, test class and example group after the first failure, so when a later one exits the process, crashes or runs into the time limit, the mutant is still killed. The run without --matrix still runs the code around the failing test (the rest of its Minitest class run wrapper, the after(:all) hooks of its RSpec groups) and the suite's own cleanup (RSpec's after(:suite)), so an end there keeps the exit status. A failure in a Minitest parallelize_me! class also keeps it: those run after every serial class and cannot be stopped once queued, while a failure in a serial class still stops the run.

Under RSpec, --matrix needs RSpec 3.3 or later, whose example ids tell apart examples on one line. With an older RSpec every row is incomplete.

The human report lists up to 20 blind and 20 redundant tests, and the HTML report lists them all. --format json adds a matrix block with every test and each mutant's killers (see the JSON schema). A test is its file and its id. The name is CalculatorTest#test_add under Minitest, or the example's full description under RSpec, where the example id (./spec/calc_spec.rb[1:2]) tells apart examples that share a description and keeps an example whose generated description changes with the mutant as one test.

Each mutant runs its whole covering set, so every mutant costs what a survivor costs. In 1.4, stopping at the first failure cut a full run of rack's lib/rack/utils.rb from 86 to 89 seconds down to 35 to 41, so expect a matrix run to take 2.1 to 2.5 times as long as a normal one. A mutant that reaches the per-mutant time limit (--timeout, 10 seconds by default) after a test already failed stays killed with its row marked incomplete. The report names each incomplete row and warns that a blind test may have killed one of those mutants, so do not delete a blind test until its rows are complete. Raise --timeout (or timeout: in .mutineer.yml) to get complete rows.

--matrix runs on the in-process backend only. It exits 2 with --daemon, --test-command, --fail-fast or --dry-run, and the message says whether each setting came from the command line or .mutineer.yml. To run one of those with matrix: true in the file, pass --no-matrix; to run --matrix with fail_fast: true in the file, pass --no-fail-fast.

Store a JSON run as a baseline, then fail the build only when a PR makes things worse:

mutineer run app/ --baseline .mutineer/baseline.json   # exit 1 on NEW survivors or a score drop

--baseline reports which survivors are new (by mutant id) and any score drop. It combines with --threshold (the worse of the two sets the exit code). Pass a directory (or several sources) to audit a whole layer in one boot — tests are auto-paired by convention and the report breaks down per source. A source app/foo/bar.rb pairs with test/foo/bar_test.rb and with unclaimed test/foo/bar_*_test.rb files in that directory, such as bar_upsert_test.rb. It does not take user_session_test.rb when user_session.rb exists in the same directory. It does not take bar_upsert_guards_test.rb when bar_upsert.rb exists. A spec that already pairs is left as that one file.

This repo ships a composite action (action.yml) that wraps the CLI for CI:

- uses: actions/checkout@v4
- uses: ruby/setup-ruby@v1
  with: { ruby-version: "3.4", bundler-cache: true }
- uses: justinallenmarsh/mutineer-mongoid@v2
  with:
    sources: app/
    baseline: .mutineer/baseline.json
    threshold: "90"

Default change: on pull_request events (not pull_request_target) the action scopes the run to the PR's changed lines, diffing against the PR's exact base commit (fetched by the action itself when the checkout is shallow; falls back to the base branch tip). Pass since: none for a full scan, or an explicit since: ref (which needs fetch-depth: 0 on checkout).

With the default JSON format the action also:

• writes a score summary to the job's step summary;
• annotates surviving mutants on the PR diff, up to 50 (error level when the gate failed, warning when it passed);
• exposes the report path via the report output for later steps (with format: human/html this needs the output input).

The CLI prints a progress line to the log at every 10% of the run, whatever the format.

Mutineer is built for programmatic use — versioned JSON, mutant ids that survive unrelated edits, structured exit codes, and diff-scoped runs. See:

• AI agents & CI recipes — the agent inner-loop and CI-gate recipes (and how to avoid infinite loops on equivalent mutants): rendered · source
• JSON schema reference — the --format json schema and its versioning contract: rendered · source
• Ruby API (YARD) — class reference for the shipped gem: https://davidteren.github.io/mutineer/api/
• Agent skill (skills/mutineer/SKILL.md): a short card for coding agents with install, the agent loop, and exit codes. Install it with gh skill install justinallenmarsh/mutineer-mongoid mutineer or npx skills add justinallenmarsh/mutineer-mongoid --skill mutineer.

Mutineer reads an optional .mutineer.yml from the project root (nearest one, walking up). CLI flags override config; config overrides defaults.

Sources are positional CLI arguments and test files come from --test. The config file accepts these keys:

Invalid values for known scalar keys, and a blank operators list, exit 2 with a message naming the file and key. An unknown operator name warns and is skipped. If none of the names are known, the run exits 2. An empty require or ignore list is valid. Boolean keys take true or false (quoted forms also work), not "yes". jobs must be positive; a string value contains digits only. String values for threshold and the CLI-only --baseline-epsilon use plain decimals such as 90 or 0.5, not +2, 1e2, or 1_0. String options such as only and baseline cannot be null or boolean. A blank since is invalid; use since: false to turn scoping off. Unknown keys warn and are ignored. Unknown operator names warn and are skipped, and the run exits 2 when none remain. --operators replaces a blank or unknown file list.

format, strategy, output, baseline_epsilon, and dry_run are CLI-only. For JSON output, use --format json, not a format: config key. To select RSpec in the file, add framework: rspec.

# .mutineer.yml
operators: [arithmetic, comparison, boolean_connector, boolean_literal, statement_removal]
threshold: 90
jobs: 4
require:
  - config/environment

Coverage results are cached in .mutineer/coverage.json (digest-keyed; rebuilt automatically when sources change). Add .mutineer/ to your .gitignore, and the directory you set with --cache-dir if it is inside the project. Keep .mutineer/ ignored even then: --test-command writes lock files to .mutineer/ beside each source file.

Mutant, by Markus Schirp, is the established mutation-testing tool for Ruby. It has been in development since 2012 and is the subject of published research. Mutineer is a separate tool, and it shares no code with Mutant.

Choose Mutineer when you want an MIT-licensed tool with no extra gems in your bundle, file-based runs, and a JSON contract built for CI gates and AI agents. Choose Mutant when you need its deeper operator set, Test::Unit support, or subject expressions that select whole namespaces. The two tools do not conflict, so you can run both on one project.

This comparison describes Mutant 0.17 (checked on 2026-10-06). Check Mutant's README for its current license terms and features.

MIT — see LICENSE.