A strict Rails testing framework where fast and non-flaky are structural, not disciplinary.

Install · Quick start · Documentation · Contributing

class UsersController::CreatesUserCase < IntegrationCase
  witness(:valid_params) { { user: { email: "a@b.com", password: "secret123" } } }

  briefing { stub_network! }

  investigate "creates a user with valid params" do
    freeze_time

    post users_path, params: valid_params

    attest(response).to be_created
    attest(User).to exist(email: "a@b.com")
  end
end

Most suites are fast and reliable because a team keeps them that way by hand. Constable makes it structural instead — isolation you cannot opt out of, nondeterminism caught by a linter instead of by CI, and an adoption path that never asks you to rewrite anything.

Installed as constable-rails. The name constable was claimed on RubyGems in 2011 by an unrelated, long-abandoned gem. That is the only thing the suffix affects — everything you actually type is constable: the module, the CLI, the config directory, the generators.

• Why
• Requirements
• Getting started
  • Path 1 — an RSpec suite
  • Path 2 — a Minitest suite
  • Path 3 — no test suite yet
• Documentation
  • The DSL
  • Tiers
  • Matchers
  • Shared behavior
  • witness_all
  • Stubs
  • Preferences
  • Rails generators
  • Adopting an existing suite
  • Escape hatches
  • The linter
  • Jail, parole and warrants
  • Publishing coverage
  • Identity survives renames
  • Command reference
  • CI: several machines
  • Reading a run
  • Knowing what your suite is doing
  • Output
  • The blotter
  • Configuration
• Contributing
• Reporting a problem
• License

1. Isolation is non-negotiable in native code. No class-level shared state, no before(:all) equivalent. Every native test gets a clean transaction and a clean object graph.
2. Nondeterminism is caught by the linter, not discovered in CI. Bare sleep, unfrozen Time.now, and unstubbed network calls are lint errors before they are flakes.
3. Adoption never requires a rewrite. A whole existing RSpec or Minitest file runs completely untouched from day one. Strictness applies to new code — it is not a precondition for installing the gem.
4. Every escape hatch is visible. An unsafe block, a cold case, a jailed test — none are ever silent. They are reported every run until someone deals with them.
5. Fast is the default, not an opt-in. Boot tiers, parallel workers and git-diff test selection all ship in the base gem.

Constable pulls in six gems, all of them small and already present in most Rails apps:

Your app's own database is untouched by any of this: the blotter is a separate SQLite file Constable owns. See The blotter.

Nothing else is required. These are all optional, and only if you want the feature:

Three ways in, depending on what you have today. All three start the same way.

# Gemfile
group :development, :test do
  gem "constable-rails", git: "https://github.com/Ray-Hughes/constable.git", tag: "v3.24.0"
  gem "rubocop-constable", require: false
end

Installed from git while the RubyGems release catches up. rubygems.org still serves 2.1.1; everything since — cold-case fixes, warrants on cold cases, coverage publishing, timings-balanced shards, --watch — is released as a git tag. Pin the tag rather than the branch, so a run is reproducible.

$ bundle install
$ rails generate constable:install

That writes test/case_helper.rb, test/support/, .constable/config.yml, a .rubocop.yml snippet, and a worked example case. Two files matter and they do not overlap:

Setting a value in the wrong one raises and names the right one, so there is no precedence rule to learn. See Configuration.

Nothing is rewritten and nothing moves. import adds one glob to the config, and your specs run from where they are, through real RSpec, with results folded into Constable's reporting, flake history and CI gate.

$ constable import --from=rspec --dry-run   # read it first
$ constable import --from=rspec
$ constable test --full                     # the whole suite, cold cases and all

import fills in one block in test/case_helper.rb and changes nothing else:

Constable.cold_cases do
  rspec "spec/**/*_spec.rb"
end

That block is the link, not a description of one. Deleting it unlinks the suite; narrowing a glob shrinks what stays cold as you port directories across. It ships with the generated helper, with its example lines commented out, so adoption is an edit to something already in front of you rather than a file appearing from nowhere.

Porting does that bookkeeping for you. With delete: true the original spec is removed, so the glob stops matching it and nothing runs twice. When you keep the original instead -- to diff it against its port before committing -- modernize adds an except line, because otherwise the same tests run once as the new native case and once as the spec they came from:

Constable.cold_cases do
  rspec "spec/**/*_spec.rb"
  except "spec/models/user_spec.rb"   # ported to test/cases/models/user_case.rb
end

It lives in the test tree rather than in .constable/config.yml on purpose: linking a legacy suite decides what constable test runs at all, and a config key made that invisible.

At this point you are done. Everything below is optional and can happen a directory at a time, over months, with a green suite the entire way.

$ constable modernize spec/models --plan     # what a port would do. writes nothing
$ constable modernize spec/models            # do it
$ constable test test/cases/models           # run what came out

modernize rewrites describe/it into Constable::Case/investigate, let into witness, before into briefing and expect into attest. What it cannot decide safely it refuses to guess at: a flagged file is moved verbatim as a cold case instead, because a half-converted file does not run. Set the flags once in .constable/config.yml under modernize: and the command stays short.

Identical, with one word changed.

$ constable import --from=minitest --dry-run
$ constable import --from=minitest
$ constable test --full

Same single file, declaring the other engine:

Constable.cold_cases do
  minitest "test/legacy/**/*_test.rb"  # 312 files
end

Both can be declared at once, so a codebase with spec/ from one era and test/ from another gets one suite, one summary and one CI gate. Naming the engine also settles files the _spec.rb/_test.rb convention cannot answer for.

modernize handles def test_foo → investigate "foo" and setup → briefing, and the same rule applies: anything ambiguous is moved verbatim rather than guessed at.

Skip import entirely. The installer already left you a working example.

$ constable test                       # runs the example case
$ rails generate constable:model User  # scaffold a case for something real
$ constable test --full

Read test/case_helper.rb before writing the first case. It is short, and it explains the tier base classes and the two things Constable deliberately does not have.

$ constable test              # the whole suite, the way `rspec` with no args does
$ constable test --changed    # only what your current git diff touches
$ constable test --watch      # keep going: each save runs the tests that cover that file
$ constable test --timeout 60  # raise or lower the hang limit for this run
$ constable test --only=native   # skip the legacy suite
$ constable test --only=cold     # run only the legacy suite
$ constable last              # everything about the most recent run
$ constable metrics           # lifetime KPIs for the suite
$ constable insights          # what to fix first, and why
$ constable tree              # every command, if you forget one of these

--changed and --watch are for your own machine; CI should run everything. --watch maps a saved file to the tests covering it by the same rules as --changed -- a case covers itself, and app/models/user.rb is covered by user_case.rb, users_*_case.rb and user_spec.rb -- and runs them in a fresh process each time, because a test environment does not reload code and a watcher running stale code is worse than a slower one. constable test --watch --changed runs your current diff first, then watches.

The vocabulary is the API, not decoration.

investigate is a registration DSL, not a method definition. Each block runs in its own fresh instance, fully isolated from every other one.

class UsersController::CreatesUserCase < IntegrationCase
  docket "as an admin" do
    briefing { sign_in(:admin) }

    investigate "creates a user with valid params" do
      post users_path, params: valid_params
      attest(response).to be_created
    end
  end

  docket "as a guest" do
    investigate "is redirected to sign in" do
      post users_path, params: valid_params
      attest(response).to redirect_to(sign_in_path)
    end
  end
end

# test/case_helper.rb
class UnitCase < Constable::Case
  tier :unit
end

class IntegrationCase < Constable::Case
  include Constable::RailsSupport::Integration
  tier :integration
end

class SystemCase < Constable::Case
  include Constable::RailsSupport::System if defined?(Capybara)
  tier :system
end

Subclass whichever fits. Path-based inference (test/cases/models/** → :unit) still works as a fallback, but ordinary inheritance is the recommended pattern — nothing to infer.

RailsSupport::Integration is what gives a case get/post, response and your app's URL helpers; RailsSupport::System gives it Capybara and driven_by. UnitCase gets neither, deliberately — that is the tier that boots without them. The installer writes all three.

Rails' testing modules expect minitest's lifecycle, so Constable::Case also answers to the setup and teardown class macros. setup is an exact synonym for briefing and exists so those modules compose — briefing is still the way to write setup.

Constable wraps every investigation in a transaction and rolls it back. That is the default and it is almost always what you want.

The exception is a suite whose own cleanup is truncation — Capybara suites usually are, because a browser talks to a server over HTTP that cannot see an open transaction. The two are not interchangeable, and the difference is not only speed: Postgres sequences are not transactional, so a rolled-back test leaves the next one's ids where it found them, while a truncation resets them to 1. Specs written against truncation end up depending on that without saying so — a task built with assigned_by_id: some_organization.id passes only because organization 1 and user 1 both exist — and they fail against a rollback for reasons that have nothing to do with the code under test.

class SystemCase < Constable::Case
  transactional false

  briefing { DatabaseCleaner[:active_record].start }
  teardown { DatabaseCleaner[:active_record].clean }

  tier :system
end

Inherited the way tier is. Turning it off means the cleanup is yours; Constable says so once per run, with the case names, rather than letting it be discovered as cross-test contamination.

# test/support/matchers.rb
Constable::Matchers.define(:be_created) { |response| response.status == 201 }
Constable::Matchers.define(:exist) { |model_class, attrs| model_class.exists?(attrs) }

Built in: eq, eql, be, include, match, raise_error, have_attributes, exist, be_created, redirect_to, have_http_status, change, contain_exactly, match_array, start_with, end_with, be_between, be_within(d).of(x), satisfy, plus be_a, be_nil, be_empty, be_truthy, be_falsey and a be_* / have_* predicate fallback. be also takes the operator form — attest(count).to be > 0. Plain assert_* and refute_* primitives are always available alongside attest.

The set is deliberately smaller than RSpec's, so constable modernize flags any matcher it does not recognize rather than converting it into a case that only fails once you run it.

Anything written against RSpec's matcher protocol works, because Constable adapts it:

attest(user).to belong_to(:organization)      # shoulda-matchers
attest(list).to have(3).items                 # rspec-collection_matchers

Include whatever builds them on your tier base class — Shoulda::Matchers::ActiveRecord and friends — and the matchers themselves need nothing. Constable's own matchers are untouched; only foreign ones are wrapped.

# test/support/procedures.rb
TaskProcedure = Constable.procedure do
  witness(:task) { create(:task) }

  investigate "starts unassigned" do
    attest(task.assignee).to be_nil
  end
end

class ColocatedTaskCase < UnitCase
  follows TaskProcedure
end

A procedure is a constant, not a string in a registry, and that is the whole difference from shared_examples. RSpec registers a block under a name and looks it up when the suite runs, so a typo surfaces as "Could not find shared examples" mid-run rather than as a NameError on the line that made it. The registry is also global, so two files that both define "a task" quietly fight over the name — which is why real suites end up with names like "a task (from the appeals side)".

Inside the block, the ordinary DSL: investigate, witness, briefing, teardown, docket. follows evaluates it in the case, so a witness the case declares afterwards wins over the procedure's — the same scoping it_behaves_like has, and the reason to follow a procedure rather than copy it. Its tests are attributed to the file that followed it, so constable test that_file.rb runs them.

constable modernize converts same-file shared_examples into a class method, which needs no procedure at all. On one real suite that was 779 of 882 uses.

For helpers rather than tests, reuse across files is an ordinary module:

# test/support/authenticatable.rb
module Authenticatable
  def sign_in(user)
    post session_path, params: { email: user.email, password: "password" }
  end
end

include Authenticatable in any case. Ruby's own composition tools are more flexible than a parallel DSL that does the same job.

rails generate asks whatever is registered as the app's test framework what a test file looks like. Constable registers itself, so scaffolds produce cases rather than Minitest files for a framework you replaced.

$ rails generate scaffold Post title:string
      create  test/cases/controllers/posts_controller_case.rb
      create  test/cases/system/posts_case.rb

Every generator Rails hooks is covered — model, controller, scaffold, integration_test, system_test, mailer, job, helper, channel, mailbox, generator, resource — each writing a case that subclasses the right tier base class.

No fixtures are generated, deliberately: a witness builds exactly what one investigation needs and throws it away with it. A factory gem registered as your fixture_replacement still gets its turn.

Nothing gets rewritten. Cold cases run your original file through its own real engine — RSpec or Minitest — and feed pass/fail/timing into Constable's reporting, flake history and CI gate alongside native cases.

One line changes. The file body is untouched:

class LegacyUsersSpec < Constable::ColdCase::RSpec
  describe UsersController do
    it "creates a user" do
      post users_path, params: valid_params
      expect(response).to have_http_status(:created)
    end
  end
end

Or nothing changes at all — declare the path in the helper's cold_cases block:

Constable.cold_cases do
  rspec "spec/controllers/**/*_spec.rb"
end

constable import fills that block in. It is the one place cold-case globs live, and the reason it is there rather than in a config key is visibility: the helper is the file you already open, so the link between your legacy suite and Constable is something you see.

$ constable import --from=rspec        # reopen everything, verbatim
$ constable modernize spec/controllers/users_controller_spec.rb --alongside

modernize converts describe/it → Constable::Case/investigate, let and let! → witness and witness_all, before → briefing, after → teardown, expect → attest, allow(x).to receive(:y) → impersonate, and def test_foo → investigate "foo".

it { is_expected.to eq(true) } becomes investigate "is expected to eq true" — that is the name RSpec already generates from the matcher and prints in every report, so writing it down makes the existing name explicit rather than inventing one. It flags before(:all) and let! rather than converting them — those need a human decision — and leaves custom matchers and shared_examples alone, logging everything to .constable/docs/modernize-report.md. It writes nothing unless you ask it to.

A conversion only helps if the file ends up somewhere the runner looks. --port writes it into the native tree, mirroring the path:

$ constable modernize spec/models --port
spec/models/tag_spec.rb — 24 converted, 7 flagged
    → test/cases/models/tag_case.rb (verbatim, as a cold case)
spec/models/widget_spec.rb — 12 converted
    → test/cases/models/widget_case.rb

spec/models/widget_spec.rb becomes test/cases/models/widget_case.rb, directories and all. A file that converts cleanly is written as a native case.

A file where some tests convert and some do not is split in two:

test/cases/models/user_case.rb          # the tests that converted, native
test/cases/models/user_legacy_case.rb   # the rest, still RSpec, still running

Shared setup is kept in both halves, both run, and the test count does not change. A file of twenty tests where one uses rspec-mocks used to move all twenty verbatim — nineteen conversions thrown away for the twentieth.

A flag that sits outside every example, like a let! at the describe level, belongs to the whole file and there is nothing to separate. That one is written verbatim as a cold case, because a flagged conversion is not runnable — the flagged constructs are left as they were, so the class raises the moment it loads. Porting a broken file and calling it progress is worse than not moving it, so --port picks the form that runs and tells you which it used. --port --cold forces the verbatim form even for files that would convert.

--plan shows what a port would do before it does any of it:

$ constable modernize spec/models/tasks --plan --port --base UnitCase --delete
PORT PLAN
─────────
  65 files  13 convert  52 move verbatim  base UnitCase

WHY THE VERBATIM ONES CANNOT CONVERT
────────────────────────────────────
    266   56%  eager_let
    129   27%  rspec_mocks

ESTIMATED RUNTIME
─────────────────
  12m 43s  across 749 tests  measured from 65 of 65 files

  2m 25s of that is the test bodies. The rest is boot, file loading, suite
  hooks and cleaning between examples -- 0.82s per test, measured from your largest
  recorded run (88 tests).

The runtime figure comes out of the blotter, not out of the air: Constable already records an average duration per test to balance workers, and per-test overhead is measured from your own largest recorded run. Summing test bodies alone understated a real directory by 4.6x, and a single multiplier over-estimated it by 4x — a run of one file is nearly all Rails boot, a run of the whole suite is nearly all tests. A suite that has never run here gets no estimate rather than an invented one.

--batch N ports N files and leaves the rest. Paired with --delete it walks a directory, because the ones already ported are no longer there for the next run to find:

$ constable modernize spec/models --port --base UnitCase --delete --batch 20
SUMMARY
───────
  20 file(s) written -- 13 converted, 7 verbatim
  20 original(s) removed
  45 file(s) left in this directory -- run the same command again for the next batch

A port takes a file's local dependencies with it. require_relative "shared_examples.rb" resolves against the file's own directory, so moving the spec and leaving the companion behind breaks it — the ported file dies on LoadError before running a line. Companions are copied, not moved: specs you have not ported yet may still require them (a --batch port guarantees some will), and a duplicated support file is harmless where a deleted one breaks whatever still points at it.

Directories the port empties are removed. Only when empty, so nothing it did not take can go with them.

Set the flags once instead of typing them every time. In .constable/config.yml:

modernize:
  base:   UnitCase
  port:   true
  delete: true
  batch:  20

Then a port is constable modernize spec/models, and a flag on the command line still wins — the file says what this project does by default, the flag says what this invocation does instead. This matters more than convenience: delete without base moves a whole directory into classes that inherit none of your tier setup, and the first sign of it is NoMethodError: undefined method 'create'.

--limit is not that. It caps how many rows are printed and changes nothing about what is ported — the two are deliberately separate flags, because conflating them with --delete in play would delete files someone believed they had excluded.

Add --delete to finish the move:

$ constable modernize spec/models/organizations --port --base UnitCase --delete
    → test/cases/models/organizations/dvc_team_case.rb (verbatim, as a cold case), original removed

Without it both files are collected, so the suite runs those tests twice and the adoption number never moves — it counts the ported file and the spec it came from. --delete only removes an original that was actually written: a refused overwrite, a parse failure or a file that could not be ported keeps its source, because the one unrecoverable mistake here is deleting a test nothing copied.

--base matters more than it looks. Without it a converted case inherits Constable::Case and none of your tier classes — which is where FactoryBot, request helpers and auth live. Porting a real directory without it produced fifteen NoMethodError: undefined method 'create' out of eighty-four tests.

Nothing is ever overwritten: run a port twice and the second refuses, because a port gets run repeatedly while a suite is converted a directory at a time and any edit made after the first pass has to survive.

Native cases are not RSpec, so they do not run RSpec hooks. Whatever your spec/support does per test — resetting a memoized singleton, clearing a fake, emptying a thread-local — does not happen for a native case. That costs nothing while the two never meet, and becomes a bug the moment a half-ported directory puts both in one process.

It surfaced on a real port exactly once, and took a while to read: a class-level @system_user ||= memo was populated inside a native case's transaction; the transaction rolled back; ActiveRecord reset that in-memory object to an unsaved record while the memo kept pointing at it. The next cold case read .id, got nil, and died on a not-null constraint in a table it had nothing to do with.

Mirror that cleanup into your tier classes and it goes away:

class UnitCase < Constable::Case
  teardown { User.clear_memoized_singletons! }
end

The generated test/case_helper.rb carries this note too.

Native and cold cases run side by side in one constable test. No big-bang cutover.

witness is memoized per test, which is the right default and is also why an expensive fixture gets paid for on every test that uses it. witness_all builds it once:

class UserCase < UnitCase
  witness_all(:appeal) { create(:appeal, :with_post_intake_tasks) }

  investigate "is assigned" do
    attest(appeal.tasks).to be_present
  end
end

Measured on a real app, 15 tests sharing one expensive factory: 2.0s to 0.9s.

This is not before(:all), and the difference is what makes it safe to offer at all. The records live in a transaction opened before the case and rolled back after it, with each investigation in a nested transaction of its own, so nothing written to the database reaches the next test. That part is test-prof's before_all, which Constable delegates to rather than reimplementing — add gem "test-prof" to use it.

What a rollback cannot undo is a mutation to the Ruby object, since every test is handed the same instance. So each one is re-read from the database before use. The saving is the INSERT; the SELECT that makes it safe is the cheap half. reload: false declines it.

A case using witness_all is scheduled as one unit, so its tests stay together on one worker — a transaction opened in one process is no use to another.

On let!: converting it to a lazy let looks like the same optimization and is not. On a 152-test file it broke 13 tests and saved 5%, because the examples that skip the fixture are exactly the ones relying on the row existing. witness_all keeps it existing and stops paying to rebuild it.

Constable shipped no mocking library, which made allow(x).to receive(:y) the single largest reason a legacy file could not be converted. "Use a stub object instead" is fine advice for a file being written and useless for ten thousand that already exist.

investigate "retries once" do
  impersonate(client, :fetch, raises: Timeout::Error)
  impersonate(logger, :warn)

  attest { subject.call }.to raise_error(Timeout::Error)
  attest(logger).to have_been_asked(:warn).with("retrying").once
end

Everything is restored at teardown, including after a failure, because Constable owns the lifecycle and does not need you to remember.

Two deliberate differences from rspec-mocks. Stubbing a method the object does not have is refused, not optional — that stub passes forever and proves nothing, which is exactly what a rename leaves behind. allow_missing: true when the method really is defined later. And there is no proxy or signature reflection per stub, which is where rspec-mocks spends its time; the singleton method is replaced directly and the original put on a restore list.

constable modernize converts the common forms for you. expect(x).to receive(:y) is deliberately not one of them: it sets an expectation before the call and verifies at the end of the example, so rewriting it as an assertion afterwards would move when the failure surfaces.

.constable/config.yml is a team agreement. How many workers CI gets, what the coverage gate is, whether flakes are jailed — answers that have to be the same for everyone or they are not answers. But some of what a run does is nobody else's business, and editing a shared file to change it means either committing a preference for the whole team or carrying a dirty file forever.

$ constable config                      # what is set, and where it came from
$ constable config output expanded      # a line per test, for you
$ constable config heartbeat 30         # say how long it has been going, every 30s
$ constable config color false
$ constable config output --unset

That writes .constable/preferences.yml, which the installer gitignores.

The list is short and closed: output, heartbeat, color, slowest. A setting that changes what passes is not a preference, and is refused here — a suite that is green on one machine and red on another, with the difference in a file nobody else can see, is worse than no preferences at all.

A slow test leaves the terminal completely still — no output, no cursor movement, nothing to distinguish "working" from "hung". The honest reaction is to reach for ctrl-c, which is the one thing that makes it worse. So a spinner runs whenever the suite is alive and between results.

Deliberately not configurable. Every other display choice is a preference; this one answers "is it still running", and a user who turns it off and then cannot tell has been handed a way to make their own tools worse. It only appears on a terminal — in CI the output is a log nobody watches live, and animation frames would be thousands of junk lines.

If nothing finishes for 30 seconds, it stops spinning silently and says so:

  still running 2m 14s on LegacyAppealAffinitySpec

That case matters more than it sounds. The clock below is asked on each result whether it is due, which makes it silent during exactly the situation it exists for — a test that never finishes produces no results, so nothing is ever asked. This is the half that still speaks.

A suite that takes half an hour gives no sign of how far in it is. Step away, come back, and there is no honest answer to "has this been running ten minutes or forty".

  ────────────────────────────────────────────────────────
  test/cases/models/check_task_tree_case.rb
  1m 22s   341 tests  ·  31 failed
  ────────────────────────────────────────────────────────

Framed on purpose. Unframed it sits under a case line and reads as an annotation on that case — the honest first reaction to a bare · 1m 22s elapsed is "did something just fail?".

The first one prints before anything runs, so a 45-minute suite says so up front:

  ────────────────────────────────────────────────────────
  starting ~45m 00s  ·  ~6435 tests
  ────────────────────────────────────────────────────────

Both numbers come from the blotter. When too little of the run has been seen before, the estimate is withheld rather than guessed — it says how many files are new instead. A confident-looking wrong number gets believed once and then the whole line is ignored.

Time-based rather than per-test, so a fast suite never prints one and a slow one prints a handful instead of a wall. Off by default; constable config heartbeat 30 turns it on for you, heartbeat: in config.yml for everyone.

A converted file moves from RSpec's engine to Constable's, and the helpers it calls have to come with it. Otherwise the file converts cleanly, parses, and then fails at runtime on a method that was never loaded.

# test/case_helper.rb
Constable.load_support("test/support/**/*.rb", "spec/support/**/*.rb")

A file that calls RSpec.configure is skipped, and said so out loud. It is configuring RSpec, and Constable owns the transaction, the isolation and the system-tier setup itself — loading it would install a second, conflicting answer. On the suite this was written against that was exactly four files: database cleaning, Capybara, cache clearing, timezone.

Loading is only half of it. RSpec does the other half with config.include SomeHelper, so the modules a case actually calls need including on your tier base classes:

class UnitCase < Constable::Case
  include DateTimeHelper
  tier :unit
end

Per-test hooks from spec/rails_helper.rb need the same treatment — briefing and teardown on the base class. Constable rolls back the database and restores its own DSL globals, but it cannot know about your Timecop, your RequestStore, or your fake service clients. Without them a converted case inherits state from the one before it, and the symptom is oblique: a not-null violation on a column a factory filled from a current user nothing had set.

investigate "times out after thirty seconds" do
  unsafe { sleep(0.1) } # testing an actual timeout path, not a code smell
  attest(subject).to have_timed_out
end

Every unsafe emits a warning with its file:line and that adjacent comment as the reason. One warning per cold-case file, one per unsafe occurrence. Warnings never fail the build by default — fail_on_warnings: true opts CI into enforcing a downward trend — but they are never silent either.

rubocop-constable is scoped to native cases only; cold cases are exempt by design.

A large red legacy suite has an on-ramp. Run once in jail mode for a clean baseline, then work the docket down.

$ constable test --jail          # failures get jailed instead of failing the build
$ constable jail add PATH:LINE   # or jail one known-broken test by name
$ constable jail                 # the docket: reason, file:line, date jailed
$ constable jail run             # re-run jailed tests sequentially
$ constable jail parole PATH:LINE
$ constable jail release PATH:LINE

Jailing isn't hiding — it swaps "blocks the build" for "tracked and skipped." Jailed tests are always their own summary category, never folded into passed, and their briefing/witness setup still runs so setup rot surfaces immediately.

Parole is "probably fixed, not fully trusted yet." A paroled test runs normally but is watched: one failure is an immediate violation straight back to jail, and parole_period consecutive clean runs (default 10) auto-releases it. jail run never auto-releases on a pass — a single green run doesn't prove anything.

Warrants answer a different question — not "does this block the build" but "is this failure even real." With warrants on, a failing test is rerun in isolation warrant_retries times (default 5). Fails every retry, it's a genuine failure. Passes even once, it's flaky rather than broken: a warrant is written to the blotter, never to your source, and the result stops blocking the build while staying loudly visible.

$ constable warrants
$ constable warrants release PATH:LINE
$ constable watchlist    # everything under supervision: jailed, paroled, warranted
$ constable status       # trend: native-vs-cold %, recent runs, slowest historically

coverage: true measures; coverage_report decides where the result goes. Nothing is published until deliver names somewhere:

# .constable/config.yml
coverage: true
coverage_report:
  host: github          # github.com and GitHub Enterprise
  ci: github_actions
  deliver: [pr_comment] # any of: pr_comment, pr_description, email, custom

The report leads with the changed lines that never ran, each linked to the code, then the whole-suite number. Changed lines are measured from where the branch left the pull request's base (GITHUB_BASE_REF), so a repository whose PRs target staging is measured against staging. The base branch has to be in the clone: fetch-depth: 0, or fetch it.

constable test --coverage publishes at the end of the run, but only in CI: a developer running with coverage never comments on a pull request or emails the team. Publishing never changes the exit status; a delivery that fails is printed and the rest still go out.

A sharded run cannot publish from each shard: each has a fraction of the picture. Each shard saves its measurement to .constable/coverage/shard-I-of-N.json, and one job after the matrix merges them and publishes once:

# each shard
- run: bundle exec constable test --coverage --shard ${{ matrix.index }}/12
- uses: actions/upload-artifact@v4
  with: { name: "coverage-${{ matrix.index }}", path: .constable/coverage/ }

# one job, after the matrix
- uses: actions/download-artifact@v4
  with: { pattern: "coverage-*", path: .constable/coverage/, merge-multiple: true }
- run: bundle exec constable coverage publish
  env: { GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}" }

constable coverage publish --dry-run prints the report instead of delivering it. --html PATH also writes the full, browsable report; upload it as an artifact and pass its URL back with --report-url and the comment links to it. title: and note: under coverage_report set the report's heading and a closing note.

Each test's key is a content hash of its investigate block body, whitespace-normalized. Class name, description and file are stored alongside purely as a display label.

• Rename the class, reword the description, move the file → hash untouched, history carries over.
• Change what the test actually does → hash changes, history starts fresh. Correct, not a limitation.
• Renamed and tweaked in one commit? Constable notices an old test vanishing as a similar new one appears and suggests constable history relink OLD NEW. Set auto_relink: true to confirm high-confidence matches automatically.

Two tests with byte-identical bodies would otherwise share a key — and bodies repeat more than the phrase "content hash" suggests, since attest(build(:thing, name: nil)).not_to be_valid is the same handful of tokens in every model case. Constable re-keys colliding tests on their class and description once the suite is loaded, so no two tests ever share a docket row. Rename-survival is weaker for exactly those tests, which is the right trade: a history belonging to two tests at once is worse than one that resets.

What to run: PATH[:LINE] --changed --watch --full --only MODE --tier T --shard i/n --shard-by-time --timings FILE.

How it runs: --seed N --workers N --timeout N --jail --warrants --coverage.

What it prints: --expanded --concise --output MODE --show warnings --verbose --no-color --failures-to FILE --timings-out FILE.

Order is randomized every run for native cases, with the seed printed and replayable via --seed. Cold cases keep their own engine's order. Workers run in parallel by default, load-balanced by a cached per-test duration index.

Each worker gets its own database, built from schema the way rails test does it — or kept between runs, with worker_databases: reuse, which is both faster and the only thing that works for an app whose schema cannot rebuild the database by itself (any app with Postgres custom types: CREATE TYPE has no schema.rb representation). Prepare those once with constable prepare, and again after a migration — kept databases do not follow one on their own. Forgetting is caught rather than suffered: Constable compares what each worker database has migrated against the real test database before it forks, and runs serially (which uses the real one, so it is correct) rather than testing yesterday's schema. Sharing one would not be a speed/safety trade but a correctness bug: on SQLite the run dissolves into database is locked, and on a client/server database tests quietly see each other's rows. If your app has ActiveRecord but cannot shard, Constable runs serially and says why — slow is a trade-off, wrong is not.

The database is not the only thing a worker needs to itself. Anything your suite keeps on disk per process — a browser cache, a download directory, a screenshot path — needs a name that differs per worker, or they race for it. Each worker is told which one it is:

worker = ENV["CONSTABLE_WORKER"] ? "_w#{ENV['CONSTABLE_WORKER']}" : ""
cache  = Rails.root.join("tmp/browser_cache#{worker}")

Some databases cannot be given to each worker at all — Oracle and anything else Rails does not manage (database_tasks: false). Constable does not try, and now says so at the start of a parallel run, because skipped and safe are different claims:

⚠ vacols (database_tasks: false) cannot be given to each worker, so all of them share it.
  Tests that write to it will interfere with each other, and the failures will not look
  like a parallelism problem -- they look like rows vanishing mid-test.

That warning is worth taking literally. On a real app with a legacy Oracle database, a four-worker run produced 163 failures that all passed serially; 53 of them were a bare VacolsRecordNotFound, because each worker's before(:suite) deleted from the one shared database while the others were midway through tests that had just written to it.

And a harder limit, if your app talks to one through a C driver: forking may not be possible at all. The same app aborts roughly half its parallel runs with SIGABRT — no output on either stream, the crash report landing inside libclntsh, Oracle's client library catching a SIGSEGV in its own signal handler. It is not a Constable failure and there is nothing Constable can do about it: a process holding OCI handles is not reliably forkable. If you see bare exit code 134 and no output, check ~/Library/Logs/DiagnosticReports (or your platform's equivalent) before assuming the test runner ate your suite, and run those specs with worker_databases: off.

CONSTABLE_WORKER is the index and CONSTABLE_WORKERS the count; both are unset in the parent, so serial runs keep whatever name they had. Use FileUtils.mkdir_p rather than Dir.mkdir ... unless File.directory? while you are there — the second is a race, and if it runs inside spec/support it takes rails_helper down with it, which costs the loser its database cleaning rather than just its cache directory.

Constable balances work across forked workers on one machine. A CI matrix is the other axis, and --shard covers it — no separate report file to generate, upload and keep current:

strategy:
  matrix:
    shard: [1, 2, 3, 4, 5, 6, 7, 8]
steps:
  - run: bundle exec constable test --full --shard ${{ matrix.shard }}/8

The split is a partition: every test lands in exactly one slice, and each machine derives the same answer without talking to any other. That property is the whole feature — a splitter that drops a file produces a green build that ran less than it claimed, and nothing downstream can tell.

Which is why the default split depends on nothing but the set of files. --shard-by-time weights it by recorded durations, so slices are even in time rather than in file count, and it is safe only when every machine reads identical duration data — a blotter restored from one shared cache, never one written back to mid-matrix. Weighting from a blotter that moves between shards repartitions: measured across three local shards, one file ran in two of them and another ran in none.

A timings file makes identical data easy. Each shard writes what it measured, one job merges them, and every shard of the next run reads that one file:

# each shard
- run: bundle exec constable test --shard ${{ matrix.shard }}/8 --shard-by-time
       --timings tmp/timings.json --timings-out tmp/timings-${{ matrix.shard }}.json
# one job after the matrix: combine them and cache the result for the next run
- run: bundle exec constable timings merge tmp/timings.json tmp/timings-*.json

Every sharded run prints Shard 3/8 · partition 1a2b3c4d5e6f. The fingerprint is equal on every shard that divided the suite the same way, and timings merge fails the job when they did not, rather than letting a build claim tests it never ran. Restore the timings file into every shard from one exact cache key, resolved once before the matrix starts; a "newest match" lookup on each machine can pick different files.

.constable/config.yml holds every setting, and is the only place any of them is set. test/case_helper.rb holds code: the tier base classes, before_suite/after_suite, matcher definitions. Assigning a setting in Constable.configure raises and says to come here — settings had two homes once, which bought a precedence rule to learn and the same setting documented twice across two generated files.

Nothing is lost by that. The file is run through ERB before it is parsed, exactly as Rails does for database.yml, so the one thing Ruby could do that YAML could not still works:

parallel_workers: <%= ENV.fetch("CI_WORKERS", 4) %>
worker_databases: <%= ENV["CI"] ? "reuse" : "off" %>

One caveat worth knowing: ERB is applied to the whole file, comments included. A <%= %> inside a # comment is still executed, so a commented-out example that raises will stop the config loading.

A CLI flag still beats the file for a single run.

The last two sections are the ones you act on, and they are last on purpose: after a long run the headline at the top has scrolled away, so the counts someone goes looking for are the ones they would otherwise scroll back for.

  FAILURES
  ────────
  ✗ SessionsCase
    "expires after inactivity"
    spec/cases/sessions_case.rb:12
    broke at app/models/session.rb:88
    1.2s  ·  failed 4 of the last 12 runs

    Expected response to be :created, got :unprocessable_entity

    Rerun just this test:
      constable test spec/cases/sessions_case.rb:12 --seed 8841

  RECOMMENDATIONS
  ───────────────
  3 failing tests have failed repeatedly before
    Jail them to stop blocking the build while they are worked on:
    constable jail spec/cases/sessions_case.rb:12

  SUMMARY
  ───────
  6 tests   2 passed   1 failed   2 jailed   1 warranted
  12.4s total · seed 8841 · 2 warnings

Three things a failure now carries that it did not: where it broke (the first backtrace frame that is not the test itself — the framework is already stripped out), how long it took, and how often this same test has failed before. The last is the one that changes what you do: a first failure is news about the change you just made, a test that has failed four of the last twelve runs is news about the test — and it is what drives the jail recommendation rather than someone deciding at 5pm. Identity is a hash of the body, so that history survives a rename and does not survive a rewrite, which is correct in both directions.

A duration only prints once it is over 50ms; <1ms under a failure tells nobody anything.

Sections collapse rather than bury: past five, warnings print as a count with --show warnings to open them. A flag, not a keypress, so the same command prints the same thing in a CI log as on a terminal.

Testing already happens in a terminal, so the answers should too. Three commands read what the blotter has been recording all along -- nothing is collected specially, which means a suite that has been running for months already has months of answers in it.

constable last -- the post-mortem for the run you just did:

LAST RUN
────────
  2026-09-09  full   seed 9172  896ms  192 passed, 0 failed

SLOWEST FILES
─────────────
      3.0s (54%)  14 tests  test/cases/controllers/api/v1/tasks_case.rb

Per file, not just per test, because a file is the unit someone actually opens. The share is of total test time rather than wall-clock: with workers in play the tests add up to more than the run took, and a percentage of the clock can exceed 100%.

constable metrics -- the lifetime view:

LIFETIME
────────
  67 runs  8816 tests executed  99.8% passed  4m 12s of runtime (61 of 67 runs timed)

FLAKIEST
────────
  5/21 failed  LegacyTimingCase "treats a task due today as not yet overdue"

NEVER PASSED
────────────
  9 runs  ReportCase "exports a quarterly summary"

Flakiest and never-passed are deliberately separate sections. A test that has never passed is not flaky, it is broken, and the two want opposite responses -- putting them in one list is how a flake report becomes noise. Runs recorded before durations were persisted have none, so the runtime figure says how many runs it actually covers rather than quietly understating the cost.

constable insights -- what to fix first:

  spec/models/appeal_spec.rb is 31% of the suite's time (4m 02s across 205 tests)
    Average 1.2s per test. A file this size is usually one expensive fixture or one
    `before` doing real work for every example.

  3 tests flip between pass and fail
    Worst: TaskCase "reassigns to the next judge" -- 5 failures in 21 runs.
    `constable test test/cases/task_case.rb:88 --warrants` reruns it in isolation.

Every line is tied to something measured -- a recorded duration, a counted status flip, a row on the docket -- and nothing prints on a hunch. A file is only named when it owns at least a tenth of the suite's time, because every suite has a slowest file and naming it at 11% is noise. A report that cries wolf is one nobody reads twice.

stdout is reserved for results — not just Constable's own output, but the app's. Rails loggers, SQL, request/response logging, and anything a gem prints to $stdout or $stderr mid-run all go to log/test.log; --verbose streams it back for active debugging.

That matters more than it sounds. A gem warning fired once per file lands in the middle of the live stream, and you get Address ✓✓✓✓✓✓... instead of a readable run. The one thing Constable deliberately does not intercept is a write straight to file descriptor 2 — capturing that would also swallow a real crash and break binding.pry, so 2>/dev/null stays yours to decide on.

While it runs, the live stream has two modes. concise is the default: one glyph per test, grouped into a run per case, so a thousand-test suite stays inside one screen and a wall of green is the point.

  ProjectCase                       ✓✓✓✓✓✓✓✓✓✓✓✓
  BillingCase                       ✓✓⚖⛓✓

expanded trades that for a line per test — glyph, name, duration — so you can see which test is hanging while it hangs, rather than after.

  ProjectCase
    ✓ validations rejects a colour outside the palette  2ms
    ✓ validations rejects a duplicate name for the same owner 12ms
    ✗ #completion_ratio is the fraction of done tasks   8ms

  BillingCase
    ⚖ charges a card                                    310ms
    ⛓ refunds a charge — assertion failed on the amount
    ✓ issues a receipt                                  1.4s

Set it in .constable/config.yml (output: concise or expanded), or per run with --expanded / --concise. A jailed test never ran its body, so it is given no duration rather than a dishonest 0ms. The summary below is identical in both modes — the mode only changes what you watch on the way there.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  CONSTABLE            6 tests · 3 cases · 12.4s
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  ✓ 2 passed   ✗ 1 failed   ⛓ 2 jailed (1 parole violation)   ◑ 1 on parole   ⚖ 1 warrant issued   ⚠ 2 warnings   ◐ 92% covered (3 files unpatrolled)

  PAROLE VIOLATED
  ───────────────
  ⛓ UsersController::CreatesUserCase
    "creates a user with valid params"
    spec/cases/users_controller/creates_user_case.rb:8
    Failed on day 3 of a 10-run parole — back to jail. This is its 2nd time in jail.

  → Somebody trusted this test again and it let them down,
    so it is back on the docket. Fix it before the next
    constable jail parole — a second violation is the signal
    that the test, not the flake, is the problem.

  FAILURES
  ────────
  ✗ SessionsCase
    "expires after inactivity"
    spec/cases/sessions_case.rb:12

    Expected response to be :created, got :unprocessable_entity

    Response body:
      { "errors": ["Email has already been taken"] }

    Rerun just this test:
      constable test spec/cases/sessions_case.rb:12 --seed 8841

  WARRANTS
  ────────
  ⚖ BillingCase
    "charges a card"
    spec/cases/sessions_case.rb:12
    Failed, then passed 4 of 5 retries run in isolation.

  → A warrant is "not reproducible", not "not a problem" —
    it stops blocking the build and stays visible until
    someone deals with it. Fixed the flake? constable
    warrants release PATH:LINE

  JAILED
  ──────
  ⛓ BillingCase
    "refunds a charge"
    spec/cases/sessions_case.rb:12
    Assertion failed on the amount.

  → Jailed means skipped and tracked, not passing. Think one
    is fixed? constable jail parole PATH:LINE runs it for
    real again — 10 clean runs and it releases itself.

  ON PAROLE
  ─────────
  ◑ SessionsCase
    "signs a user in"
    spec/cases/sessions_case.rb:12
    Day 4 of 10 — 6 clean runs to go.

  → A paroled test runs for real and is watched: one failure
    sends it straight back to jail. constable watchlist
    shows everything under supervision.

  WARNINGS
  ────────
  ⚠ spec/legacy/old_users_spec.rb
    running as a cold case (Constable::ColdCase::RSpec) — 12
    tests not yet under native rules

  ⚠ spec/controllers/sessions_case.rb:44
    unsafe { sleep(0.1) } — "testing an actual timeout path,
    not a code smell"

  SLOWEST
  ────────
  3.2s  UsersController::CreatesUserCase "creates a user with valid params"
  1.1s  SessionsCase "times out after thirty seconds"
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Sections print worst-to-least-urgent: parole violations, failures, warrants, jailed, on parole, warnings, slowest. A section only appears when it has something to say.

Each supervision section ends with one line saying what to do next, because "2 jailed" is a fact and constable jail parole PATH:LINE is an action. Failures don't get a hint — they already end with the exact command to rerun them.

Flake history, the jail docket and warrants live in a store Constable owns entirely — by default a self-contained .constable/constable.sqlite3 in WAL mode. Never your app's database: native cases roll back their transaction and would roll this data back with it, :unit-tier runs skip booting the DB stack for speed, and the workload is a handful of tables that doesn't need a client-server database.

Teams who need one queryable store across many CI machines can point it elsewhere — always a separate connection from the app's own:

storage:
  adapter: postgres
  url: postgres://user:pass@host/constable_metadata

Settings can be written in Ruby, in test/case_helper.rb — the same place RSpec puts RSpec.configure — or in .constable/config.yml, or on the command line. The most specific wins:

a CLI flag              --workers 4, --expanded      one run
Constable.configure     test/case_helper.rb          code you deliberately ran
.constable/config.yml   the project's declared default
Constable's defaults

Every key below can be set in either place. Put settings that differ per machine or per branch in the YAML, where they are obvious and greppable; put settings that have to be computed in Ruby, because YAML cannot:

# test/case_helper.rb
Constable.configure do |c|
  c.parallel_workers = ENV.fetch("CI_WORKERS", 4).to_i
  c.coverage         = ENV["CI"] == "true"
  c.output           = :expanded
end

Not config/initializers/, which is where a runtime gem like Devise goes. Initializers run on every boot including production, where a test-only gem is not in the bundle, so an initializer calling Constable.configure takes the app down. It is the same reason RSpec, SimpleCov, WebMock and Capybara all configure from the test helper.

config.yml is optional, and one setting is the reason it exists: storage can only be set there. The blotter is opened before case_helper.rb loads, so that constable jail, warrants, watchlist and status can read the docket without booting the app — a broken app should not stop you reading the docket. Setting it in Ruby raises rather than being quietly ignored.

Beyond that it is a preference. A settings file is greppable and diffable without running anything, which suits values that differ per project or per branch; Ruby suits anything computed.

config.yml doubles as the reference, so re-run rails generate constable:install --skip after an upgrade: it appends any settings your file does not mention and leaves your own values and comments alone. (--skip so the other generated files, which you have probably edited, are left as they are.)

# .constable/config.yml
storage:
  adapter: sqlite                # sqlite (default) | postgres | mysql
  path: .constable/constable.sqlite3

warrants: false                  # opt-in flaky detector
warrant_retries: 5
auto_relink: false

parole_period: 10                # consecutive clean runs to auto-release

coverage: false
coverage_threshold: 90           # diff-based — only lines changed in the current diff
coverage_html: false

fail_on_warnings: false
output: concise                  # live stream detail: concise | expanded
parallel_workers: auto

tiers:                           # fallback inference; base classes are primary
  unit: "test/cases/models/**/*"
  integration: "test/cases/controllers/**/*"
  system: "test/cases/system/**/*"

Bug reports and pull requests are welcome at https://github.com/Ray-Hughes/constable.

$ git clone git@github.com:Ray-Hughes/constable.git
$ cd constable
$ bin/setup
$ bundle exec rake test      # the framework's own suite
$ bundle exec rake cops      # the RuboCop extension's suite
$ bundle exec rubocop        # lint

The repo holds two gems: constable-rails at the root, and rubocop-constable in its own directory with its own gemspec and suite. docs/ARCHITECTURE.md is the interface contract between components and is worth reading before a substantial change; docs/SPEC.md is the product spec.

A few house rules, so a change lands cleanly:

• Constable's own suite is Minitest, not Constable — it cannot test itself before it works. Add tests under test/unit/ or test/integration/.
• New behavior needs a test that would fail without it. Several of the nastiest bugs in this gem were invisible to unit tests and only appeared when the real binary ran against a real Rails app; an integration test is often the honest one.
• Keep rake test and rubocop green. CI runs both on Ruby 3.1, 3.2 and 3.3.
• Comments explain why, not what.

Please open a GitHub issue. Include:

• what you ran, and the full summary block it printed
• the seed, so the order is replayable (constable test --seed N)
• your Ruby and Rails versions, and whether the case is native or a cold case

If a test behaves differently alone than in a full run, say so explicitly — that is an order-dependency bug and Constable has machinery specifically for it.

MIT.