0.0
The project is in a healthy, maintained state
When a commit weakens an existing test, Slopbouncer stops it until the commit message says which behaviour changed and why. The same check runs as a plain command in a pipeline, and as an optional turn-end hook for coding agents, comparing each test before and after a change without running any code.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
 Dependencies

Runtime

>= 1.5, < 3
 Project Readme

Slopbouncer

Gem version Test

A commit hook that catches weakened tests, whoever weakened them.

A coding agent's job is to make the checks pass. One way is to weaken the check. When a commit weakens an existing test, Slopbouncer stops it until the commit message says which behaviour changed and why. It reads code and never runs it, so it takes a fraction of a second.

Status: in development. It reads Ruby tests today: Minitest, as it blocks or test_ methods, and RSpec.

Setting it up

It needs Ruby 3.2 or later and git.

gem install slopbouncer

From the root of your repository:

slopbouncer install

This sets up git's commit-msg hook, so every commit is checked, whoever or whatever makes it. It goes wherever git runs hooks from, .git/hooks or the directory core.hooksPath names, as Husky and Lefthook set. If the repository already has a commit-msg hook, Slopbouncer adds one line to it, after its first, and the rest of the hook runs as before. Running it again changes nothing.

Name your coding agent to set it up for that agent too:

Agent Command What it adds
Claude Code slopbouncer install claude-code The commit hook, and rules in .claude/settings.json that stop Claude Code committing with --no-verify
Cursor slopbouncer install cursor The commit hook
Gemini CLI slopbouncer install gemini The commit hook
Codex slopbouncer install codex The commit hook

For Claude Code, --local writes the rules to .claude/settings.local.json instead, which keeps them out of the settings your team shares.

The hook needs the slopbouncer command on the PATH of whatever commits. When it cannot run, because the command is not there, its settings file cannot be read, or git fails, it says so and lets the commit through, with "This commit was not checked." A broken setup never stops anyone committing.

When it stops a commit

It reads only what the commit will hold: the test files as staged, whatever else is in the working tree. A commit that changes no tests, or only adds them, goes through without a word. One that weakens an existing test is stopped, with a line for each finding and the lines to add to the commit message:

Slopbouncer: this commit looks to weaken 1 existing test.

test/basket_test.rb: Basket adds up: assertion_removed: assert_equal 2, size

Put each test back, or add this to the commit message, with a reason on each line:

Test changes:
- test/basket_test.rb: Basket adds up: 

Fill in a reason after each test and commit again:

Make the basket simpler

Test changes:
- test/basket_test.rb: Basket adds up: size moved to its own test

A test named with no reason does not count. Once a message has explained a finding, Slopbouncer remembers it, so the same change committed again, after a reset or an amend, goes through quietly. A change to the assertions declared equivalent is asked about every time, however often it was explained. The memory is .git/slopbouncer/explained.json, in the git directory, and is never committed.

A change that only rewords tests, with no other code changed, is not stopped.

How often it stops a commit

Every commit the hook checks is logged in .git/slopbouncer/log.jsonl. After a week or so, see how often it stopped one and why the others went through:

slopbouncer stats
Commits checked: 214
Stopped: 9
Findings: 31
Explained in the commit message: 17
Already explained (repeats not asked about): 6
Not checked, because Slopbouncer could not run: 0

--json prints the same counts as JSON.

Getting round it

git commit --no-verify, or -n, skips every hook, this one included. A person can choose to do that. To stop Claude Code doing it, slopbouncer install claude-code adds these rules to .claude/settings.json, and you can add them by hand:

{
  "permissions": {
    "deny": [
      "Bash(git commit --no-verify:*)",
      "Bash(git commit -n:*)"
    ]
  }
}

Keep running the plain command in a pipeline as the backstop (see below).

Turn-end hooks for coding agents

Slopbouncer can also check an agent's work each time the agent is about to finish, before anything is committed. This is off unless you set it up:

slopbouncer install claude-code --turn-end
Agent Command What it adds
Claude Code slopbouncer install claude-code --turn-end A Stop hook in .claude/settings.json
Cursor slopbouncer install cursor --turn-end A stop hook in .cursor/hooks.json
Gemini CLI slopbouncer install gemini --turn-end An AfterAgent hook in .gemini/settings.json
Codex slopbouncer install codex --turn-end A Stop hook in .codex/hooks.json

It adds one hook and keeps everything else in the file. Running it again changes nothing. For Claude Code, --local writes to .claude/settings.local.json.

When the agent is about to finish and has weakened an existing test, it is sent back once and told which tests (the first 20, and how to list the rest), with two ways to answer: put the test back and make the code pass it, or leave the change and tell the person which behaviour changed. The second time it is let finish, so its reasons are on the record for the person to read.

The hook checks the work the agent has not committed. If your agent commits as it goes, edit the hook's command to name the commit its work started from, such as origin/main, in place of HEAD.

Codex skips a new hook until you have trusted it: after installing, open /hooks in Codex and review it.

Claude Code is the one this has been run under. The Cursor, Gemini CLI and Codex hooks are written to those agents' published hook contracts and tested against them, and have not yet been run inside the agents themselves. If one misbehaves, please open an issue.

This is what the turn-end hook printed for an agent that had removed an assertion from one test and edited the value another expects:

Slopbouncer: this change looks to weaken 2 existing tests.

test/checkout_test.rb: Checkout lets a guest pay: assertion_removed: assert_equal 'paid', order.status
test/refund_test.rb: Refund refunds part of an order: expected_value_changed: 100 → 90

For each one, do one of two things before you finish:
1. Put the test back as it was, and make the code pass it.
2. Leave the change, and tell the person which behaviour changed and why the test had to change with it.

The turn-end hook remembers what it has shown the agent for the work in hand, and sends it back only for findings it has not been shown. When nothing is new it lets the agent finish and says nothing. A finding that is put right and later brought back is shown again, and once HEAD moves, as it does on a commit, the memory starts empty. It keeps this in slopbouncer-explained.json in the git directory, never in the working tree. A change to the assertions declared equivalent is shown on every stop. The command run by hand, the pre-push stage and --json never use this memory and always report everything.

What it flags

Rule When
assertion_removed An assertion is gone from a test that still exists
expected_value_changed The value an assertion expects was edited
subject_changed The thing an assertion checks was edited, so the test now checks something else
assertion_changed An argument was edited in an assertion Slopbouncer cannot read sides into, such as a helper of your own, or the message an assertion shows when it fails
test_deleted A test, or its whole file, was deleted
skip_added A test was marked to be skipped or pending
no_assertions A test the change touched asserts nothing
equivalents_changed The assertions declared equivalent in .slopbouncer.yml changed, naming each group added or removed
helper_changed A method defined in the tests changed, and tests assert through it. Reported once, with how many tests depend on it

What it leaves alone

  • New tests added with new code.
  • Tests that gained assertions.
  • A test that was reworded, moved to another file, or both, with its checks unchanged.
  • Files that are not tests.

A change that only rewords tests

When a change touches nothing but tests, their directories and .slopbouncer.yml, an assertion removed from a test that gained as many as it lost, and has as many in all as before or more, is taken to be reworded. At a terminal and in an agent's hook those are one line:

Test refactor: 23 assertions reworded in 6 files, no other code changed.

--all lists them, and piped output and --json always do. Everything else is still reported in full, among them a deleted test, a test that asserts nothing, a test with fewer assertions than before, expected_value_changed, subject_changed and equivalents_changed. An agent is not sent back for the line alone. As soon as the same change touches a file outside the tests, every finding is reported as usual.

Running it yourself and in a pipeline

Anyone can get round a hook with --no-verify, so keep running the plain command where a person will see it. In a pipeline, or before a push, it is the backstop to the hook.

Run it from the root of the repository you want checked:

slopbouncer main..HEAD

The argument is a range of commits. Give one commit instead, such as HEAD or origin/main, and it checks everything since then, committed or not. At a terminal it lays the findings out by file, with each rule in colour:

test/basket_test.rb
  Basket applies a voucher
    skip_added  marked to be skipped

test/checkout_test.rb
  Checkout lets a guest pay
    assertion_removed  assert_equal "paid", order.status

test/refund_test.rb
  Refund refunds part of an order
    expected_value_changed  100 → 90

3 tests look weaker in 3 files

When its output goes to a file or another program, it prints one plain line for each finding:

test/basket_test.rb: Basket applies a voucher: skip_added: marked to be skipped
test/checkout_test.rb: Checkout lets a guest pay: assertion_removed: assert_equal "paid", order.status
test/refund_test.rb: Refund refunds part of an order: expected_value_changed: 100 → 90
Option What it does
--json Prints the findings as JSON
--strict Exits with 1 when anything is flagged
--all At a terminal, lists every finding. Without it, more than 30 findings are shown as a count of each rule in each file
--color Uses the terminal layout, in colour, wherever the output goes. For a script whose own output a person reads
--agent=NAME Runs as that coding agent's turn-end hook (see above)
--help, -h Prints how to use it
--version, -v Prints the version

Setting NO_COLOR keeps the terminal layout and leaves the colour out. Without --strict it exits with 0 whatever it finds: it flags for review and does not block. It exits with 2 when it is given nothing to compare.

RSpec

An expectation is read whole: its subject, its direction and its matcher. For a plain matcher such as eq(5) an edited value shows as 5 → 6. For a matcher built from several calls, such as change { total }.by(1), the whole matcher is shown before and after.

  • A spec with no name, such as it { is_expected.to be_valid }, is known by its body.
  • :skip, :pending, skip: 'reason' and pending: 'reason' beside a spec's or a group's name count as skipping it.
  • A before, after or around hook that makes an expectation is read as a test of its own, named after its group. The specs in that group are not reported as asserting nothing.
  • should and should_not are read as expectations too.

Your own assertion helpers

Slopbouncer counts calls such as assert_equal, refute_nil, must_equal and expect as assertions. If your tests assert through helpers of your own, name them in .slopbouncer.yml at the root of your repository:

assertions:
  - sees_total
  - is_refused

A test that only calls helpers Slopbouncer has not been told about looks like a test with no assertions.

When two assertions make the same check, as a framework's assertion and the helper you wrap round it might, declare them equivalent in the same file:

equivalents:
  - [assert_selector, assert_shown]
  - [assert_no_selector, refute_shown]

An assertion replaced by an equivalent one with the same arguments is then not flagged. With different arguments it is flagged as an edit, as assertion_changed. A change to these declarations is flagged as equivalents_changed in the same change, so declaring them cannot quietly hide a weaker test.

Its limits

  • It catches tests that are visibly weaker. A test that looks thorough and checks nothing needs mutation testing, which is slower and runs the code. Use both: Slopbouncer on every change, mutation testing behind it.
  • It cannot tell a stronger assertion from a weaker one. An assertion swapped for another kind is reported as removed.
  • It compares what assertions parse to, so what a formatter changes is ignored: a string's quote style, white space and line breaks, parentheses round an assertion's arguments, a block's braces against do and end, a trailing comma. Anything that parses differently is still flagged, including the braces taken from round a trailing hash, as in eq({ a: 1 }) rewritten as eq(a: 1).
  • It reads each test on its own. When a method defined in the tests changes and tests assert through it, as in assert paid?(order) or assert_predicate order, :paid?, it says so once as helper_changed, with how many tests depend on it, and leaves the judging to you: it cannot tell whether the helper checks more or less than it did. It finds those tests by the helper's name, in the helper's own file when that holds tests and in every test file when it does not. A helper reached only through another helper is not counted, and a check that lives elsewhere, such as an RSpec shared example pulled in with it_behaves_like, is still not followed, so a test that relies on one can look as if it asserts nothing.
  • It cannot see intent. A test changed on purpose, because the behaviour changed, is flagged the same way as one weakened to get a build to pass.

On its first run, over the 706 commits in Mainline's history that touch a test, it flagged 210 of them. In a sample of those, every finding was a real change to a test and every one looked intended. So read a finding as "this test changed in a way worth a look".

Working on it

bundle install
bundle exec rake

The tests are scenarios. Each one builds a throwaway git repository, makes a change in it, and runs the real command over that change. PROJECT.md holds the plan and the rules still to come.

Releasing

Move the version on with rake bump:patch, rake bump:minor or rake bump:major, and commit. Then tag that commit with the same version and push the tag, as the command tells you:

git tag v0.2.0 && git push origin v0.2.0

GitHub runs the scenarios and publishes the gem to RubyGems. A tag that does not match the version is refused before anything is published.

Licence

MIT. See LICENSE.