hen’itai

変異体 (へんいたい, hen’itai) Pronunciation: he-n-i-ta-i (4 morae; no stress accent) Meaning: mutant, mutated organism, or variant entity

A Ruby mutation testing framework

• This is alpha software, there will be bugs
• Henitai tests itself and other projects
• It might break with a release
• If you need a mature solution for mutation testing in ruby, take a look at the [https://github.com/mbj/mutant](Mutant gem)

Mutation testing answers the question that code coverage cannot: does your test suite actually verify the behaviour of your code?

A mutation testing tool makes small, systematic changes — mutants — to your source code (e.g. replacing > with >=, removing a return statement, flipping a boolean) and then runs your tests. A mutant that causes at least one test to fail is killed. A mutant that passes all tests is survived — evidence that your tests are not covering that behaviour.

The ratio of killed mutants to total mutants is the Mutation Score (MS). A high mutation score is a strong quality signal.

As mutation testing modifies your code you make sure that your tests cannot have any unintended side-effects.

Add to your Gemfile:

gem "henitai", group: :development

Or install globally:

gem install henitai

Requires Ruby 3.3.6+

# Run mutation testing on the entire project
bundle exec henitai run

# Run only on subjects changed since main (CI-friendly). Includes uncommitted
# working-tree changes; changed test files select the sources they cover.
bundle exec henitai run --since origin/main

# Run on a specific subject pattern
bundle exec henitai run 'MyClass#my_method'
bundle exec henitai run 'MyNamespace*'

# Re-run only survivors from a prior mutation report
bundle exec henitai run --survivors-from reports/mutation-report.json

Configuration lives in .henitai.yml:

# yaml-language-server: $schema=./assets/schema/henitai.schema.json
integration:
  name: rspec

includes:
  - lib

excludes:
  - lib/henitai/eager_load.rb   # standalone entry points with no in-process coverage

test_excludes:
  - spec/end_to_end/*_spec.rb   # never run these files, mutants or coverage baseline

mutation:
  operators: light   # light | full
  timeout: 10.0
  timeout_multiplier: 3.0
  max_flaky_retries: 3
  flaky_detection: false   # run the baseline twice to find flaky test files
  max_log_bytes: 5000000
  max_timeout: 30.0
  control_runs: true   # re-run detected subjects with the unmutated method
  fork_server: false   # opt-in: boot the RSpec/Minitest environment once, fork each mutant run from it (any jobs; falls back to per-mutant boot with a warning)
  sampling:
    ratio: 0.05
    strategy: stratified
    seed: 0   # integer >= 0; selects which mutants of each stratum are kept
reports_dir: reports
reporters:
  - terminal
  - json
  - html

reports:
  checkpoint: true
  checkpoint_every: 200
  checkpoint_interval: 30.0

thresholds:
  high: 80
  low: 60

Henitai warns on unknown config keys and aborts with Henitai::ConfigurationError when a value is invalid.

CLI flags override the corresponding values from .henitai.yml.

test_excludes removes matching test files from per-mutant selection and from the coverage-bootstrap suite. A mutant with no tests left is reported as NoCoverage and is not executed. Excluding slow end-to-end or integration suites therefore also keeps them out of the baseline run — which matters, because a test that never runs against a mutant can never kill one, and the baseline suite has a fixed timeout. Lines covered only by excluded tests count as uncovered and are not mutated. Excluding every test file aborts with Henitai::ConfigurationError. mutation.max_log_bytes is a positive per-stream byte cap for captured child output. Without an explicit mutation.timeout, each mutant's timeout is calibrated as mutation.timeout_multiplier (default 3.0) × (the summed durations of the selected test files + the measured suite boot time), with a 2 s floor. mutation.max_timeout (default 30.0) caps the calibrated value, but never below the measured baseline plus 2 s; when the baseline alone exceeds max_timeout, Henitai warns once and uses baseline + 2 s. An explicit mutation.timeout still takes precedence. When JSON or HTML reporting is enabled, reports.checkpoint* controls periodic canonical-report checkpoints. Each checkpoint updates mutation-report.json; when the HTML reporter is configured, it also regenerates mutation-report.html from that merged JSON.

mutation.sampling keeps ceil(size * ratio) mutants per subject (at least one). Mutants are stratified by operator: each operator's mutants are ordered by a SHA256 of seed and the mutant's stable ID, and the strata are visited round-robin, so every operator is represented before any contributes a second mutant. The kept mutants retain their original order. The same seed selects the same mutants across runs and line shifts; change seed to draw another sample. The applied {strategy, ratio, seed} is recorded in the report under sampling.

Scoped runs (--since, subject patterns, survivor reruns) are merged into the existing mutation-report.json by stableId. A run that mutated every subject of a file (a non-authoritative run without subject patterns, such as --since) lists it under the top-level completeFiles and replaces that file's prior entry wholesale, dropping it when it yields no mutants, so stale mutants of edited or deleted code do not linger. Each file entry carries sessionId and gitSha provenance of the run that produced it; a file merged from several sessions has none. --since diffs against the merge base (REF...HEAD), so commits that landed on the base branch after you forked do not count as your changes.

Before mutation testing starts, Henitai checks whether the current coverage data covers the configured source files. If not, Henitai runs the configured test suite once to bootstrap a usable coverage baseline. If coverage is still unavailable for the current sources, henitai run aborts with Henitai::CoverageError.

Survivors are never retried: flakiness can only turn a survivor into a kill, so a retry could only inflate the score. A killed mutant is retried, up to mutation.max_flaky_retries times (default 3), only when every test file that failed for it is a known flaky test; if any retry survives, the mutant is reported as Survived.

Known flaky tests come from the opt-in mutation.flaky_detection: true. The baseline suite then runs twice (existing coverage is not reused), and test files whose pass/fail outcome differs between the runs are written to reports/henitai_flaky_tests.json. Without flaky_detection there is no flaky set and nothing is retried. At the end of the run Henitai reports the flaky tests and how many kills were re-checked.

A mutant only counts as detected when a test failed because of the mutation. Before a mutant runs, Henitai checks that its activated code differs from the original method by exactly the reported mutation; a mutant that fails this check is reported as CompileError with the reason in statusReason instead of running. Each mutant child reports what it got to — a failed activation, a finished test run, or a SystemExit raised by the code under test — and only a finished test run yields Killed or Survived. A failed activation is a CompileError; a child killed by a signal, one that exits without reporting, and a SystemExit during the tests are RuntimeError. With mutation.control_runs (on by default), every subject that has detected mutants is run once more with its unmutated method injected; if its tests fail then, the injection itself is broken for that subject and its detected verdicts are reclassified as CompileError.

Per-test coverage reporting is currently wired through the RSpec child runner. Minitest integration reuses the same selection and execution flow, but does not yet enable the per-test coverage formatter.

Test selection is coverage first: the per-test coverage map is the primary source of candidate tests for a mutant, and name matching is only the fallback for when the map cannot decide (no map, no location, or no covering test left in the integration).

Henitai currently defaults to linear mutant execution. Set jobs in .henitai.yml or pass --jobs N to opt into parallel mutant execution.

Every forked test child sees a HENITAI_WORKER_SLOT environment variable holding a stable worker-slot index (0..jobs-1; always 0 on the linear path). A flaky-retry respawn keeps the original attempt's value. Test suites that touch shared external resources can isolate themselves per slot without any henitai-side hooks — for example a per-worker database:

database: "myapp_test_#{ENV.fetch('HENITAI_WORKER_SLOT', '0')}"

By default, Henitai keeps child test output out of the live terminal. Each baseline or mutant run writes captured stdout/stderr to reports/mutation-logs/ and the terminal only shows progress plus a concise summary. Pass --all-logs (or --verbose) to print every captured child log.

henitai version prints the installed version. henitai run exits with 0 when the mutation score meets the low threshold, 1 when it does not, and 2 for framework errors. With --strict-exit-codes (opt-in, additive) it also exits 3 when one or more mutants timed out and 4 when runtime/compile errors are present; precedence 2 > 3 > 4 > 1 > 0. The timeout code is informational — a run can pass its threshold and still exit 3. A run with no scoreable mutant passes its threshold only when there was nothing to score (no changed code, everything ignored). When CompileError harness errors emptied the score denominator, nothing was tested, and the run exits 1. Pending mutants (left over from an interrupted run, without a verdict) are excluded from both the MS and the MSI denominators; MSI is killed / (total - pending).

For pull-request feedback, add the github reporter to reporters: in .henitai.yml: it prints one ::warning file=...,line=... workflow command per survived mutant, so survivors appear as inline annotations on the PR diff in GitHub Actions. All other statuses stay silent.

henitai run --incremental reuses still-valid Killed and Survived verdicts from the history store (reports/mutation-history.sqlite3) instead of re-executing them. A Killed verdict is reused when the subject's source and every covering test file are byte-identical to what was recorded. A Survived verdict needs more proof, because a new test could now kill it: it is reused only when, additionally, the live covering set derived from the current per-test coverage map matches the recorded set exactly (no test added, dropped, or changed). Both kinds additionally require the reuse context to be unchanged: a fingerprint of the test dependency files (spec helpers, spec/support, fixtures/factories, Gemfile.lock, .henitai.yml, .rspec), the henitai version, the integration, the operator set, test_excludes and coverage_criteria. Verdicts recorded without a reuse context are never reused. When per-test coverage is unavailable, survivors simply re-execute — reuse never happens on doubt (see ADR-11). Reused mutants stay visible in the report (fromCache: true beside stableId) and count toward MS/MSI; the terminal prints N of M verdicts reused from history (K killed, S survived) plus an executed-only MS/MSI line so cached verdicts are never mistaken for fresh results. Generated mutants temporarily include legacyStableId so scoped runs replace canonical entries written before site-offset IDs were introduced instead of duplicating them. Timeouts and errors always re-execute. --force bypasses reuse, including the reuse applied to --survivors-from reruns. henitai run --dry-run lists the post-filter mutant set without executing mutants and always exits 0. Gate 0 may still run the configured test suite to refresh baseline coverage.

Every henitai run, including a dry run, acquires a fail-fast advisory lock at <reports_dir>/.henitai-run.lock. henitai clean uses the same lock, so it cannot delete artifacts from a live run. Contention is a framework error (exit code 2); direct Runner#run callers receive Henitai::ConcurrentRunError. The persistent JSON lock file records the last owner's PID and start time. It is coordination metadata, not evidence of an active run: the kernel lock is authoritative, and the file may be the only artifact a dry run creates.

henitai run --survivors-from ... performs a partial rerun: it reports only the selected survivors, skips threshold-based exit checks, and does not update the run trend history. The selected survivors go through the same reuse rules as --incremental, always, not only with that flag: a survivor is skipped (reported with fromCache: true) only when its subject source, its covering test set and the reuse context are unchanged, so new or edited tests and helpers invalidate it. Dirty worktree changes are included, so you can edit tests locally without committing first; if source files under includes are dirty, the activation-recipe fast path is not used and the survivors are re-matched against freshly generated mutants. Survivors restored from activation recipes have no source range to fingerprint and are always re-executed. When --survivors-from names a canonical report, Henitai uses the matching session snapshot only if every file entry carries that report's sessionId; a report merged from several runs is used as is. Only authoritative runs (no --since, no subject patterns, no survivor rerun) add a row to the trend history.

A # henitai:disable magic comment excludes code from mutation at the call site, without touching .henitai.yml:

def risky_calc(x)
  x * 2 # henitai:disable            — skips mutants on this line
end

# henitai:disable                     — skips every mutant in this method
def legacy_shim(x)
  x - 1
end

The line form applies to mutants starting on that line. The method form is a standalone comment in the contiguous comment block directly above a def (a blank line breaks the association). Trailing text is allowed (# henitai:disable -- reviewed defensive branch). Skipped mutants are reported as Ignored, so they stay visible in the report instead of silently vanishing. mutation.ignore_patterns is matched against the source line where each mutant's node starts, so only mutants on a matching line are reported as Ignored; an enclosing if or block whose body matches stays mutated. Use mutation.ignore_patterns for repo-wide policy and # henitai:disable for one-off, reviewed exclusions.

Directives can also target specific operators, carry a reason (serialized as statusReason, visible in the HTML report), and cover regions:

x = a + b  # henitai:disable ArithmeticOperator            — only this operator
x = a + b  # henitai:disable ArithmeticOperator: log noise — with a reason

# henitai:disable RegexMutator: timing-sensitive matcher
def parse(line)
  ...
end

# henitai:disable-start ConditionalExpression
...region...
# henitai:disable-end

Operator names must exactly match the canonical names from henitai operator list; unknown names, unmatched or nested disable-start/disable-end directives abort the run (exit 2) with the offending file and line. Regions do not nest.

The repository ships a JSON Schema at assets/schema/henitai.schema.json for editor autocompletion.

Light (default) — high-signal, low-noise operators covering the majority of real-world defects:

• ArithmeticOperator — + ↔ -, * ↔ /
• EqualityOperator — == ↔ !=, > ↔ <, etc. (relational operators only)
• LogicalOperator — && ↔ ||
• BooleanLiteral — true ↔ false, !expr
• ConditionalExpression — remove branch bodies
• StringLiteral — empty string replacement
• ReturnValue — mutate return expressions

Full — adds lower-signal operators:

• ArrayDeclaration, HashLiteral, RangeLiteral
• MethodChainUnwrap — remove a link from a method chain
• RegexMutator — mutate regexp quantifiers, anchors, char-class negation
• SafeNavigation — &. → .
• PatternMatch — case/in arm removal
• BlockStatement — remove blocks
• MethodExpression — remove calls
• AssignmentExpression — mutate compound assignment
• UnaryOperator — remove unary - and ~
• UpdateOperator — swap compound assignments (+=↔-=, *=↔/=, ||=↔&&=)

Hard — adds usually-unkillable mutations on top of full, for hunting the last survivors (see ADR-12):

• EqualityIdentityOperator — == ↔ eql?/equal? (hardest equality pairing to kill; see ADR-10)
• HashKeyType — { a: 1 } → { "a" => 1 } (frameworks that normalize key types, e.g. ActiveRecord order/where, make these mutants equivalent at many call sites; disable per site with # henitai:disable HashKeyType)

HashLiteral (full set) empties the hash and removes one pair at a time ({ a: 1, b: 2 } → {} / { b: 2 }).

# .henitai.yml
reporters:
  - terminal
  - html
  - json
  - dashboard

dashboard:
  project: "github.com/your-org/your-repo"
  base_url: "https://dashboard.stryker-mutator.io"

Set STRYKER_DASHBOARD_API_KEY in your CI environment to publish reports. When the key, project, and version are present, the dashboard reporter uploads the Stryker-schema report to the dashboard REST API (default https://dashboard.stryker-mutator.io); otherwise it is skipped silently. Scoped runs upload the merged canonical report rather than their partial one. The project defaults to the git remote and the version to the current branch or CI ref (GITHUB_REF_NAME/GITHUB_REF/GITHUB_SHA).

JSON reports are written to reports/mutation-report.json by default. Set reports_dir to change the output directory.

git clone https://github.com/martinotten/henitai
cd henitai
bundle install
bundle exec rspec        # run tests
bundle exec ruby bin/verify-process-free-specs # verify self-mutation-safe specs
bundle exec rake smoke:integration:all # run rspec/minitest integration smoke projects
bundle exec rake oracle # measure false kills/survivors on the oracle corpus
bundle exec rake oracle:reuse # check --incremental verdicts equal fresh runs
bundle exec rubocop      # lint
bundle exec henitai clean # remove stale generated report artifacts
bundle exec henitai run  # dogfood

Framework integration smoke projects live under spec/fixtures/integration_smoke/ and exercise henitai against small RSpec and Minitest apps via the local path dependency.

Git hook support is tracked in .githooks/pre-commit. Enable it with git config core.hooksPath .githooks so commits run RuboCop, RSpec, and the integration smoke suite before they are created.

A Dev Container configuration is included (.devcontainer/) for VS Code with the official ruby:4 image, the Codex CLI, and RTK preinstalled. Codex support is bootstrapped with rtk init -g --codex --auto-patch during container creation.

See docs/architecture/architecture.md for the full design document, including:

• Phase-Gate pipeline (5 gates)
• AST-based operator implementation
• Fork isolation model
• Stryker JSON schema integration
• Architecture decisions in docs/architecture/adr/
• Three-phase roadmap

MIT License — © 2026 Martin Otten