split-test-rb

A simple Ruby CLI tool to balance RSpec tests across parallel CI nodes using RSpec JSON reports.

split-test-rb reads RSpec JSON test reports containing execution times and distributes test files across multiple nodes for parallel execution. It uses a greedy algorithm to ensure balanced distribution based on historical test execution times.

Add to your Gemfile:

gem 'split-test-rb'

Then run:

bundle install

First, add split-test-rb to your Gemfile:

# Gemfile
gem 'split-test-rb'

For a working example, see this project's own CI configuration:

• .github/workflows/ci.yml

split-test-rb [options]

Options:
  --node-index INDEX          Current node index (0-based)
  --node-total TOTAL          Total number of nodes
  --json-path PATH            Path to directory containing RSpec JSON reports (required)
  --test-dir DIR              Test directory (default: spec)
  --test-pattern PATTERN      Test file pattern (default: **/*_spec.rb)
  --split-by-example-threshold SECONDS
                              Split files with execution time >= threshold into individual examples
  --dry-run-json PATH         RSpec JSON from `rspec --dry-run --format json`, used to list current examples of heavy files
  --debug                     Show debug information
  -h, --help                  Show help message

By default, split-test-rb looks for test files in the spec/ directory with the pattern **/*_spec.rb. You can customize this for projects with different test directory structures:

Using Minitest with test/ directory:

split-test-rb --json-path tmp/test-results \
  --node-index $CI_NODE_INDEX \
  --node-total $CI_NODE_TOTAL \
  --test-dir test \
  --test-pattern '**/*_test.rb'

Custom test directory structure:

split-test-rb --json-path tmp/test-results \
  --node-index 0 \
  --node-total 4 \
  --test-dir tests \
  --test-pattern 'unit/**/*.rb'

The test directory and pattern options are useful for:

• Projects using Minitest (test/ directory)
• Custom test directory structures
• Different naming conventions for test files
• Monorepos with multiple test suites

When you have test files that take significantly longer than others, you can use --split-by-example-threshold to automatically split them into individual RSpec examples. This enables finer-grained load balancing across CI nodes.

bundle exec rspec --dry-run --format json --out tmp/dry-run.json
split-test-rb --json-path tmp/test-results \
  --node-index $CI_NODE_INDEX \
  --node-total $CI_NODE_TOTAL \
  --split-by-example-threshold 10.0 \
  --dry-run-json tmp/dry-run.json

--dry-run-json is recommended together with this option. Without it, examples missing from the cached JSON are not run (see Keeping heavy files up to date with --dry-run-json).

With this option:

• Files with execution time below the threshold are distributed as whole files (e.g., spec/fast_spec.rb)
• Files with execution time at or above the threshold are split into individual examples (e.g., spec/slow_spec.rb[1:1], spec/slow_spec.rb[1:2])

This is useful when:

• A single test file contains many slow examples that dominate a CI node's runtime
• You want to maximize parallelization without manually splitting large test files
• Some test files are bottlenecks that prevent even distribution

Note: The JSON report must contain the id field for each example (RSpec's default JSON formatter includes this). The tool uses these IDs to generate the example-specific paths that RSpec can run.

Without --dry-run-json, heavy files are assigned only by the example IDs found in the cached JSON reports, and a warning is printed. Examples that are not in the cache are not assigned to any node:

• Examples newly added to a heavy file
• Examples whose IDs shifted because other examples were inserted or moved (RSpec example IDs are position based, e.g. [1:2:1])

Pass the output of rspec --dry-run --format json to use the examples that currently exist (see the example above).

Every node must use a dry-run JSON with the same examples, or the nodes compute different splits and some tests are run on no node. Generating it once (e.g. in a setup job) and sharing it with all nodes as a CI artifact guarantees this, and boots the app only once instead of on every node.

With this option, for each heavy file:

• Examples in the dry-run JSON are assigned, using cached timings when available and the default timing (1.0s) otherwise
• Cached example IDs that no longer exist are ignored
• If the dry-run JSON has no examples for the file, the whole file is assigned

1. Parse RSpec JSON: Extracts test file paths and execution times from the JSON report
2. Greedy Balancing: Sorts files by execution time (descending) and assigns each file to the node with the lowest cumulative time
3. Output: Prints the list of test files for the specified node

split-test-rb provides intelligent fallback handling to ensure tests can run even without historical timing data:

If the specified JSON file is not found, the tool will:

• Display a warning: Warning: JSON directory not found: <path>, using all test files with equal execution time
• Find all test files matching the specified directory and pattern (default: spec/**/*_spec.rb)
• Assign equal execution time (1.0 seconds) to each file
• Distribute them evenly across nodes

This is useful for:

• First-time runs when no test history exists yet
• Local development environments
• New CI pipelines

If new test files exist that aren't in the JSON report, the tool will:

• Display a warning: Warning: Found N test files not in JSON, adding with default execution time
• Add the missing files with default execution time (1.0 seconds)
• Include them in the distribution

This ensures newly added test files are always included in the test run.

The tool expects RSpec JSON output format (generated with --format json):

{
  "examples": [
    {
      "file_path": "./spec/models/user_spec.rb",
      "run_time": 1.234
    },
    {
      "file_path": "./spec/models/post_spec.rb",
      "run_time": 0.567
    }
  ]
}

To generate JSON reports with RSpec, use the built-in JSON formatter:

bundle exec rspec --format json --out tmp/rspec-results/results.json

To release a new version, use the Release workflow:

1. Go to the Actions tab
2. Click "Run workflow"
3. Select the version bump type (patch/minor/major)
4. Click "Run workflow"

The workflow will automatically update lib/split_test_rb/version.rb, commit it to main, create a git tag (v*), publish the gem to RubyGems via trusted publishing, and create a GitHub Release.

MIT