🥘 Kettle::Soup::Cover

Four lines of code to get a configured, curated, opinionated, set of dependencies for Test Coverage, and that's including the two lines for require "simplecov", and SimpleCov.start.

Configured for what? To work out of the box on every CI*. Batteries included. For apps and libraries. Any test framework. Many code coverage related GitHub Actions (example configs 1, 2).

This library's local dev / testing / CI dependency structure serves as an example of a "modular gemfile" pattern enabling a discrete gemfile for each CI workflow.

This modular pattern has the following benefits:

• All dependencies are DRY, never repeated.
• All modular gemfiles are shared between the main Gemfile, and the workflow gemfiles/*.gemfiles that need them.
• All gemfiles source from the gemspec.

For another example of this pattern see the kettle-soup-cover gem.

If you like this idea, there is an even better alternative.

I've codified it for reuse in my appraisal2 gem, which is a hard fork of the venerable appraisal gem due to that gem's lack of support for modular gemfiles.

📔 ME TOO! 📔

One of the major benefits of using this library is not having to figure out how to get multiple coverage output formats working. I did that for you, and I got all of them working, at the same time together, or al la carte. Kum-ba-ya.

A quick shot of 12-factor coverage power, straight to your brain:

export K_SOUP_COV_DO=true # Means you want code coverage
export K_SOUP_COV_FORMATTERS="html,tty" # Set to some slice of "html,xml,rcov,lcov,json,tty"
export K_SOUP_COV_MIN_BRANCH=53 # Means you want to enforce X% branch coverage
export K_SOUP_COV_MIN_HARD=true # Means you want the build to fail if the coverage thresholds are not met
export K_SOUP_COV_MIN_LINE=69 # Means you want to enforce X% line coverage
export MAX_ROWS=5 # Setting for simplecov-console gem for tty output, limits to the worst N rows of bad coverage

I hope I've piqued your interest enough to give it a ⭐️ if the forge you are on supports it.

A Covered Kettle of SOUP (Software of Unknown Provenance)

The name is derived in part from the medical devices field, where this library is considered a package of SOUP.

Federated DVCS Repository	Status	Issues	PRs	Wiki	CI	Discussions
🧪 kettle-rb/kettle-soup-cover on GitLab	The Truth	💚	💚	💚	🏀 Tiny Matrix	➖
🧊 kettle-rb/kettle-soup-cover on CodeBerg	An Ethical Mirror (Donate)	💚	💚	➖	⭕️ No Matrix	➖
🐙 kettle-rb/kettle-soup-cover on GitHub	A Dirty Mirror	💚	💚	➖	💯 Full Matrix	💚
🎮️ Discord Server		Let's	talk	about	this	library!

• 💡Subscribe for support guarantees covering all FLOSS dependencies
• 💡Tidelift is part of Sonar
• 💡Tidelift pays maintainers to maintain the software you depend on!
  📊@Pointy Haired Boss: An enterprise support subscription is "never gonna let you down", and supports open source maintainers

Alternatively:

Tokens to Remember
Works with MRI Ruby 3
Works with MRI Ruby 2
Source
Documentation
Compliance
Style
Support
Maintainer 🎖️
... 💖	🧊 🐙 🛖 🧪

This tool leverages other tools to make hard things easier, but sometimes those other tools break... I'll try to track that here.

If you find this working/not working different than above please open an issue / PR!

This gem does not add coverage parsing to CI's that don't have it, since that's impossible. Vendor-specific formats which are not shared by other vendors are also not supported (e.g. BuildKite).

You'll have to configure them manually if you use them:

• BuildKite's custom simplecov extension
• GitHub Actions doesn't parse test output, but...
  • I configure my coverage workflow (see example) to upload coverage reports to SaaS services like:
    • codecov.io (has tokenless OIDC option!)
    • QLTY.sh (needs token for upload)
    • coveralls.io
  • This gem helps me configure my coverage workflow to use Github Actions designed to report coverage like:
    • Repo: irongut/CodeCoverageSummary
    • Repo: marocchino/sticky-pull-request-comment

This library is based on ideas I originally introduced in the gem rspec-stubbed_env.

Install the gem and add to the application's Gemfile by executing:

$ bundle add kettle-soup-cover

If bundler is not being used to manage dependencies, install the gem by executing:

$ gem install kettle-soup-cover

This gem is cryptographically signed, and has verifiable SHA-256 and SHA-512 checksums by stone_checksums. Be sure the gem you install hasn’t been tampered with by following the instructions below.

Add my public key (if you haven’t already, expires 2045-04-29) as a trusted certificate:

gem cert --add <(curl -Ls https://raw.github.com/galtzo-floss/certs/main/pboling.pem)

You only need to do that once. Then proceed to install with:

gem install kettle-soup-cover -P MediumSecurity

The MediumSecurity trust profile will verify signed gems, but allow the installation of unsigned dependencies.

If you want to up your security game full-time:

bundle config set --global trust-policy MediumSecurity

MediumSecurity instead of HighSecurity is necessary if not all the gems you use are signed.

NOTE: Be prepared to track down certs for signed gems and add them the same way you added mine.

In your spec/spec_helper.rb or tests/test_helper.rb, just before loading the library under test, add two lines of code:

require "kettle-soup-cover"
require "simplecov" if Kettle::Soup::Cover::DO_COV # `.simplecov` is run here!
# IMPORTANT: If you are using MiniTest instead of RSpec, also do this (and not in .simplecov):
# SimpleCov.external_at_exit = true

In your spec/rails_helper.rb

# External gems
require "kettle-soup-cover"

# This file is copied to spec/ when you run 'rails generate rspec:install'
# We provide a preconfigured version compatible with Rails 8
require "spec_helper"
ENV["RAILS_ENV"] ||= "test"

# Last thing before loading the app-under-test is code coverage.
require "simplecov" if Kettle::Soup::Cover::DO_COV # `.simplecov` is run here!
require File.expand_path("../config/environment", __dir__)

P.S. Ensure that you have require: false on the gem in the Gemfile, and that it is in both :development and :test groups, since it ships a coverage rake task:

group :development, :test do
  gem "kettle-soup-cover", "~> 1.0", ">= 1.0.10", require: false
end

# NOTE: Gemfiles for older rubies won't have kettle-soup-cover.
#       The rescue LoadError handles that scenario.
begin
  require "kettle-soup-cover"

  if Kettle::Soup::Cover::DO_COV
    require "simplecov" # `.simplecov` is run here!

    # IMPORTANT: If you are using MiniTest instead of RSpec, also do this (and not in .simplecov):
    # SimpleCov.external_at_exit = true
  end
rescue LoadError => error
  # check the error message, if you are so inclined, and re-raise if not what is expected
  raise error unless error.message.include?("kettle")
end

In your .simplecov file, add 2 lines of code:

require "kettle/soup/cover/config" # 12-factor, ENV-based configuration, with good defaults!
# you could do this somewhere else, up to you, but you do have to do it somewhere
SimpleCov.start

See Advanced Usage below for more info, but the simplest thing is to run all the coverage things, which is configured by default on CI. To replicate that locally you could:

CI=true bundle exec rake test # or whatever command you run for tests.

That's it!

You'll need to have your test task defined. If you use spec instead, you can make it a pre-requisite of the test task with:

desc "run spec task with test task"
task test: :spec

This gem provides a coverage task. It runs the test task (see just above about that), and opens the coverage results in a browser.

require "kettle-soup-cover"
Kettle::Soup::Cover.install_tasks

There are two built-in SimpleCov filters which can be loaded via Kettle::Soup::Cover.load_filters.

You could use them like this:

SimpleCov.add_group("Too Long", Kettle::Soup::Cover::Filters::GtLineFilter.new(1000))

There are a number of ENV variables that control things within this gem. All of them can be found, along with their default values, in lib/kettle/soup/cover.rb.

Most are self explanatory. I tried to follow POLS, the principle of least surprise, so they mostly DWTFYT. Want to help improve this documentation? PRs are easy!

K_SOUP_COV_COMMAND_NAME
K_SOUP_COV_DEBUG
K_SOUP_COV_DIR
K_SOUP_COV_DO
K_SOUP_COV_FILTER_DIRS
K_SOUP_COV_FORMATTERS
K_SOUP_COV_MERGE_TIMEOUT
K_SOUP_COV_MIN_BRANCH
K_SOUP_COV_MIN_HARD
K_SOUP_COV_MIN_LINE
K_SOUP_COV_MULTI_FORMATTERS
K_SOUP_COV_PREFIX
K_SOUP_COV_OPEN_BIN
K_SOUP_COV_USE_MERGING
K_SOUP_COV_VERBOSE

Additionally, some of the included gems, like simplecov-console, have their own complete suite of ENV variables you can configure.

If you don't want to configure a SaaS service to update your pull requests with code coverage there are alternatives.

After the step that runs your test suite use one or more of the following.

Repo: irongut/CodeCoverageSummary

      - name: Code Coverage Summary Report
        uses: irongut/CodeCoverageSummary@v1.3.0
        if: ${{ github.event_name == 'pull_request' }}
        with:
          filename: ./coverage/coverage.xml
          badge: true
          fail_below_min: true
          format: markdown
          hide_branch_rate: false
          hide_complexity: true
          indicators: true
          output: both
          thresholds: '100 100' # '<MIN LINE COVERAGE> <MIN BRANCH COVERAGE>'
        continue-on-error: ${{ matrix.experimental != 'false' }}

Repo: marocchino/sticky-pull-request-comment

      - name: Add Coverage PR Comment
        uses: marocchino/sticky-pull-request-comment@v2
        if: ${{ github.event_name == 'pull_request' }}
        with:
          recreate: true
          path: code-coverage-results.md
        continue-on-error: ${{ matrix.experimental != 'false' }}

See SECURITY.md.

If you need some ideas of where to help, you could work on adding more code coverage, or if it is already 💯 (see below), check the REEK list, GitHub issues, GitLab issues, CodeBerg issues, or use the gem and think about how it could be better.

We so if you make changes, remember to update it.

See CONTRIBUTING.md for more detailed instructions.

See CONTRIBUTING.md.

Everyone interacting with this project's codebases, issue trackers, chat rooms and mailing lists agrees to follow the .

Made with contributors-img.

Also see GitLab Contributors: https://gitlab.com/kettle-rb/kettle-soup-cover/-/graphs/main

This Library adheres to . Violations of this scheme should be reported as bugs. Specifically, if a minor or patch version is released that breaks backward compatibility, a new version should be immediately released that restores compatibility. Breaking changes to the public API will only be introduced with new major versions.

Yes. But I'm obligated to include notes...

SemVer should, but doesn't explicitly, say that dropping support for specific Platforms is a breaking change to an API. It is obvious to many, but not all, and since the spec is silent, the bike shedding is endless.

dropping support for a platform is both obviously and objectively a breaking change

• Jordan Harband (@ljharb, maintainer of SemVer) in SemVer issue 716

To get a better understanding of how SemVer is intended to work over a project's lifetime, read this article from the creator of SemVer:

• "Major Version Numbers are Not Sacred"

As a result of this policy, and the interpretive lens used by the maintainer, you can (and should) specify a dependency on these libraries using the Pessimistic Version Constraint with two digits of precision.

For example:

spec.add_dependency("kettle-soup-cover", "~> 1.0")

See CHANGELOG.md for a list of releases.

The gem is available as open source under the terms of the MIT License . See LICENSE.txt for the official Copyright Notice.

• Copyright (c) 2023-2025 Peter H. Boling, of Galtzo.com , and kettle-soup-cover contributors

Having arrived at the bottom of the page, please endure a final supplication. The primary maintainer of this gem, Peter Boling, wants Ruby to be a great place for people to solve problems, big and small. Please consider supporting his efforts via the giant yellow link below, or one of the smaller ones, depending on button size preference.

P.S. If you need help️ or want to say thanks, 👇 Join the Discord.