๐ฅ Kettle::Soup::Cover
Four lines of code to get a configured, curated, opinionated, set of dependencies for Test Coverage, and that's including the two lines for require "simplecov"
, and SimpleCov.start
.
Configured for what? To work out of the box on every CI*. Batteries included. For apps and libraries. Any test framework. Many code coverage related GitHub Actions (example configs 1, 2).
Test Framework | Helper | Config |
---|---|---|
MiniTest | test helper | .simplecov |
RSpec | spec helper | .simplecov |
Federated DVCS Repository | Status | Issues | PRs | Wiki | CI | Discussions |
---|---|---|---|---|---|---|
๐งช kettle-rb/kettle-soup-cover on GitLab | The Truth | ๐ | ๐ | ๐ | ๐ Tiny Matrix | โ |
๐ง kettle-rb/kettle-soup-cover on CodeBerg | An Ethical Mirror (Donate) | โ | ๐ | โ | โญ๏ธ No Matrix | โ |
๐ kettle-rb/kettle-soup-cover on GitHub | A Dirty Mirror | ๐ | ๐ | โ | ๐ฏ Full Matrix | ๐ |
๐ฎ๏ธ Discord Server | Let's | talk | about | this | library! |
One of the major benefits of using this library is not having to figure out how to get multiple coverage output formats working. I did that for you, and I got all of them working, at the same time together, or al la carte. Kum-ba-ya.
A quick shot of 12-factor coverage power, straight to your brain:
export K_SOUP_COV_DO=true # Means you want code coverage
export K_SOUP_COV_FORMATTERS="html,tty" # Set to some slice of "html,xml,rcov,lcov,json,tty"
export K_SOUP_COV_MIN_BRANCH=53 # Means you want to enforce X% branch coverage
export K_SOUP_COV_MIN_HARD=true # Means you want the build to fail if the coverage thresholds are not met
export K_SOUP_COV_MIN_LINE=69 # Means you want to enforce X% line coverage
export MAX_ROWS=5 # Setting for simplecov-console gem for tty output, limits to the worst N rows of bad coverage
I hope I've piqued your interest enough to give it a โญ๏ธ if the forge you are on supports it.
A Covered Kettle of SOUP (Software of Unknown Provenance)
The name is derived in part from the medical devices field, where this library is considered a package of SOUP.
Format / Library x CI Matrix
This tool leverages other tools to make hard things easier, but sometimes those other tools break... I'll try to track that here.
Format | Library | Status | Web | Circle CI |
Git Lab |
Travis CI |
Jenkins X |
Jenkins | Hudson | Semaphore | Bit Bucket |
Team City |
๐ค Nerds |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
html |
simplecov-html |
โ | โ | โ | |||||||||
xml |
simplecov-cobertura |
โ ๏ธ works (with warnings); upvote #30! | โ | โ | โ | ||||||||
rcov |
simplecov-rcov |
โ | โ | โ | |||||||||
lcov |
simplecov-lcov |
โ | โ | โ | โ | โ | โ | โ | |||||
json |
simplecov_json_formatter |
โ | โ | โ | โ | โ | โ | ||||||
tty |
simplecov-console |
โ | โ |
If you find this working/not working different than above please open an issue / PR!
โ๏ธ Not actually every CI
This gem does not add coverage parsing to CI's that don't have it, since that's impossible. Vendor-specific formats which are not shared by other vendors are also not supported (e.g. BuildKite).
You'll have to configure them manually if you use them:
- BuildKite's custom simplecov extension
- GitHub Actions doesn't parse test output, but...
- I configure my
coverage
workflow (see example) to upload coverage reports to SaaS services like:- codecov.io (needs token for upload)
- QLTY.sh (needs token for upload)
- coveralls.io
- This gem helps me configure my
coverage
workflow to use Github Actions designed to report coverage like:
- I configure my
This library is based on ideas I originally introduced in the gem rspec-stubbed_env.
๐ก Info you can shake a stick at
Tokens to Remember |
|
---|---|
Works with MRI Ruby 3 |
|
Works with MRI Ruby 2 | |
Source |
|
Documentation |
|
Compliance |
|
Style |
|
Support |
|
Enterprise Support |
๐กSubscribe for support guarantees covering all FLOSS dependencies! ๐กTidelift is part of Sonar! ๐กTidelift pays maintainers to maintain the software you depend on! ๐ @ Pointy Haired Boss: An enterprise support subscription is "never gonna let you down", and supports open source maintainers! |
Comrade BDFL ๐๏ธ |
|
... ๐ |
|
โจ Installation
Install the gem and add to the application's Gemfile by executing:
$ bundle add kettle-soup-cover
If bundler is not being used to manage dependencies, install the gem by executing:
$ gem install kettle-soup-cover
๐ Secure Installation
kettle-soup-cover
is cryptographically signed, and has verifiable SHA-256 and SHA-512 checksums by
stone_checksums. Be sure the gem you install hasnโt been tampered with
by following the instructions below.
Add my public key (if you havenโt already, expires 2045-04-29) as a trusted certificate:
gem cert --add <(curl -Ls https://raw.github.com/kettle-rb/kettle-soup-cover/main/certs/pboling.pem)
You only need to do that once. Then proceed to install with:
gem install kettle-soup-cover -P MediumSecurity
The MediumSecurity
trust profile will verify signed gems, but allow the installation of unsigned dependencies.
This is necessary because not all of kettle-soup-cover
โs dependencies are signed, so we cannot use HighSecurity
.
If you want to up your security game full-time:
bundle config set --global trust-policy MediumSecurity
NOTE: Be prepared to track down certs for signed gems and add them the same way you added mine.
๐ง Basic Usage
RSpec or MiniTest
In your spec/spec_helper.rb
or tests/test_helper.rb
, just before loading the library under test,
add two lines of code:
With Ruby 2.7+
require "kettle-soup-cover"
require "simplecov" if Kettle::Soup::Cover::DO_COV # `.simplecov` is run here!
# IMPORTANT: If you are using MiniTest instead of RSpec, also do this (and not in .simplecov):
# SimpleCov.external_at_exit = true
Projects that run tests against older Ruby versions, e.g. with Appraisals
# NOTE: Gemfiles for older rubies won't have kettle-soup-cover.
# The rescue LoadError handles that scenario.
begin
require "kettle-soup-cover"
if Kettle::Soup::Cover::DO_COV
require "simplecov" # `.simplecov` is run here!
# IMPORTANT: If you are using MiniTest instead of RSpec, also do this (and not in .simplecov):
# SimpleCov.external_at_exit = true
end
rescue LoadError => error
# check the error message, if you are so inclined, and re-raise if not what is expected
raise error unless error.message.include?("kettle")
end
All projects
In your .simplecov
file, add 2 lines of code:
require "kettle/soup/cover/config" # 12-factor, ENV-based configuration, with good defaults!
# you could do this somewhere else, up to you, but you do have to do it somewhere
SimpleCov.start
See Advanced Usage below for more info, but the simplest thing is to run all the coverage things, which is configured by default on CI. To replicate that locally you could:
CI=true bundle exec rake test # or whatever command you run for tests.
That's it!
Rakefile
You'll need to have your test
task defined.
If you use spec
instead, you can make it a pre-requisite of the test
task with:
desc "run spec task with test task"
task test: :spec
This gem provides a coverage
task.
It runs the test
task (see just above about that),
and opens the coverage results in a browser.
require "kettle-soup-cover"
Kettle::Soup::Cover.install_tasks
Filters
There are two built-in SimpleCov filters which can be loaded via Kettle::Soup::Cover.load_filters
.
You could use them like this:
SimpleCov.add_group("Too Long", Kettle::Soup::Cover::Filters::GtLineFilter.new(1000))
Advanced Usage
There are a number of ENV variables that control things within this gem. All of them can be found, along with their default values, in lib/kettle/soup/cover.rb.
Handy List of ENV Variables
Most are self explanatory. I tried to follow POLS, the principle of least surprise, so they mostly DWTFYT. Want to help improve this documentation? PRs are easy!
K_SOUP_COV_COMMAND_NAME
K_SOUP_COV_DEBUG
K_SOUP_COV_DIR
K_SOUP_COV_DO
K_SOUP_COV_FILTER_DIRS
K_SOUP_COV_FORMATTERS
K_SOUP_COV_MERGE_TIMEOUT
K_SOUP_COV_MIN_BRANCH
K_SOUP_COV_MIN_HARD
K_SOUP_COV_MIN_LINE
K_SOUP_COV_MULTI_FORMATTERS
K_SOUP_COV_PREFIX
K_SOUP_COV_OPEN_BIN
K_SOUP_COV_USE_MERGING
K_SOUP_COV_VERBOSE
Additionally, some of the included gems, like simplecov-console
,
have their own complete suite of ENV variables you can configure.
Compatible with GitHub Actions for Code Coverage feedback in pull requests
If you don't want to configure a SaaS service to update your pull requests with code coverage there are alternatives.
After the step that runs your test suite use one or more of the following.
irongut/CodeCoverageSummary
Repo: irongut/CodeCoverageSummary
- name: Code Coverage Summary Report
uses: irongut/CodeCoverageSummary@v1.3.0
if: ${{ github.event_name == 'pull_request' }}
with:
filename: ./coverage/coverage.xml
badge: true
fail_below_min: true
format: markdown
hide_branch_rate: false
hide_complexity: true
indicators: true
output: both
thresholds: '100 100' # '<MIN LINE COVERAGE> <MIN BRANCH COVERAGE>'
continue-on-error: ${{ matrix.experimental != 'false' }}
marocchino/sticky-pull-request-comment
Repo: marocchino/sticky-pull-request-comment
- name: Add Coverage PR Comment
uses: marocchino/sticky-pull-request-comment@v2
if: ${{ github.event_name == 'pull_request' }}
with:
recreate: true
path: code-coverage-results.md
continue-on-error: ${{ matrix.experimental != 'false' }}
๐ Release Instructions
See CONTRIBUTING.md.
๐ Security
See SECURITY.md.
๐ค Contributing
If you need some ideas of where to help, you could work on adding more code coverage, or if it is already ๐ฏ (see below) check issues, or PRs, or use the gem and think about how it could be better.
We so if you make changes, remember to update it.
See CONTRIBUTING.md for more detailed instructions.
Code Coverage
๐ช Code of Conduct
Everyone interacting in this project's codebases, issue trackers,
chat rooms and mailing lists is expected to follow the .
๐ Contributors
Made with contributors-img.
Also see GitLab Contributors: https://gitlab.com/kettle-rb/kettle-soup-cover/-/graphs/main
โญ๏ธ Star History
๐ Versioning
This Library adheres to .
Violations of this scheme should be reported as bugs.
Specifically, if a minor or patch version is released that breaks backward compatibility,
a new version should be immediately released that restores compatibility.
Breaking changes to the public API will only be introduced with new major versions.
๐ Is "Platform Support" part of the public API?
Yes. But I'm obligated to include notes...
SemVer should, but doesn't explicitly, say that dropping support for specific Platforms is a breaking change to an API. It is obvious to many, but not all, and since the spec is silent, the bike shedding is endless.
dropping support for a platform is both obviously and objectively a breaking change
- Jordan Harband (@ljharb, maintainer of SemVer) in SemVer issue 716
To get a better understanding of how SemVer is intended to work over a project's lifetime, read this article from the creator of SemVer:
As a result of this policy, and the interpretive lens used by the maintainer, you can (and should) specify a dependency on these libraries using the Pessimistic Version Constraint with two digits of precision.
For example:
spec.add_dependency("kettle-soup-cover", "~> 1.0")
See CHANGELOG.md for list of releases.
๐ License
The gem is available as open source under the terms of
the MIT License .
See LICENSE.txt for the official Copyright Notice.
ยฉ Copyright
Copyright (c) 2023 - 2025 Peter H. Boling,
RailsBling.com
๐ค One more thing
You made it to the bottom of the page, so perhaps you'll indulge me for another 20 seconds. I maintain many dozens of gems, including this one, because I want Ruby to be a great place for people to solve problems, big and small. Please consider supporting my efforts via the giant yellow link below, or one of the others at the head of this README.