Ruby Raider
Explore the docs
Rubygems
·
Report Bug
·
Request Feature
For more information and updates on releases, see https://ruby-raider.onrender.com/
What is Ruby Raider?
Ruby Raider is a CLI gem for scaffolding and generating UI test automation frameworks. It supports interactive command-line usage and flag-driven invocation (for example raider new my_project -p framework:rspec automation:selenium), which is how Raider Desktop drives it.
Supported Frameworks
Ruby Raider works on Mac OS, Linux and Windows.
Web Testing
Web projects use Selenium and RSpec. They get:
- Locators people see on the page (
label:,text:,by_role(...)and more), with strict matching - Automatic waiting for elements and actions: no implicit wait, no
sleep - Retrying RSpec matchers
- Login reuse, failure video, failure trace, network mocking and screenshot comparison
- Optional accessibility tests (axe) and a choice of GitHub Actions or Azure Pipelines
See What's Included for each one.
Mobile Testing
Mobile projects use Appium and RSpec:
| Test Framework | iOS | Android | Cross-Platform |
|---|---|---|---|
| RSpec | ✅ | ✅ | ✅ |
They get:
- Appium locators:
accessibility_id:,id:,xpath:andclass_name: - Automatic waiting:
find_elementwaits for a match, andclick,send_keysandclearwait until the element is displayed and enabled - Retrying RSpec matchers
- Gestures:
swipe,scrollandscroll_to, on iOS and Android - Failure video from Appium's screen recording
- Raider reporting and a GitHub Actions workflow that starts a simulator or emulator and Appium
On native app screens, find_element returns the first match (strict matching is web only) and submit is not checked.
To run the examples, download the example app and start the server:
raider u start_appiumOptional Add-ons
| Add-on | Flag | Description |
|---|---|---|
| Accessibility (web) | --accessibility |
Adds the axe gem and example accessibility tests |
| CI provider (web) | -p ci:azure |
Generates azure-pipelines.yml instead of the GitHub Actions workflow (default ci:github) |
What's Included
Every generated project loads the Raider runtime (ruby_raider/runtime). It adds locators, waiting and assertions to Selenium and Appium themselves.
Locators
Find elements the way a person sees them, on top of Selenium's id:, css:, xpath:, name:, class: and link_text::
| Key | Finds |
|---|---|
label: |
the field with that label, aria-label or aria-labelledby
|
placeholder: |
the field with that placeholder |
alt: |
the element with that alt text |
title: |
the element with that title
|
text: |
the innermost element whose text contains it |
test_id: |
the element whose data-testid equals it |
by_role(...) |
the element with that ARIA role and name |
driver.find_element(label: 'Email')
driver.find_element(label: ['Email', { exact: true }]) # whole text only
driver.find_element(by_role('button', name: 'Save'))
driver.find_element(by_role('heading', level: 1))- Text keys match part of the text and ignore extra spaces. Pass
{ exact: true }for the whole text. -
test_id:reads another attribute withRaider.test_id_attribute = 'data-qa'orRAIDER_TEST_ID_ATTRIBUTE. -
Strict matching:
find_elementfails withRaider::AmbiguousMatchwhen a locator matches more than one element.find_elementsnever does. Turn it off withraider u strict off.
These keys are for web pages. Mobile projects use Appium's locators (see Mobile Testing).
Waiting and actionability
-
find_elementandfind_elementswait for a match. There is no implicit wait and nosleep. - Before an action, the runtime waits until the element is ready:
| Action | Waits until the element is |
|---|---|
click |
visible, enabled, not inert, stable and not covered |
send_keys, clear
|
visible, enabled, editable and not inert |
submit |
visible, enabled and not inert |
-
force: trueskips the checks for one call:button.click(force: true). - Set the rules with
timeout,strictandactionabilityinconfig/config.yml, orRAIDER_TIMEOUT,RAIDER_STRICTandRAIDER_ACTIONABILITY.
Assertions
RSpec matchers retry until the page matches or timeout runs out:
have_text, have_value, have_attribute, have_css, have_xpath, have_element, have_count, have_title, have_url, be_visible, be_hidden, be_enabled, be_disabled, be_checked
expect { account_page.welcome_message }.to have_text('Welcome')
expect { driver.find_element(id: 'status').text }.to eq('done') # any value matcher retries in a blockRaider reporting
Projects include raider_reporter.
- Results go to
raider-results/. - Runs upload to your Raider Hideout server when
RAIDER_TOKENandRAIDER_ENDPOINTare set. - CI runs the suite through
raider-reporter execwithRAIDER_QUARANTINE, so quarantined tests do not fail the build. - Set the CI variable
RAIDER_GATEto1to add araider-gatejob that fails CI when the Raider quality gate fails.
Failure video replay
Each test is recorded. The video is kept only when the test fails and attached to the report as video.
-
Web: WebM built from page screenshots. ffmpeg is downloaded once, SHA-256 checked and cached in
~/.cache/raider: an LGPL ffmpeg 7.1 build with libvpx for macOS (Apple silicon and Intel), Linux x64 and Windows x64, built by.github/workflows/ffmpeg.yml. On other platforms setRAIDER_FFMPEG_PATHto an ffmpeg with libvpx. -
Mobile: Appium's own MP4. iOS needs
ffmpegon the Appium server host. Android stops at 180 s by default, so it needs no ffmpeg; a longervideo.time_limitneeds ffmpeg too. - Turn it off per run with
VIDEO=off, or per project withraider u video off.
Failure trace replay (web)
Failing tests keep a step-by-step trace:
- each action with its locator and wait time
- a screenshot and the page HTML after every step that changes the page
- network requests and console messages on Chrome, Edge and Firefox
It is saved as raider-results/attachments/<n>-trace.zip, attached to the report as trace, and opens in Raider Desktop or on Hideout.
-
TRACE=onkeeps passing tests too.TRACE=offcaptures nothing.raider u trace on|offsets it for the project. - Password fields and
RAIDER_SECRET_*values are hidden. Other typed values, page text and URLs are kept, so treat a trace like the video. - Request headers, cookies and bodies are never captured.
Network mocking (web)
Answer a request with your own response, or block it:
mock_request('**/api/users', json: { users: [] })
mock_request(url, status: 500)
block_request('**/*.png')-
mock_requestreturns the mock;mock.callslists the requests it answered. - Works through WebDriver BiDi on Chrome, Edge and Firefox. Safari and
BIDI=offraiseNetworkHelper::Unavailable. - Handles CORS for cross-origin mocks, clears mocks after each test, and marks mocked requests in the trace.
Screenshot comparison (web)
match_screenshot('home')- Compares the page, or one element with
element:, with a baseline inscreenshots/baselines/<browser>/. - The first run saves the baseline. Later runs fail on a difference and attach the expected, actual and diff pictures to the report and the trace.
-
SCREENSHOTS=updaterewrites baselines. A missing baseline fails on CI. - Tolerances (
threshold,max_diff_pixels,max_diff_ratio) go inconfig/config.ymlunderscreenshots:. - Generate baselines where CI runs: a Retina laptop makes 2x pictures.
Login reuse (web)
Example projects log in once and restore the saved session for examples tagged :logged_in.
-
RAIDER_LOGIN_STATE=off|refreshandlogin_state: max_age:control it. - The state stays in git-ignored
.raider/auth/. - Skeletons (
examples:false) getSessionState#reuseto wire your own login.
Scheduling tests
Your computer's own scheduler runs the project's tests: all of them, one tag, or a list of spec files. Nothing needs to stay open, and scheduling never asks for sudo or admin rights. raider sc is short for raider schedule.
raider schedule add nightly --every day --at 02:00 --notify
raider schedule add smoke --every hour --at :15 --tag smoke --retries 1
raider schedule add checkout spec/cart_spec.rb spec/checkout_spec.rb --every weekday --at 08:30 --show-browser
raider schedule add weekly --every week --on fri --at 18:00 --project ~/code/my_tests
raider schedule list [--json] # each schedule, its next run, last result and OS state
raider schedule run nightly # run it now, the way the scheduler does
raider schedule remove nightly-
Rules:
--every hour|day|weekday|week.--at HH:MM(24h, local time) is required forday,weekdayandweek; forhouruse--at :MM(default:00).--on mon|tue|wed|thu|fri|sat|sunis required forweekand refused otherwise. Names use 1 to 60 ofa-z,0-9and_. Adding a name again replaces its schedule. -
What runs: no files and no
--tagruns all tests.--tag TAGruns one tag (letters, digits,_,:and-). Spec files are relative paths inside the project ending in_spec.rb, at most 200, and they can go before or after the options.--tagand files can't be used together. -
Your project's gem: the scheduler runs
bundle exec raider schedule runin your project, soaddchecks itsGemfile.lockfirst and refuses an older ruby_raider ("Scheduling needs ruby_raider X or later in this project: bundle update ruby_raider"), or a project with noGemfile.lockyet (runbundle installfirst). -
--project PATH: works from any folder. Put it after the subcommand (raider schedule add nightly ... --project DIR); before the subcommand Thor shows the help instead. -
Per machine: schedules live in
.raider/schedules/schedules.yml, whichaddgit-ignores (it holds machine paths; projects made with 4.2 have the line already), so a fresh clone has none. The OS entry holds only the schedule's name:raider schedule run NAME --project DIR --trigger schedule. The selection, retries, notify and show-the-browser setting are read from schedules.yml at each run, which checks every entry again before anything starts. -
Show the browser: scheduled runs are hidden (
HEADLESS=true) unless you add--show-browser. It shows the browser whateverheadlesssays inconfig/config.yml, and no driver change is needed: the runtime handles it (RAIDER_SHOW_BROWSER=1drops--headless). Neither config.yml norhelpers/driver_helper.rbis written.
| Project Headless (config.yml) | Show the browser off | Show the browser on |
|---|---|---|
| off / absent |
HEADLESS=true: hidden |
RAIDER_SHOW_BROWSER=1, HEADLESS removed: visible |
| on |
HEADLESS=true: hidden |
RAIDER_SHOW_BROWSER=1: visible (the runtime drops --headless) |
-
Where a browser can show: on macOS while you are logged in (a launchd agent runs in your GUI session). On Windows the task runs in your session while you are logged on; a visible browser there is not verified yet. On Linux
--show-browseris refused ("Show the browser isn't available for scheduled runs on Linux"): systemd user units and cron have no display. A project made before 4.0 is refused too ("Show the browser needs a project made with Ruby Raider 4.0 or later"). -
Retries:
--retries N(0 to 3) re-runs each failed test up to N times inside the same run, with rspec-retry (RETRY_COUNTis N + 1). From 4.2 the runtime gives a retried test a fresh browser. Before 4.2, a retry in a generated project failed with a refused connection, because the driver was already quit. The fix is in the gem (ruby_raider/runtime/rspec), so it also applies to manual runs such asRETRY_COUNT=2 bundle exec rspec;bundle update ruby_raiderbrings it to existing projects. A stop doesn't cut rspec-retry's tries short: the test running when you stop may still get its remaining tries before the run ends. -
Results: a run is a normal
bundle exec rspec, so its report goes toraider-results/and shows on the Dashboard. Each run also adds one line to.raider/schedules/<name>/runs.jsonl(statuspassed,failed,setup_errororbusy, the counts and the trigger; the last 50 lines are kept), and the last report is copied to.raider/schedules/<name>/last/(run.json and index.html, plus the attachments when they come to 50 MB or less). -
Exit codes:
raider schedule runexits 0 passed, 1 failed, 2 could not start or refused, 3 another run was going (anotherraider schedule runin the project, or any run that holds.raider/schedules/running.json). Only one scheduled run per project goes at a time, and a busy run leavesraider-results/alone. Other commands exit 0, or 2 with one line on stderr. Thor's own usage errors (a missing NAME, a bad--retries) exit 1 with Thor's text. -
Notify:
--notifyshows a desktop notification only when a run failed or could not start (not when it passed, was busy or was stopped). The title is "Ruby Raider: failed"; the body says how many tests failed, or why the run could not start. macOS usesosascript, Linuxnotify-send, Windows a PowerShell toast. A notification that can't be shown prints a warning and never changes the exit code. On Linuxnotify-sendneeds your desktop session bus, so under cron, and in a systemd user session that has not imported it, it usually can't show. -
Schedulers: macOS uses launchd (
~/Library/LaunchAgents/dev.rubyraider.schedule.<hash>.<name>.plist). Windows uses Task Scheduler (a task in the\RubyRaider\folder). Linux uses a systemd user timer in~/.config/systemd/user, or, whensystemctl --userdoes not answer, one tagged line in your crontab. Times are local time (cron uses the system time zone). -
Missed runs: launchd runs a run missed while the Mac slept once on wake. Task Scheduler runs it as soon as it can. systemd (
Persistent=true) runs it once your user manager is back, at the next boot or login. cron skips it, andraider schedule listsays so. -
Logged in: on macOS and Windows a run happens only while you are logged in. On Linux, systemd user timers stop when you log out unless lingering is on:
loginctl enable-linger $USER(raider does not run it). A systemd schedule can only be removed from a session wheresystemctl --userworks; until thenremoverefuses and keeps the entry. -
States:
ok,missing from launchd(orTask Scheduler,systemd,cron) when the OS entry is gone,missing from schedules.ymlwhen the OS still runs a schedule the file does not list,Ruby not found: re-schedule,test file not found: <file>when a listed spec file was deleted or renamed, andunknown (...)when the scheduler could not be asked. -
Ruby: the OS entry runs the absolute
rubyandbundlepaths captured when you added it, plusGEM_HOMEandGEM_PATHon macOS and Linux. After a Ruby upgrade or move, runraider schedule addagain. -
Logs: on macOS and Linux the scheduler's output goes to
.raider/schedules/<name>/scheduler.log, moved toscheduler.log.1at the start of a scheduled run once it is over 1 MB. Windows tasks keep no scheduler.log; each run's result is inruns.jsonlas always. -
Windows refusals: Task Scheduler and ruby.exe would expand
%...%,*,?,[or{, so a spec file or path holding them is refused: rename the file or use a tag. -
Moving the project: each OS entry is tied to the project's path (
<hash>is made from it). Runraider schedule removebefore you move the project. Entries made before a move keep the old path's id, solistin the new place can't find them. Remove them by hand; their names start withdev.rubyraider.schedule.:- macOS:
launchctl bootout gui/$UID/<label>, then delete~/Library/LaunchAgents/<label>.plist. - Windows:
schtasks /Delete /TN \RubyRaider\<id> /F. - Linux (systemd):
systemctl --user disable --now <id>.timer, delete<id>.timerand<id>.servicefrom~/.config/systemd/user, thensystemctl --user daemon-reload. - Linux (cron):
crontab -eand delete the# raider-schedule <id>line and the line under it.
- macOS:
- Tested on: macOS (launchd) is tested against the real scheduler: adding, a run started by launchd hidden and shown with Headless on, a failed run with a notification, a busy run, and removing. Task Scheduler, systemd and cron are covered by unit tests with a fake scheduler only.
Upgrading from 4.1
-
Old schedules are removed automatically. Schedules made with Ruby Raider 4.1 can't run in 4.2. They are removed the first time any
raider schedulecommand runs in the project (such asraider schedule list), or the next time one of them fires. One line says which were removed. -
Add them again with
raider schedule add. -
Leftover folders:
flows/,flow-results/and.raider/flows/(it holds old secrets inenv) are left alone. Delete them only after the line saying the old schedules were removed has appeared (run anyraider schedulecommand, such asraider schedule list): the cleanup looks for.raider/flows/, and the old entries write their logs there. -
A project moved after scheduling can't be cleaned automatically: its entries are tied to the old path. Remove them by hand; their names start with
dev.rubyraider.flow.:- macOS: find them with
ls ~/Library/LaunchAgents | grep dev.rubyraider.flow., runlaunchctl bootout gui/$(id -u)/<label>, then delete~/Library/LaunchAgents/<label>.plist(<label>isdev.rubyraider.flow.<hash>.<name>). - Windows: find them with
schtasks /Query /FO CSV | findstr dev.rubyraider.flow., thenschtasks /Delete /TN "\RubyRaider\<id>" /F. - Linux (systemd): find them with
ls ~/.config/systemd/user | grep dev.rubyraider.flow.(or$XDG_CONFIG_HOME/systemd/userwhen it is set), runsystemctl --user disable --now <id>.timer, delete<id>.timerand<id>.servicefrom~/.config/systemd/user, thensystemctl --user daemon-reload. - Linux (cron): find them with
crontab -l | grep -A1 '# raider-flow ', thencrontab -eand delete each# raider-flow <id>line and the line under it.
- macOS: find them with
Also included
- Page Object Model: base page and component classes with example pages
-
Example tests: working tests against the Raider Test Store (web) or the Raider test app (mobile), or an empty skeleton with
examples:false(see Getting Started) - CI: GitHub Actions, or Azure Pipelines for web. It caches ffmpeg and comments results on pull requests
- Failure screenshots: each failing test attaches a screenshot to the report
- Rake tasks: default, smoke and regression, filtered by tag
-
Retries: RSpec retry, with the count set by
RETRY_COUNT -
Re-run failed tests: RSpec keeps
spec/examples.txtfor--only-failures -
Parallel execution:
parallel_testsandparallel_split_test - Configurable browser: Chrome or Firefox, with arguments, viewport, timeout and headless mode
- Linting: RuboCop and Reek configs
Getting Started
Install the gem:
gem install ruby_raiderCreate a new project interactively:
raider new [project_name]Or skip the menu with parameters:
raider new [project_name] -p framework:rspec automation:seleniumCreate a mobile project (automation:ios, automation:android or automation:cross_platform):
raider new my_app -p framework:rspec automation:androidAdd accessibility testing:
raider new my_project -p framework:rspec automation:selenium --accessibilityGenerate an Azure Pipelines file instead of GitHub Actions (web projects only):
raider new my_project -p framework:rspec automation:selenium ci:azureAzure Pipelines for mobile (Appium) projects is not available yet: with ci:azure no pipeline file is generated and a notice is printed.
Azure Repos ignores pr: triggers, so add the pipeline as a Build validation branch policy to get pull request builds.
Generate an empty skeleton, without the example pages and tests (the menu asks Do you want example tests?):
raider new my_project -p framework:rspec automation:selenium examples:falseThe skeleton keeps the helpers, config, CI and README, and its empty suite passes; grow it with raider g.
Commands
Anything between square brackets ([...]) is where your input goes
Main Commands
raider new [PROJECT_NAME] # Create a new framework project
raider generate # Access scaffolding commands
raider utility # Access utility commands
raider schedule # Schedule the tests: add, list, remove, run
raider version # Show current version
raider help [COMMAND] # Describe available commands
Shortcuts: n (new), g (generate), u (utility), sc (schedule), v (version)
Scaffolding Commands
raider g page [NAME] # Create a page object
raider g spec [NAME] # Create an RSpec test
raider g helper [NAME] # Create a helper class
raider g component [NAME] # Create a component class
raider g scaffold [NAME(S)] # Create page + spec
Options:
-
--path [PATH]— Custom output path
Utility Commands
raider u path [PATH] # Set default paths for scaffolding
raider u url [URL] # Set default project URL
raider u browser [BROWSER] # Set default browser
raider u browser_options [OPTS] # Set browser arguments (e.g., no-sandbox, headless)
raider u headless [on/off] # Toggle headless mode
raider u strict [on/off] # Toggle strict find_element (fail on multiple matches)
raider u actionability [on/off] # Toggle the actionability wait before element actions
raider u raid # Run all tests (-p for parallel)
raider u timeout [SECONDS] # Set test timeout
raider u viewport [DIMENSIONS] # Set viewport size (e.g., 1920x1080)
raider u platform [PLATFORM] # Set platform for cross-platform tests
raider u start_appium # Start Appium server
raider u desktop # Download Raider Desktop
raider u video [ACTION] # Failure videos: install (ffmpeg), status, on, off
raider u trace [on/off] # Failure traces: turn on or off (Selenium)
Development
bundle install # Install dependencies
bundle exec rspec # Run all tests
bundle exec rspec --tag ~slow # Run fast tests only (skip E2E)
bundle exec rubocop # Run linter
bundle exec reek # Run code smell detectionSecurity checks
bundle exec bundler-audit check --update # Check gems for known vulnerabilities
bundle exec ruby-audit check # Check Ruby and RubyGems versions for known vulnerabilities
bundle exec rubocop --only Security # Run RuboCop's Security copsThe same checks (plus OSV-Scanner) run in CI via .github/workflows/security.yml.