Rails Benchmark Suite 🚀

Standardized Hardware Benchmarking for Rails 8.1+

A standardized performance suite designed to measure the "Heft" of a machine using realistic, high-throughput Rails 8+ workloads.

Think of this as a "Test Track" for Rails servers. Unlike profilers that measure your specific application code, this gem runs a fixed, standardized set of Rails operations (Active Record object allocation, SQL query complexity, ActionView rendering, and background job throughput) to measure the raw performance of your server and Ruby configuration.

To ensure a level playing field, the gem boots an isolated, in-memory SQLite environment. It creates its own schema and records, meaning it never touches your production data and returns comparable results across any machine.

The Heft Score is a weighted metric representing a machine's ability to handle Rails tasks.

• Baseline: A score of 100 is calibrated to represent an AWS c6g.large (ARM) instance.
• Objective: To provide a simple, comparable number for evaluating different computing platforms (Cloud VMs, bare-metal, or local dev rigs).

Visualize your threading efficiency and bottlenecks with a self-contained dashboard:

bundle exec rails_benchmark_suite -t 8 --html

This generates rails_benchmark_report.html containing:

• Scaling Curve: Interactive bar chart comparing 1-thread vs Max-thread throughput.
• Efficiency Heatmap: Score cards identifying which workloads are Gil-bound vs CPU-bound.
• Detailed Metrics: Raw IPS and detailed scaling factors.

Ensure you are in your Rails root directory and run:

ruby --yjit -S bundle exec rails_benchmark_suite

Note: bundle exec is mandatory for Rails environment stability and to prevent Minitest version conflicts.

Rails Benchmark Suite prioritizes Benchmarking (via benchmark-ips) over Profiling.

• Benchmarking: Focuses on macro-throughput—"How many iterations can the hardware handle?" This provides the final Heft Score.
• Why no Profiling? Profiling tools (like StackProf or Vernier) introduce instrumentation overhead that skews hardware metrics. We aim for "Conceptual Compression"—one clear number to inform infrastructure decisions.

• Ruby: 3.3+ (Ruby with YJIT support highly recommended)
• Rails: 8.1+
• Database: SQLite3

• Ruby: 3.3+ (3.4+ recommended for YJIT)
• Rails: 8.0+
• Database: SQLite3

• macOS: brew install vips
• Linux (Ubuntu/Debian): sudo apt install libvips-dev

bundle exec rails_benchmark_suite

The easy way - run the benchmark with your current Ruby configuration.

RUBY_OPT="--yjit" bundle exec rails_benchmark_suite

Enable YJIT for maximum performance measurement accuracy. This is the recommended method for Rails 8+ benchmarking.

bundle exec rails_benchmark_suite --json > report.json

Perfect for CI/CD pipelines and programmatic analysis. Outputs clean JSON without any UI elements.

To diagnose scaling bottlenecks, run the tool in Profile Mode (--profile) and generate the HTML Report (--html). This compares Single-Thread vs Multi-Thread performance side-by-side.

bundle exec rails_benchmark_suite --profile --html

Opening the Report:

• Local: open tmp/rails_benchmark_report.html
• Remote: scp user@server:/current/path/tmp/rails_benchmark_report.html .

• --json: Output results in JSON format
• --skip-rails: Run in isolated mode without loading Rails environment
• --version: Display gem version
• -h / --help: Show usage help

If you want to test hardware performance without an existing application:

git clone https://github.com/overnet/rails_benchmark_suite.git
cd rails_benchmark_suite
bundle install
bin/rails_benchmark_suite

The Rails Heft Index (RHI) measures your hardware's ability to handle Rails workloads using this formula:

RHI Score = Σ (4-Thread IPS × Weight)

Why 4-Thread IPS? We use 4-thread performance to simulate production concurrency where multiple requests are handled simultaneously.

Dynamic Weight Redistribution: If a workload is skipped (e.g., Image Processing without libvips), its weight is redistributed proportionally among remaining workloads to maintain a 100% scale.

Your RHI score maps to these performance tiers:

• < 50: Entry/Dev - Suitable for local development
• 50-200: Production-Ready - Handles moderate production traffic
• > 200: High-Performance - Optimized for high-traffic applications

The gem measures performance across critical Rails subsystems using a dedicated, isolated schema:

• Active Record Heft: Standardized CRUD: Creation, indexing, and complex querying.
• Cache Heft: High-frequency read/writes to the Rails memory store.
• Solid Queue Heft: Background job enqueuing and database-backed polling stress.
• View Heft: Partial rendering overhead and ActionView throughput.
• Image Heft: Image processing performance (requires libvips).

If you see YJIT: Disabled, it means your Ruby was not compiled with YJIT support.

• Fix (rbenv): RUBY_CONFIGURE_OPTS="--enable-yjit" rbenv install 3.4.1
• Fix (rvm): rvm install 3.4.1 --enable-yjit

Version 0.2.9+ includes surgical connection resets and randomized backoffs to handle SQLite concurrency. If issues persist, ensure no other processes are accessing the benchmark database.

• Engine: Built on benchmark-ips.
• Database: Uses In-Memory SQLite with cache=shared and a 50-connection pool for multi-threaded accuracy.
• Isolation: Uses transactional rollbacks and Mutex-wrapped schema creation.
• Threading: Supports 1-thread and 4-thread scaling tests.

• Vision: Inspired by @dhh in rails/rails#50451.
• Initial Roadmap: Based on suggestions by @JoeDupuis.
• Implementation: The Rails Community.

The gem is available as open source under the terms of the MIT License.