🤖 Spec AI

AI-powered test generation for Ruby on Rails applications.

Spec AI analyzes your Rails application, understands the model, schema, factory, and routes context, generates test scenarios with an LLM, writes the RSpec file, and runs the generated tests.

Generate better Rails tests with AI — locally or using hosted LLMs.

It is not a raw “LLM writes an RSpec file” wrapper. Ruby owns control flow. An RCrewAI crew reasons about the app; a deterministic generator writes the spec.

🤝 Collaborators are welcome. Issues, pull requests, and ideas to improve this gem are all encouraged — you do not need to be a maintainer to contribute.

• 🤖 AI-powered test case generation
• 💎 Built specifically for Ruby on Rails
• 🧠 Rails-aware application context
• 📋 Automatic test scenario planning
• 🧪 RSpec test generation
• 🚀 Automatic test execution
• 🔧 Local LLMs with Ollama
• ☁️ Hosted LLMs such as OpenAI
• 🔍 Analyzes models, schema, factories, associations, and routes
• ⚡ Compare different AI models for test generation
• 🔐 Local AI option for privacy-sensitive projects

Spec AI collects relevant context from your Rails application before asking the model to generate tests.

1. Context collector reads the model, db/schema.rb, associations, FactoryBot factory, existing spec, and matching routes.
2. RailsAnalyst (RCrewAI) may fetch extra related files through sandboxed tools.
3. TestArchitect returns a JSON test plan (not Ruby). Ruby then keeps only scenarios the model actually enforces and fills any gaps from the schema and validations. The LLM can suggest extras; it cannot invent columns, associations, or Devise rules.
4. RspecGenerator (plain Ruby) writes spec/models/user_spec.rb.
5. RspecRunner executes the file.
6. On failure, Spec AI drops examples the model does not enforce, then QADebugger may return a corrected plan; Ruby regenerates and reruns, up to max_repair_attempts.

Agents can only read app/, spec/, db/, config/, and test/ inside the application root. They cannot execute application code or write files directly.

A single generate run against the same Devise User model, local Ollama vs hosted OpenAI:

Both models generated executable RSpec tests with 0 failures for the tested User model.

The hosted gpt-4o-mini run planned more scenarios in this test and finished generation substantially faster. The local llama3.1 run shows that Rails test generation can also stay on your machine.

These numbers are a single comparison run, not a general benchmark. Results vary with the Rails app, model complexity, prompt, hardware, model version, and configuration.

Generate with a local Ollama model:

The same flow with OpenAI:

The gem is on RubyGems. Add it to your Rails app:

# Gemfile
group :development, :test do
  gem "spec_ai"
end

bundle install
bin/rails generate spec_ai:install

That creates config/spec_ai.yml.

To install a specific version:

gem "spec_ai", "~> 0.1.0"

# config/spec_ai.yml
provider: ollama
model: llama3.1
temperature: 0.1

testing:
  framework: rspec

generation:
  include_edge_cases: true
  include_security_cases: true
  include_authorization: true

execution:
  run_after_generation: true
  auto_fix: true
  max_repair_attempts: 2

Set verbose: true (or pass --verbose) to print RCrewAI and HTTP logs.

API keys come from the environment, not the YAML file:

RCrewAI is the only LLM layer. You can switch providers without changing the gem:

bundle exec spec-ai generate User --provider ollama --model llama3.1
bundle exec spec-ai generate User --provider openai --model gpt-4o-mini

Local Ollama is the path to keep application source on the developer’s machine.

bundle exec spec-ai generate User
bundle exec spec-ai generate app/models/user.rb
bin/rails spec_ai:generate[User]

If FactoryBot or RSpec bootstrap files are missing, Spec AI creates them and stops when you need to bundle install:

bundle install
bundle exec spec-ai generate User

Inspect Rails context without calling an LLM:

bundle exec spec-ai analyze User

Generate RSpec for a model:

bundle exec spec-ai generate User

Re-run the generated spec the usual way:

bundle exec rspec
bundle exec rspec spec/models/user_spec.rb

RCrewAI HTTP logs are hidden by default. Pass -v to show them.

Other commands:

spec-ai analyze User                          # print collected context, no LLM
spec-ai fix spec/models/user_spec.rb          # run + repair
spec-ai generate User --no-run                # write spec only
spec-ai generate User --no-fix                # do not auto-repair
spec-ai generate User --force                 # replace an existing spec (this is the default)
spec-ai generate User --merge                 # add examples instead of replacing the spec
spec-ai generate User --verbose               # show RCrewAI / HTTP logs

spec-ai missing and spec-ai improve are reserved for 0.3 / 0.5.

• Ruby 3.1+
• Rails 7+
• RSpec (rspec-rails in the Rails app Gemfile; this version generates model specs only)

If FactoryBot is missing, Spec AI adds factory_bot_rails to the Gemfile, writes a factory from the schema, and asks you to bundle install before generating.

If the spec/ folder is missing (rails_helper, spec_helper), Spec AI creates those bootstrap files so generated specs can load.

bundle install
bundle exec rspec

Gem tests stub RCrewAI crews. They never call a live LLM provider.

Collaborators are welcome. This gem is better when more people use it, file issues, and send pull requests.

1. Fork karthick965938/spec-ai.
2. Create a branch, add tests for the change, and run bundle exec rspec.
3. Open a pull request with a short description of why the change helps.

Bug reports, docs fixes, and small CLI/UX improvements are as useful as large features.

Built by Karthick Nagarajan.

If you find Spec AI useful, consider giving the project a ⭐ on GitHub.

More features and AI-powered Rails testing capabilities are coming soon.

MIT. See LICENSE.txt.