Evals

A Ruby library for evaluating LLM responses.

Based on the Prompt Evaluation example in the Anthropic Skilljar course.

Install the gem and add to the application's Gemfile by executing:

bundle add evals

If bundler is not being used to manage dependencies, install the gem by executing:

gem install evals

Copy .env.example to .env and add your Anthropic API key.

See examples/demo.rb for an example.

After checking out the repo, run bin/setup to install dependencies. Then, run rake test to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and the created tag, and push the .gem file to rubygems.org.

Bug reports and pull requests are welcome on GitHub at https://github.com/andyw8/evals.

The gem is available as open source under the terms of the MIT License.