Braintrust Ruby SDK

This is the official Ruby SDK for Braintrust, for tracing and evaluating your AI applications.

NOTE: This SDK is currently in BETA status and APIs may change between minor versions.

• Quick Start
• Installation
  • Setup script
  • CLI Command
  • Braintrust.init
  • Environment variables
• Tracing
  • Supported providers
  • Manually applying instrumentation
  • Creating custom spans
  • Attachments
  • Viewing traces
• Evals
  • Datasets
  • Remote scorers
• Documentation
• Troubleshooting
• Contributing
• License

Add to your Gemfile:

gem "braintrust", require: "braintrust/setup"

Set your API key and install:

export BRAINTRUST_API_KEY="your-api-key"
bundle install

Your LLM calls are now automatically traced. View them at braintrust.dev.

The SDK also offers additional setup options for a variety of applications.

See our examples for more detail.

For most applications, we recommend adding require: "braintrust/setup" to your Gemfile or an initializer file in your Ruby application to automatically setup the SDK. This will automatically apply instrumentation to all available LLM libraries.

You can use environment variables to configure behavior.

You can use this CLI command to instrument any Ruby application without modifying the source code.

First, make sure the gem is installed on the system:

gem install braintrust

Then wrap the start up command of any Ruby application to apply:

braintrust exec -- ruby app.rb
braintrust exec -- bundle exec rails server
braintrust exec --only openai -- ruby app.rb

You can use environment variables to configure behavior.

NOTE: Installing a package at the system-level does not guarantee compatibility with all Ruby applications on that system; conflicts with dependencies can arise.

For stronger assurance of compatibility, we recommend either:

• Installing via the application's Gemfile and bundle install when possible.
• OR for OCI/Docker deployments, gem install braintrust when building images in your CI/CD pipeline (and verifying their safe function.)

For more control over when auto-instrumentation is applied:

require "braintrust"

Braintrust.init

Options:

Example with options:

Braintrust.init(
  default_project: "my-project",
  auto_instrument: { only: [:openai] }
)

The SDK automatically instruments these LLM libraries:

For fine-grained control, disable auto-instrumentation and instrument specific clients:

require "braintrust"
require "openai"

Braintrust.init(auto_instrument: false) # Or BRAINTRUST_AUTO_INSTRUMENT=false

# Instrument all OpenAI clients
Braintrust.instrument!(:openai)

# OR instrument a single client
client = OpenAI::Client.new
Braintrust.instrument!(:openai, target: client)

Wrap business logic in spans to see it in your traces:

tracer = OpenTelemetry.tracer_provider.tracer("my-app")

tracer.in_span("process-request") do |span|
  span.set_attribute("user.id", user_id)

  # LLM calls inside here are automatically nested under this span
  response = client.chat.completions.create(...)
end

Log binary data (images, PDFs, audio) in your traces:

require "braintrust/trace/attachment"

att = Braintrust::Trace::Attachment.from_file("image/png", "./photo.png")

# Use in messages (OpenAI/Anthropic format)
messages = [
  {
    role: "user",
    content: [
      {type: "text", text: "What's in this image?"},
      att.to_h
    ]
  }
]

# Log to span
span.set_attribute("braintrust.input_json", JSON.generate(messages))

Create attachments from various sources:

Braintrust::Trace::Attachment.from_bytes("image/jpeg", image_data)
Braintrust::Trace::Attachment.from_file("application/pdf", "./doc.pdf")
Braintrust::Trace::Attachment.from_url("https://example.com/image.png")

See example: trace_attachments.rb

Get a permalink to any span:

tracer = OpenTelemetry.tracer_provider.tracer("my-app")

tracer.in_span("my-operation") do |span|
  # your code here
  puts "View trace at: #{Braintrust::Trace.permalink(span)}"
end

Run evaluations against your AI systems:

require "braintrust"

Braintrust.init

Braintrust::Eval.run(
  project: "my-project",
  experiment: "classifier-v1",
  cases: [
    {input: "apple", expected: "fruit"},
    {input: "carrot", expected: "vegetable"}
  ],
  task: ->(input) { classify(input) },
  scorers: [
    ->(input, expected, output) { output == expected ? 1.0 : 0.0 }
  ]
)

Load test cases from a Braintrust dataset:

Braintrust::Eval.run(
  project: "my-project",
  dataset: "my-dataset",
  task: ->(input) { classify(input) },
  scorers: [...]
)

Use scoring functions defined in Braintrust:

Braintrust::Eval.run(
  project: "my-project",
  cases: [...],
  task: ->(input) { ... },
  scorers: [
    Braintrust::Scorer.remote("my-project", "accuracy-scorer")
  ]
)

See examples: eval.rb, dataset.rb, remote_functions.rb

• Braintrust Documentation
• API Reference

First verify there are no errors in your logs after running with BRAINTRUST_DEBUG=true set.

Your application needs the following for this to work:

require 'bundler/setup'
Bundler.require

It is present by default in Rails applications, but may not be in Sinatra, Rack, or other applications.

Alternatively, you can add require 'braintrust/setup' to your application initialization files.

See CONTRIBUTING.md for development setup and guidelines.

Apache License 2.0 - see LICENSE.