BenchGecko for Ruby

The data layer of the AI economy. Official Ruby SDK for querying thousands of AI models with cross-provider pricing and daily price history, company valuations, funding timelines, revenue estimates, benchmark scores, agent leaderboards, and a live changelog of every price drop, every launch, every deprecation.

If it moved in AI today, it's already on BenchGecko.

• Models. Thousands of AI models with cross-provider pricing and daily price history.
• Companies. Hundreds of AI companies with valuations, funding timelines, and revenue estimates.
• Benchmarks. Reasoning, coding, math, instruction following, safety, multimodal, multilingual, long context.
• Agents. Developer adoption signals and agent leaderboards.
• Changelog. Every price drop, every launch, every deprecation, as it happens.

Add to your Gemfile:

gem "benchgecko"

Or install directly:

gem install benchgecko

require "benchgecko"

# Look up any model
model = BenchGecko.get_model("claude-3.5-sonnet")
puts model.name       #=> "Claude 3.5 Sonnet"
puts model.provider   #=> "Anthropic"
puts model.score("MMLU")  #=> 88.7

# List all tracked models
BenchGecko.list_models.each { |id| puts id }

The comparison engine surfaces benchmark differences and pricing ratios, making it straightforward to evaluate tradeoffs between models:

result = BenchGecko.compare_models("gpt-4o", "claude-3.5-sonnet")

puts result[:cheaper]           #=> "gpt-4o"
puts result[:cost_ratio]        #=> 0.69
puts result[:benchmark_diff]    #=> {"MMLU" => 0.0, "HumanEval" => -1.8, ...}

# Positive diff means model_a scores higher
result[:benchmark_diff].each do |bench, diff|
  next unless diff
  winner = diff >= 0 ? "GPT-4o" : "Claude 3.5 Sonnet"
  puts "#{bench}: #{winner} wins by #{diff.abs} points"
end

Estimate inference costs before committing to a provider. Prices are per million tokens:

cost = BenchGecko.estimate_cost("gpt-4o",
  input_tokens: 2_000_000,
  output_tokens: 500_000
)

puts cost[:input_cost]   #=> 5.0
puts cost[:output_cost]  #=> 5.0
puts cost[:total]        #=> 10.0

Filter models by benchmark performance to find the best fit for your workload:

# All models scoring 87+ on MMLU
strong_reasoners = BenchGecko.top_models("MMLU", min_score: 87.0)
strong_reasoners.each { |m| puts "#{m.name}: #{m.score('MMLU')}" }

# Cheapest model above a quality threshold
budget_pick = BenchGecko.cheapest_above("MMLU", 85.0)
puts "#{budget_pick.name} at $#{budget_pick.cost_per_million}/M tokens"

BenchGecko organizes benchmarks into categories covering reasoning, coding, math, instruction following, safety, multimodal, multilingual, and long context evaluation:

BenchGecko.benchmark_categories.each do |key, info|
  puts "#{info[:name]}: #{info[:benchmarks].join(', ')}"
  puts "  #{info[:description]}"
end

The gem ships with a curated catalog of major models from OpenAI, Anthropic, Google, Meta, Mistral, and DeepSeek. Each entry includes benchmark scores, parameter counts, context window sizes, and per-token pricing.

model = BenchGecko.get_model("deepseek-v3")
puts model.parameters       #=> 671
puts model.context_window   #=> 128000
puts model.cost_per_million  #=> 0.685

• Model selection pipelines. Programmatically pick the cheapest model that meets your quality bar.
• Cost monitoring. Estimate monthly spend across different model configurations.
• Benchmark dashboards. Pull structured scores into internal reporting tools.
• Agent evaluation. Compare AI agents across capability dimensions.
• Pricing intelligence. Track every price drop and launch through the live changelog.

• BenchGecko. The data layer of the AI economy.
• Source Code. Contributions welcome.

MIT License. See LICENSE.txt for details.