LlmProviders

A lightweight, unified interface for multiple LLM providers. Only depends on faraday — no ActiveSupport required.

日本語版 README

• Lightweight — Single dependency (faraday), no ActiveSupport
• Unified interface — Same API for all providers
• Streaming — Real-time token streaming with block syntax
• Tool calling — Consistent tool/function calling across providers
• Token tracking — Usage stats (input, output, cached) in every response

Add to your Gemfile:

gem "llm_providers"

require "llm_providers"

provider = LlmProviders::Providers.build(:anthropic)

# Synchronous
result = provider.chat(
  messages: [{ role: "user", content: "Hello!" }],
  system: "You are a helpful assistant."
)
puts result[:content]

# Streaming
provider.chat(messages: [{ role: "user", content: "Hello!" }]) do |chunk|
  print chunk[:content]
end

LlmProviders.configure do |config|
  config.logger = Rails.logger  # or any Logger instance
end

provider = LlmProviders::Providers.build(
  :openai,
  model: "gpt-4.1",
  temperature: 0.7,   # optional; defaults to nil (not sent)
  max_tokens: 4096,
  effort: "medium"    # optional; "low", "medium", "high", "xhigh" or "max"
)

• temperature defaults to nil and is sent only when you set it. Versions up to 0.2.x always sent 0.7. Claude 5.x and OpenAI reasoning models reject the parameter.
• effort sets how hard the model thinks (nil means the provider default). Any other value raises ArgumentError.
  • Anthropic: sent as output_config: { effort: ... }.
  • OpenAI: gpt-6* sends it unchanged as reasoning: { effort: ... } (Responses API, see below). Other reasoning models (gpt-5*, o-series) send reasoning_effort, with xhigh / max clamped to high. Ignored for gpt-3.5* / gpt-4*.
  • Google: mapped to thinkingConfig.thinkingLevel (low, medium, high; xhigh and max become high). Only sent for Gemini 3 and later (and *-latest aliases); ignored for Gemini 2.x.
  • OpenRouter: sent as reasoning: { effort: ... }, with xhigh and max clamped to high.
• OpenAI gpt-6* models use the Responses API (/v1/responses), because on chat/completions gpt-6-astra and gpt-6.1-sol reject function tools, and gpt-6-luna accepts them only with reasoning effort none. The system prompt goes in instructions, the token cap in max_output_tokens, tools are sent with strict: false, and requests use store: false. The return value and streamed chunks have the same shape as on chat/completions. All other OpenAI models stay on chat/completions, and OpenRouter always uses chat/completions. Reasoning items are not passed back between tool rounds, and usage output includes reasoning tokens.
• OpenAI treats every other model except gpt-3.5* / gpt-4* as a reasoning model: it uses max_completion_tokens and sends no temperature. OpenRouter ids look like vendor/model, so it keeps using max_tokens and sends temperature only when given.
• For Google, usage output includes thinking tokens.

tools = [
  {
    name: "get_weather",
    description: "Get the current weather",
    parameters: {
      type: "object",
      properties: {
        location: { type: "string", description: "City name" }
      },
      required: ["location"]
    }
  }
]

result = provider.chat(
  messages: [{ role: "user", content: "What's the weather in Tokyo?" }],
  tools: tools
)

result[:tool_calls].each do |tc|
  puts "#{tc[:name]}: #{tc[:input]}"
end

Every chat call returns a hash with:

{
  content: "Response text",
  tool_calls: [
    { id: "...", name: "...", input: {...} }
  ],
  usage: {
    input: 100,           # Input tokens
    output: 50,           # Output tokens
    cached_input: 80      # Cached input tokens (Anthropic only)
  },
  latency_ms: 1234,
  raw_response: {...}
}

OpenRouter gives you access to 300+ models through a single API — ideal for providers not directly supported by this gem (DeepSeek, Meta Llama, Mistral, Qwen, etc.).

provider = LlmProviders::Providers.build(
  :openrouter,
  model: "deepseek/deepseek-chat",
  app_name: "MyApp",
  app_url: "https://myapp.example.com",
  provider: { order: ["DeepSeek", "Together"], allow_fallbacks: true }
)

# List available models
models = LlmProviders::Providers::Openrouter.models
models.each { |m| puts "#{m[:id]} (ctx: #{m[:context_length]})" }

begin
  result = provider.chat(messages: messages)
rescue LlmProviders::ProviderError => e
  puts "Error: #{e.message}"  # OpenRouter: "[DeepSeek] Model is unavailable"
  puts "Code: #{e.code}"      # e.g., "anthropic_error", "openrouter_error"
end

# Interactive chat
ANTHROPIC_API_KEY=your-key ruby examples/simple_chat.rb

# One-shot
ANTHROPIC_API_KEY=your-key ruby examples/one_shot.rb "Hello!"

# With tools
ANTHROPIC_API_KEY=your-key ruby examples/with_tools.rb

# Other providers
OPENAI_API_KEY=your-key ruby examples/simple_chat.rb openai
GOOGLE_API_KEY=your-key ruby examples/simple_chat.rb google

# OpenRouter (DeepSeek, Llama, Mistral, etc.)
OPENROUTER_API_KEY=your-key ruby examples/simple_chat.rb openrouter deepseek/deepseek-chat
OPENROUTER_API_KEY=your-key ruby examples/with_tools.rb openrouter meta-llama/llama-3.3-70b-instruct

MIT