AgentHarness

A unified Ruby interface for CLI-based AI coding agents like Claude Code, Cursor, Gemini CLI, GitHub Copilot, and more.

Features

Unified Interface: Single API for multiple AI coding agents
8 Built-in Providers: Claude Code, Cursor, Gemini CLI, GitHub Copilot, Codex, Aider, OpenCode, Kilocode
Full Orchestration: Provider switching, circuit breakers, rate limiting, and health monitoring
Flexible Configuration: YAML, Ruby DSL, or environment variables
Token Tracking: Monitor usage across providers for cost and limit management
Error Taxonomy: Standardized error classification for consistent error handling
Dynamic Registration: Add custom providers at runtime

Installation

Add to your Gemfile:

gem "agent-harness"

Or install directly:

gem install agent-harness

Quick Start

require "agent_harness"

# Send a message using the default provider
response = AgentHarness.send_message("Write a hello world function in Ruby")
puts response.output

# Use a specific provider
response = AgentHarness.send_message("Explain this code", provider: :cursor)

Configuration

Ruby DSL

AgentHarness.configure do |config|
  # Logging
  config.logger = Logger.new(STDOUT)
  config.log_level = :info

  # Default provider
  config.default_provider = :claude
  config.fallback_providers = [:cursor, :gemini]

  # Timeouts
  config.default_timeout = 300

  # Orchestration
  config.orchestration do |orch|
    orch.enabled = true
    orch.auto_switch_on_error = true
    orch.auto_switch_on_rate_limit = true

    orch.circuit_breaker do |cb|
      cb.enabled = true
      cb.failure_threshold = 5
      cb.timeout = 300
    end

    orch.retry do |r|
      r.enabled = true
      r.max_attempts = 3
      r.base_delay = 1.0
    end
  end

  # Provider-specific configuration
  config.provider(:claude) do |p|
    p.enabled = true
    p.timeout = 600
    p.model = "claude-sonnet-4-20250514"
  end

  # Callbacks
  config.on_tokens_used do |event|
    puts "Used #{event.total_tokens} tokens on #{event.provider}"
  end

  config.on_provider_switch do |event|
    puts "Switched from #{event[:from]} to #{event[:to]}: #{event[:reason]}"
  end
end

Providers

Built-in Providers

Provider	CLI Binary	Description
`:claude`	`claude`	Anthropic Claude Code CLI
`:cursor`	`cursor-agent`	Cursor AI editor CLI
`:gemini`	`gemini`	Google Gemini CLI
`:github_copilot`	`copilot`	GitHub Copilot CLI
`:codex`	`codex`	OpenAI Codex CLI
`:aider`	`aider`	Aider coding assistant
`:opencode`	`opencode`	OpenCode CLI
`:kilocode`	`kilocode`	Kilocode CLI

Direct Provider Access

# Get a provider instance
provider = AgentHarness.provider(:claude)
response = provider.send_message(prompt: "Hello!")

# Check provider availability
if AgentHarness::Providers::Registry.instance.get(:claude).available?
  puts "Claude CLI is installed"
end

# List all registered providers
AgentHarness::Providers::Registry.instance.all
# => [:claude, :cursor, :gemini, :github_copilot, :codex, :opencode, :kilocode, :aider]

Custom Providers

class MyProvider < AgentHarness::Providers::Base
  class << self
    def provider_name
      :my_provider
    end

    def binary_name
      "my-cli"
    end

    def available?
      system("which my-cli > /dev/null 2>&1")
    end
  end

  protected

  def build_command(prompt, options)
    [self.class.binary_name, "--prompt", prompt]
  end

  def parse_response(result, duration:)
    AgentHarness::Response.new(
      output: result.stdout,
      exit_code: result.exit_code,
      provider: self.class.provider_name,
      duration: duration
    )
  end
end

# Register the custom provider
AgentHarness::Providers::Registry.instance.register(:my_provider, MyProvider)

Orchestration

Circuit Breaker

Prevents cascading failures by stopping requests to unhealthy providers:

# After 5 consecutive failures, the circuit opens for 5 minutes
config.orchestration.circuit_breaker.failure_threshold = 5
config.orchestration.circuit_breaker.timeout = 300

Rate Limiting

Track and respect provider rate limits:

manager = AgentHarness.conductor.provider_manager

# Mark a provider as rate limited
manager.mark_rate_limited(:claude, reset_at: Time.now + 3600)

# Check rate limit status
manager.rate_limited?(:claude)

Health Monitoring

Monitor provider health and automatically switch on failures:

manager = AgentHarness.conductor.provider_manager

# Record success/failure
manager.record_success(:claude)
manager.record_failure(:claude)

# Check health
manager.healthy?(:claude)

# Get available providers
manager.available_providers

Token Tracking

# Track tokens across requests
AgentHarness.token_tracker.on_tokens_used do |event|
  puts "Provider: #{event.provider}"
  puts "Input tokens: #{event.input_tokens}"
  puts "Output tokens: #{event.output_tokens}"
  puts "Total: #{event.total_tokens}"
end

# Get usage summary
AgentHarness.token_tracker.summary

Error Handling

begin
  response = AgentHarness.send_message("Hello")
rescue AgentHarness::AuthenticationError => e
  puts "Auth failed for provider: #{e.provider}"
  # Optionally trigger re-auth flow (see Authentication Management below)
rescue AgentHarness::TimeoutError => e
  puts "Request timed out"
rescue AgentHarness::RateLimitError => e
  puts "Rate limited, retry after: #{e.reset_time}"
rescue AgentHarness::NoProvidersAvailableError => e
  puts "All providers unavailable: #{e.attempted_providers}"
rescue AgentHarness::Error => e
  puts "Provider error: #{e.message}"
end

Error Taxonomy

Classify errors for consistent handling:

category = AgentHarness::ErrorTaxonomy.classify_message("rate limit exceeded")
# => :rate_limited

AgentHarness::ErrorTaxonomy.retryable?(category)
# => false (rate limits should switch provider, not retry)

AgentHarness::ErrorTaxonomy.action_for(category)
# => :switch_provider

Authentication Management

AgentHarness can detect authentication failures and manage credentials for CLI agents.

Auth Type

Providers declare their authentication type:

provider = AgentHarness.provider(:claude)
provider.auth_type
# => :oauth  (token-based auth that can expire)

provider = AgentHarness.provider(:aider)
provider.auth_type
# => :api_key  (static API key, no refresh needed)

Auth Status Check

Pre-flight check auth before starting a run:

AgentHarness.auth_valid?(:claude)
# => true/false

AgentHarness.auth_status(:claude)
# => { valid: false, expires_at: <Time>, error: "Session expired" }

For providers without a built-in auth check (including :api_key providers), auth_valid? returns false and auth_status returns an error indicating the check is not implemented. Custom providers can implement an auth_status instance method to provide their own check.

Auth Error Detection

When a CLI agent fails due to expired or invalid authentication, send_message raises AuthenticationError with the provider name. Authentication errors are always surfaced directly to the caller (never auto-switched to another provider) so your application can trigger the appropriate re-auth flow:

begin
  AgentHarness.send_message("Hello", provider: :claude)
rescue AgentHarness::AuthenticationError => e
  puts e.provider  # => :claude
  puts e.message   # => "oauth token expired"
  # Trigger re-authentication flow for the specific provider
end

OAuth URL Generation

For OAuth providers, get the URL the user should visit to start the login flow:

AgentHarness.auth_url(:claude)
# => "https://claude.ai/oauth/authorize"

This raises NotImplementedError for :api_key providers.

Credential Refresh

Accept a pre-exchanged OAuth token and update the provider's stored credentials. The OAuth authorization code exchange is provider-specific and should be handled by your application or CLI login command before calling this method:

AgentHarness.refresh_auth(:claude, token: "new-oauth-token")
# => { success: true }

Any existing expiry metadata in the credentials file is cleared on refresh so that auth_valid? returns true immediately after a successful refresh.

This raises NotImplementedError for :api_key providers. Credential file paths respect the CLAUDE_CONFIG_DIR environment variable.

Development

# Install dependencies
bin/setup

# Run tests
bundle exec rake spec

# Run linter
bundle exec standardrb

# Interactive console
bin/console

License

MIT License. See LICENSE.txt.