Project

ask-agent

0.0
The project is in a healthy, maintained state
ask-agentask-rb/ask-agentHomepageDocumentationSource CodeBug Tracker
Agent loop, session management, tool execution, context compaction, hooks, and extensions.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
 Popularity
Downloads
11,494
Stars
0
Forks
0
Watchers
0
 Releases
Current version
0.20.0
Total releases
42
First release
2026-06-09
Latest release
2026-07-26
 Development
Primary Language
Ruby
Licenses
MIT
Average date of last 50 commits
2026-07-07
Reverse Dependencies
1

 Dependencies

Development

minitest
~> 5.25
mocha
~> 3.1
rake
~> 13.0

Runtime

ask-tools
>= 0.1
ask-core
>= 0.1
ask-llm-providers
>= 0.1
ask-skills
>= 0.1
ask-instrumentation
>= 0.1
rufus-scheduler
~> 3.9
ask-state-providers
>= 0.1
 Project Readme

ask-agent

Agent runtime for the ask-rb ecosystem. The core agent loop: think → call tools → execute → feed back → repeat.

Ported from RubyLLM::Conductor into the Ask::Agent namespace.

Installation

gem "ask-agent"

Quick Start

require "ask-agent"

session = Ask::Agent::Session.new(
  model: "gpt-4o",
  tools: [Ask::Tools::Shell::Bash, Ask::Tools::Shell::Read]
)

response = session.run("What files are in the current directory?")
puts response

Components

Component File Purpose
Ask::Agent::Session session.rb Full agent loop — message → tool calls → results → follow-up
Ask::Agent::Loop loop.rb Turn management, loop detection, max-turn guard
Ask::Agent::ToolExecutor tool_executor.rb Parallel/sequential tool execution with retry and abort
Ask::Agent::Compactor compactor.rb Context window management with proactive/overflow compaction
Ask::Agent::Hooks hooks.rb Before/after tool lifecycle callbacks
Ask::Agent::Events events.rb Data.define event types for streaming and monitoring
Ask::Agent::Telemetry telemetry.rb File-backed telemetry for error tracking
Ask::Agent::Reflector reflector.rb Assistant response self-evaluation
Ask::Agent::MetaAgent meta_agent.rb LLM-powered self-improvement from telemetry
Ask::Agent::Evaluator evaluator.rb Independent response evaluation with structured rubric — different model, isolated context
Ask::Agent::Configuration configuration.rb Global config: model, turns, concurrency, evaluator

Evaluator

Independent response evaluation with generator/evaluator separation. The evaluator uses a separate model (different from the session's model) and an isolated context to judge the agent's output — preventing the anti-pattern of a model grading its own work.

Quick start

session = Ask::Agent::Session.new(
  model: "gpt-4o",
  evaluator: { model: "claude-sonnet-4", goal: "Write an email validator" }
)
session.run("Write email validation")

Verdicts

Verdict Behavior
:accept Output passes — falls through to reflection
:revise Evaluator provides feedback; session runs another turn with it injected
:block Output is fundamentally wrong — returns blocked message, emits EvaluationBlocked

Configuration

# Set a global default evaluator model
Ask::Agent.configure do |c|
  c.default_evaluator_model = "claude-sonnet-4"
end

# Then use evaluator: true to enable with the default
session = Ask::Agent::Session.new(model: "gpt-4o", evaluator: true)

Custom rubric

evaluator = Ask::Agent::Evaluator.new(
  model: "claude-sonnet-4",
  rubric: [
    Ask::Agent::Evaluator::Dimension.new(
      name: "performance",
      description: "Is the implementation efficient?",
      weight: 2
    )
  ]
)

result = evaluator.evaluate(
  goal: "Write an email validator",
  response: agent_output
)
result.accept?  # => true/false
result.scores   # => { performance: 2 }
result.feedback # => "Add edge case for unicode characters"

Events

The evaluator emits its own events during evaluation:

session.on_event do |event|
  case event
  when Ask::Agent::Events::EvaluationStart
    puts "Evaluating against: #{event.dimensions.join(', ')}"
  when Ask::Agent::Events::EvaluationDelta
    print event.content
  when Ask::Agent::Events::EvaluationEnd
    puts "Decision: #{event.decision}"
    puts "Scores: #{event.scores}"
  when Ask::Agent::Events::EvaluationBlocked
    puts "Blocked: #{event.feedback}"
  end
end

Events

Stream session execution in real-time:

session.on_event do |event|
  case event
  when Ask::Agent::Events::TextDelta
    print event.content
  when Ask::Agent::Events::ToolExecutionStart
    puts "\nRunning #{event.name}..."
  when Ask::Agent::Events::ToolExecutionEnd
    puts "  → #{event.duration_ms}ms #{event.is_error ? 'error' : 'ok'}"
  end
end

Extensions

Opt-in safety modules:

  • Permissions — Access control for tools. Supports named access modes (:full_access, :read_only, :ask_before_changes) or custom blocked-tool lists.
  • RateLimiter — Prevent runaway tool calls (configurable per-minute and per-turn limits)
  • AuditLog — Immutable, append-only log of every tool call
extensions = [
  Ask::Agent::Extensions::Permissions.new(mode: :read_only),
  Ask::Agent::Extensions::RateLimiter.new(max_calls_per_minute: 30),
  Ask::Agent::Extensions::AuditLog.new(path: "agent.log")
]

session = Ask::Agent::Session.new(
  model: "gpt-4o",
  tools: [...],
  hooks: {
    before_tool: extensions.map(&:method(:before_tool_call)),
    after_tool: extensions.select { |e| e.respond_to?(:after_tool_call) }.map(&:method(:after_tool_call))
  }
)

Middleware

Wrapping LLM provider calls with cross-cutting behavior:

  • RetryOnFailure — Retry on rate limits and server errors with exponential backoff
  • ModelFallback — Switch to a fallback model+provider on transient errors
  • LogCalls — Log every LLM provider call
  • DefaultSettings — Inject default generation parameters
Ask::Agent.configure do |c|
  c.middleware.use :retry_on_failure, max_retries: 3
  c.middleware.use :model_fallback, fallbacks: [
    { model: "claude-sonnet-4",  provider: :anthropic },
    { model: "gemini-2.0-flash", provider: :google }
  ]
  c.middleware.use :log_calls, logger: Rails.logger
  c.middleware.use :default_settings, temperature: 0.7
end

ModelFallback

When the primary LLM is overloaded or down, ModelFallback transparently switches to a backup model+provider. Credentials for each provider are resolved automatically.

Static fallbacks — ordered list tried in sequence:

c.middleware.use :model_fallback, fallbacks: [
  { model: "claude-sonnet-4",  provider: :anthropic },
  { model: "gemini-2.0-flash", provider: :google }
]

Dynamic fallbacks — lambda that receives the error and request:

c.middleware.use :model_fallback, fallbacks: ->(error, request) {
  if request[:messages].sum { |m| m[:content].to_s.length } > 100_000
    [{ model: "claude-sonnet-4", provider: :anthropic }]  # long-context
  else
    [{ model: "gpt-4o-mini", provider: :openai }]           # cheaper
  end
}

Custom eligible errors — by default rate limits, server errors, and service unavailable:

c.middleware.use :model_fallback,
  fallbacks: [{ model: "claude-sonnet-4", provider: :anthropic }],
  eligible_errors: [Ask::RateLimitError, Ask::ServerError]

Configuration

Ask::Agent.configure do |c|
  c.default_model = "claude-sonnet-4"
  c.default_max_turns = 50
  c.compactor_enabled = true
  c.compactor_threshold = 0.8
  c.parallel_tool_execution = true
  c.max_tool_retries = 3
end

Persistence

store = Ask::Agent::Persistence::InMemory.new
session = Ask::Agent::Session.new(model: "gpt-4o", persistence: store)
session.run("Hello")
session.save  # persisted to store

Development

bundle exec rake test

License

MIT