Agentf

Agentf is a Ruby CLI and installer for running multi-agent development workflows with shared Redis memory.

It helps you install provider manifests, run workflows, inspect memory, and expose tools via MCP.

1. Install prerequisites

• Ruby 3.3+
• Redis Stack (required for RedisJSON + RediSearch)

docker run -d -p 6379:6379 redis/redis-stack:latest

2. Install the gem (local development)

bundle exec rake install

Or install the released gem:

gem install agentf

3. Verify install

agentf version
agentf help

If you use GitHub Copilot in VS Code, agentf install --provider=copilot --scope=local also writes .vscode/mcp.json so Copilot can start the local agentf mcp-server and discover tools like agentf-memory-recent.

Set environment variables (or create a .env file). Example:

export REDIS_URL=redis://localhost:6379
export AGENTF_PROJECT_NAME=my-project

Install provider manifests (example providers shown):

agentf install --provider=opencode,copilot --scope=local

Try useful commands after install:

agentf memory recent -n 5
agentf code glob "lib/**/*.rb"
agentf metrics summary -n 100
agentf mcp-server
agentf eval list

• agentf install install provider manifests from agent metadata
• agentf update regenerate manifests when gem version changes
• agentf memory ... inspect/search/add memory entries
• agentf code ... explore project files and patterns
• agentf architecture ... analyze layer distribution
• agentf metrics ... inspect workflow quality and provider parity
• agentf mcp-server run MCP server over stdio
• agentf eval ... run black-box eval scenarios against agentf agent
• agentf help show command help

For command details, run:

agentf <command> help

• REDIS_URL Redis connection string (default: redis://localhost:6379)
• AGENTF_PROJECT_NAME project key used for memory isolation
• AGENTF_METRICS_ENABLED enable/disable metrics (default: true)
• AGENTF_WORKFLOW_CONTRACT_ENABLED enable/disable workflow stage checks (default: true)
• AGENTF_WORKFLOW_CONTRACT_MODE workflow stage mode: advisory|enforcing|off (default: advisory)
• AGENTF_AGENT_CONTRACT_ENABLED enable/disable per-agent pre/post checks (default: true)
• AGENTF_AGENT_CONTRACT_MODE per-agent mode: advisory|enforcing|off (default: enforcing)
• AGENTF_GEM_PATH optional gem root hint for OpenCode plugin binary resolution

• Episodic memory is stored as episode records with an outcome such as positive, negative, or neutral
• lesson remains distilled memory, and playbook remains procedural memory
• agentf memory search uses embedding-based semantic retrieval
• When Redis Stack supports vector fields and KNN queries, Agentf uses native RediSearch vector search for episodic candidate retrieval
• If vector indexing is unavailable, Agentf automatically falls back to semantic scan/rerank over stored embeddings

• VS Code Copilot discovers local MCP tools from .vscode/mcp.json
• agentf install --provider=copilot --scope=local now writes that file with an agentf stdio server entry
• After install, start or restart the server from VS Code with MCP: List Servers, or open .vscode/mcp.json and use the Start action
• In Copilot Chat, use Agent mode and confirm the agentf server is trusted/running before testing tool calls

If Redis requires auth, include credentials in REDIS_URL (example: redis://:password@localhost:6379).

• Security guidance: docs/security.md
• RubyGems package: https://rubygems.org/gems/agentf

Run dependencies and the test suite locally:

bundle install
bundle exec rspec spec/

When making changes to the CLI or gemspec use bundle exec rake install to install the locally built gem into your system Ruby gems for manual testing.

Agentf includes a black-box eval harness under evals/ for proof-based testing. Each scenario includes:

• scenario.yml metadata such as agent, timeout, tags, and env overrides
• prompt.txt the exact payload sent to agentf agent
• setup.sh optional environment preparation
• verify.sh required assertions against filesystem, CLI output, or Redis state

Supported scenario metadata includes:

• execution_mode: agent|mcp
• retry_on_confirmation: true|false for non-interactive retry flows
• execution_mode: provider for installer + provider-facing artifact proof
• execution_mode: provider_runtime for generated host/plugin runtime proof
• providers / models tags for matrix summaries

Useful commands:

agentf eval list
agentf eval run engineer_episode_positive
agentf eval run engineer_confirmation_retry
agentf eval run mcp_add_lesson
agentf eval run provider_install_opencode
agentf eval run provider_runtime_opencode_recent
agentf eval run provider_runtime_copilot_recent
agentf eval report
agentf eval report --scenario=engineer_confirmation_retry --since=2026-03-16T00:00:00Z
agentf eval run all --json

Artifacts are written to tmp/evals/ by default. The initial scenarios focus on direct agentf agent execution plus in-process MCP tool evals. Opencode installs now default to MCP-first configuration; the legacy plugin runtime remains available as an explicit fallback. Results are also appended to tmp/evals/history.jsonl and aggregated into provider/model matrix summaries so compatibility trends can be compared over time.

MIT