lex-llm-ollama

LegionIO LLM provider extension for Ollama.

This gem lives under Legion::Extensions::Llm::Ollama and depends on lex-llm >= 0.4.3 for shared provider-neutral routing, response normalization, fleet envelopes, responder execution, transport, and registry primitives. It does not carry a runtime legion-llm dependency; legion-llm owns higher-level routing and discovers this provider through normal extension loading.

Load it with require 'legion/extensions/llm/ollama'.

All responses pass through the shared Lex-llm normalization layer: Message, Chunk, Embedding, and Model::Info.

lib/
  legion/extensions/llm/ollama.rb              # Extension entry point, instance discovery, default settings
  legion/extensions/llm/ollama/provider.rb     # Provider — chat, stream, embed, models, offerings
  legion/extensions/llm/ollama/version.rb      # VERSION constant
  legion/extensions/llm/ollama/actors/
    discovery_refresh.rb                       # Periodic model discovery actor (Every, 30min default)
    fleet_worker.rb                            # Fleet request subscription actor (Subscription)
  legion/extensions/llm/ollama/runners/
    fleet_worker.rb                            # Fleet request execution runner (delegates to lex-llm)

Legion::Extensions::Llm::Ollama
├── Provider                          # Ollama provider implementation
│   ├── Capabilities                  # Capability predicates (chat, streaming, vision, functions, embeddings)
│   ├── #render_payload               # Build Ollama chat payload from messages, tools, schema
│   ├── #stream_response              # NDJSON streaming via Faraday on_data
│   ├── #discover_offerings           # Build ModelOffering array from live/cached models
│   ├── #fetch_model_detail           # Call /api/show, extract context_window + capabilities
│   ├── #render_embedding_payload     # Build Ollama embedding payload
│   └── (inherited from lex-llm)      # Chat, embedding, connection, registry helpers
├── Actor::DiscoveryRefresh           # Every actor; refreshes model list, repopulates auto rules
├── Actor::FleetWorker                # Subscription actor; gates on respond_to_requests
└── Runners::FleetWorker              # Module function; delegates to ProviderResponder.call

Shared from lex-llm:
├── Fleet::ProviderResponder          # Fleet request execution harness
├── RegistryPublisher                 # Publishes readiness + model events to llm.registry
├── RegistryEventBuilder              # Builds registry event payloads
├── AutoRegistration                  # Self-registers discovered instances
└── CredentialSources                 # Socket probing + setting lookup for instance discovery

• default_settings — Returns the full settings schema via Lex-llm.provider_settings.
• provider_class — Returns Provider.
• discover_instances — Probes 127.0.0.1:11434 socket + reads configured instances from settings.
• normalize_instance_config(config) — Normalizes endpoint/api_base/ollama_api_base aliases to base_url.
• registry_publisher — Lazily instantiated RegistryPublisher for the :ollama family.

Extends Legion::Extensions::Llm::Provider. Implements the Ollama-specific contract:

Module functions providing capability predicates used during offering construction:

Static fallback map keyed by model name prefix (e.g., 'qwen3' => 128_000). Used when /api/show is unavailable to infer context window. Covers qwen, llama, gemma, mistral, deepseek, phi, command-r, codellama, and embedding families.

An Every actor that runs every 30 minutes (configurable via settings[:extensions][:llm][:ollama][:discovery_interval]). On each tick:

1. Calls Legion::LLM::Discovery.refresh_discovered_models!(provider: :ollama)
2. Repopulates auto routing rules if Legion::LLM::Router is available
3. Invalidates the offerings cache if Legion::LLM::Inventory is available

A Subscription actor that starts only when at least one instance has fleet.respond_to_requests: true. Routes messages to the fleet worker runner.

A module with handle_fleet_request(payload, delivery:, properties:). Delegates to Legion::Extensions::Llm::Fleet::ProviderResponder.call with the Ollama provider family, provider class, and instance discovery lambda.

Legion::Extensions::Llm::Ollama.default_settings
# {
#   enabled: true,
#   provider_family: :ollama,
#   instances: {
#     default: {
#       endpoint: 'http://127.0.0.1:11434',
#       default_model: 'qwen3.5:latest',
#       tier: :local,
#       transport: :http,
#       credentials: {},
#       usage: { inference: true, embedding: true, image: false },
#       limits: { concurrency: 1 },
#       fleet: {
#         enabled: false,
#         respond_to_requests: false,
#         capabilities: %i[chat stream_chat embed],
#         lanes: [],
#         concurrency: 1,
#         queue_suffix: nil
#       }
#     }
#   }
# }

discover_instances auto-detects a local instance when the socket at 127.0.0.1:11434 is reachable. Additional instances can be defined in settings using any of the recognized endpoint aliases (base_url, endpoint, api_base, ollama_api_base); the extension normalizes all to base_url.

extensions:
  llm:
    ollama:
      discovery_interval: 1800          # DiscoveryRefresh actor interval (seconds)
      instances:
        lab:
          base_url: http://ollama-lab:11434
          default_model: qwen3.5:latest

Provider instances can opt in to consuming Legion LLM fleet requests. The fleet actor only starts when at least one instance enables respond_to_requests, and the runner delegates execution to the shared lex-llm responder helper.

extensions:
  llm:
    ollama:
      instances:
        local:
          fleet:
            enabled: true
            respond_to_requests: true
            capabilities:
              - chat
              - stream_chat
              - embed

Every rescue block uses handle_exception from Legion::Logging::Helper with explicit level, handled:, and operation: parameters. Connection failures during discover_offerings produce a warn-level log and return an empty array (never raise).

require 'legion/extensions/llm/ollama'

# Access the module
Legion::Extensions::Llm::Ollama.discover_instances
Legion::Extensions::Llm::Ollama.default_settings

# Create a provider instance (usually done by lex-llm routing)
provider = Legion::Extensions::Llm::Ollama::Provider.new(config:)

# Discover offerings
provider.discover_offerings(live: true)

# Chat
result = provider.chat(messages: [...], model: 'llama3', temperature: 0.7)

# Stream chat
provider.stream_chat(messages: [...], model: 'llama3') do |chunk|
  print chunk.content
end

# Embeddings
embeddings = provider.embed(text: "Hello world", model: 'nomic-embed-text')

cd /Users/matt.iverson@optum.com/rubymine/legion/extensions-ai/lex-llm-ollama
bundle install

# Run specs
bundle exec rspec

# Lint (auto-correct)
bundle exec rubocop -A

Spec count: 52 examples across 7 spec files.

MIT