PatientLLM

Integrate LLM APIs with your Ruby backend applications without blocking threads. This gem uses asynchronous HTTP requests to call LLM providers and handles the response via callbacks. It supports multiple API formats natively via PromptBuilder serializers:

• OpenAI Chat Completions (:chat_completion) -- for OpenAI and compatible providers
• OpenAI Responses (:open_responses) -- for the newer OpenAI Responses API
• Anthropic Messages (:messages) -- for the Anthropic Claude API
• Bedrock Converse (:converse) -- for AWS Bedrock Converse API
• Gemini (:gemini) -- for the Google Gemini API

LLM API calls can take a long time to complete. With traditional synchronous HTTP clients, these requests tie up application threads while waiting for responses. This gem solves that problem by using async HTTP via PatientHttp, freeing up your threads to do other work while waiting for the LLM provider to respond.

Configure a provider (built-in presets know each vendor's URL, API format, and authentication header):

PatientLLM.configure do |config|
  config.provider :anthropic, preset: :anthropic, api_key: -> { ENV["ANTHROPIC_API_KEY"] }
end

Define an agent:

class TripPlannerAgent < PatientLLM::Agent
  provider :anthropic
  model "claude-sonnet-4-5"
  system "You are a travel assistant. Be concise."
  max_output_tokens 2_000

  tool :weather, "Get the weather forecast for a city" do
    param :city, :string, "City name", required: true
    param :country, :string
  end

  output do
    field :summary, :string, required: true
    field :packing_list, array: :string
  end

  # Tool handler: the instance method with the tool's name.
  def weather(city:, country: nil)
    WeatherService.forecast(city: city, country: country)
  end

  # Runs in the worker when the final response arrives (after any tool rounds).
  def completed(response)
    trip = Trip.find(response.context[:trip_id])
    trip.update!(plan: response.object, agent_state: response.state)
  end

  def failed(failure)
    Rails.logger.error("Trip planning failed: #{failure.error_type} #{failure.message}")
  end
end

Call it:

# Asynchronously through your job system:
TripPlannerAgent.ask("Plan a weekend in NYC", context: {trip_id: trip.id})

# Continue a saved conversation:
TripPlannerAgent.continue(trip.agent_state, "Make it kid-friendly", context: {trip_id: trip.id})

# Inline (blocking) for consoles, development, and tests:
response = TripPlannerAgent.ask!("Plan a weekend in NYC", context: {trip_id: trip.id})
response.text                 # the raw response text
response.object["summary"]    # parsed structured output per the output schema
response.usage.output_tokens  # token usage

Everything the agent declares stays in code — only JSON-safe data (the serialized conversation, the agent's class name, your context, and the resolved request metadata: provider and serializer names, per-request overrides, and tool-loop counters) travels through the job queue. Tool handlers, API keys, and hooks are re-resolved in the worker process.

This gem delegates HTTP dispatch to patient_http. In production you get a request handler by adding one of the job-system integrations:

• patient_http-sidekiq
• patient_http-solid_queue

For development, consoles, and tests you can execute requests synchronously in-process instead:

PatientHttp.inline!   # register the inline handler (no job system needed)

Verify everything is wired at boot (surfaces missing secrets, preprocessors, or handlers as boot errors instead of dispatch-time job failures):

PatientLLM.verify_configuration!

Presets bundle the vendor-specific details — base URL, API format, authentication header name, and key format:

PatientLLM.configure do |config|
  config.provider :openai, preset: :openai, api_key: -> { ENV["OPENAI_API_KEY"] }
  config.provider :anthropic, preset: :anthropic, api_key: -> { ENV["ANTHROPIC_API_KEY"] }
  config.provider :gemini, preset: :gemini, api_key: -> { ENV["GEMINI_API_KEY"] }
  config.provider :bedrock, preset: :bedrock_runtime, region: "us-east-1", preprocessors: :aws_sigv4
end

The api_key is registered as a PatientHttp secret named patient_llm.<provider>.api_key and referenced from the provider's authentication header. The key value is resolved on the processor side at dispatch time and is never serialized into job payloads. Prefer a lambda (resolved lazily, supports rotation); a plain String also works.

Bedrock supports two authentication styles: pass api_key: with an Amazon Bedrock API key (sent as a bearer token), or sign requests with IAM credentials using SigV4 as shown above — see request signing.

Every preset value can be overridden, so pointing a preset at a proxy or gateway is one keyword:

config.provider :my_proxy, preset: :openai, url: "https://llm-gateway.internal", api_key: -> { ENV["GATEWAY_KEY"] }

To keep managing your own secret names, pass a secret reference and no registration happens:

config.provider :openai, preset: :openai, api_key: PatientHttp.secret("openai.bearer_token")

Providers without a preset use the explicit form. Authentication headers must be secret references — inline API keys raise an error so keys can never end up serialized in the job queue:

PatientLLM.configure do |config|
  config.provider :local, url: "http://localhost:1234", headers: {}

  config.provider :custom,
    url: "https://llm.internal",
    headers: {"x-api-key" => PatientHttp.secret("custom.api_key")},
    serializer: :messages,
    timeout: 300,               # per-provider request timeout (seconds)
    max_tool_iterations: 25     # per-provider tool loop cap
end

# Register the referenced secret with PatientHttp (can be done from any initializer;
# registrations are applied to the job integration's configuration whenever it configures):
PatientHttp.register_secret("custom.api_key") { ENV["CUSTOM_API_KEY"] }

Provider options: url, headers, serializer, path, params (merged into every payload), preprocessors, timeout, stream, read_timeout, max_response_size (see streaming responses), max_tool_iterations, api_key (see provider presets), and region (used by region-templated presets such as :bedrock_runtime).

Every provider option (except name, preset, and api_key) may also be a callable, evaluated each time the provider is looked up — i.e. on every request — so values can be generated dynamically at runtime (api_key is consumed at registration time to build the authentication header; a callable key is resolved on the processor side at dispatch):

PatientLLM.configure do |config|
  config.provider :gateway,
    url: -> { LLMConfiguration.gateway_url },
    timeout: -> { LLMConfiguration.request_timeout }
end

Validation of a callable's result (serializer names, authentication headers) happens at lookup time instead of registration time.

Job-system integrations for patient_http can run multiple named processors with different configurations — for example, one tuned for long-running LLM requests and another for quick webhook deliveries. Set processor to route every LLM request to a named processor; it can also be a callable, resolved on every request. When unset, requests use the default processor. A per-request processor: option on ask overrides the configured value.

PatientLLM.configure do |config|
  config.processor = :llm
end

LLM requests that generate for a long time can fail without streaming: the connection sits idle while the model works, and proxies, NAT gateways, or the provider's own front end may drop it before the response arrives. Anthropic, for example, recommends streaming for any request that might take longer than 10 minutes. With stream: true, the provider sends the response as it is generated, so data keeps flowing for the whole request.

The streamed body is reassembled into the same response the non-streaming endpoint would have returned before your callback runs (with prompt_builder's Response.parse_stream), so callbacks, hooks, tool loops, and structured output work the same way. Tokens are not delivered to your code as they arrive.

PatientLLM.configure do |config|
  config.provider :anthropic, preset: :anthropic, api_key: -> { ENV["ANTHROPIC_API_KEY"] },
    stream: true,
    timeout: 30 * 60,                    # limit for the full stream
    read_timeout: 120,                   # limit for each wait for data
    max_response_size: 16 * 1024 * 1024  # streamed bodies are larger than JSON bodies
end

• stream turns streaming on for the provider's requests. A stream: option on ask (or an agent's stream declaration) overrides it for a request, including stream: false. A stream key in params is treated the same way: request settings win over provider settings, and at each level the stream option wins over a stream param. When none is set, the session's stream field applies.
• timeout still limits the full request, so raise it for long generations.
• read_timeout is the maximum number of seconds to wait for data from the server. The timer starts again each time a chunk arrives, so a stream that stops fails fast while a long stream that keeps sending data runs until timeout. It only applies to streamed requests: without streaming, the server sends nothing until it has generated the whole response.
• max_response_size overrides the PatientHttp max_response_size (1MB by default) for the provider's requests. A streamed body is several times larger than the equivalent JSON body.

Streaming works with every API format. The Chat Completions, Open Responses, and Messages formats send stream: true in the request body. Gemini and Bedrock Converse stream from a different endpoint, which is used by default when streaming (v1beta/models/{model}:streamGenerateContent?alt=sse and model/{model}/converse-stream); an explicit path is used as is.

A stream that reports an error after the response started (such as an Anthropic overloaded_error event, which arrives with a 200 status), ends before its final event, or is malformed is sent to on_error (or an agent's failed hook) as a PatientHttp::RequestError with an error_type of :stream_error. The error_class names the prompt_builder error (PromptBuilder::ErrorResponseError, PromptBuilder::IncompleteStreamError, or PromptBuilder::ParseError), and http_response holds the streamed body.

Streaming requires patient_http 1.8.0 or later.

Some providers require request signing rather than a static authentication header — for example, AWS Bedrock with SigV4, where a signature is computed over the final outgoing request. For these, register a request preprocessor on the PatientHttp configuration and reference it by name from the provider. Like secrets, only the preprocessor name is serialized into the job queue; the signing logic and credentials stay on the processor side.

For SigV4, PatientLLM::AwsRequestSigner is a ready-made callable that can be registered directly as the preprocessor:

PatientLLM.configure do |config|
  config.provider :bedrock, preset: :bedrock_runtime, region: "us-east-1", preprocessors: :aws_sigv4
end

PatientHttp::Sidekiq.configure do |config|
  config.register_preprocessor(:aws_sigv4, PatientLLM::AwsRequestSigner.new(
    credentials: Aws::CredentialProviderChain.new
  ))
end

credentials: is required and accepts a credential chain (anything responding to resolve, like Aws::CredentialProviderChain; resolved lazily on the first request), a credentials provider (responding to credentials), or a static credentials object (responding to access_key_id and secret_access_key, like Aws::Credentials). The signing service: and region: can be passed explicitly; when omitted they are derived from each request's URL host for standard <service>.<region> AWS endpoints, including dual-stack api.aws hosts (bedrock-runtime.us-east-1.amazonaws.com signs as service "bedrock" in region "us-east-1"; bedrock-mantle.us-east-1.api.aws signs under its own "bedrock-mantle" service name).

The signer needs the aws-sigv4 gem, which is not a dependency of this gem — add it to your bundle (it is included with aws-sdk-core, which also provides the credential chain).

Multiple preprocessors can be given as an array; they run in order at dispatch time.

PatientLLM::Agent is the high-level way to use this gem: one class declaring the provider, model, generation settings, tools, output schema, and completion handling.

class ResearchAgent < PatientLLM::Agent
  provider :openai              # a registered provider name
  model "gpt-5"
  system "You are a research assistant."
  instructions "Cite your sources."
  temperature 0.2
  max_output_tokens 4_000
  reasoning :medium             # portable effort level, or reasoning budget_tokens: 8_000
  max_tool_iterations 5         # tool loop cap for this agent (default 10)
  stream true                   # stream responses (see "Streaming responses")
  extra guardrail_config: {guardrailIdentifier: "gr-1", guardrailVersion: "1"}
end

instructions sets system-level instructions for the request. On APIs without a separate instructions field they are appended to the system prompt.

reasoning configures reasoning (extended thinking), applied with prompt_builder's session.think. It accepts a portable effort level (:minimal, :low, :medium, :high, :xhigh, :max), effort: or budget_tokens:, or false to turn reasoning off. nil and false are different: reasoning nil means no reasoning configuration, so the request carries no reasoning parameter and the API's default applies (some models, such as recent Claude models, reason by default), while reasoning false explicitly turns reasoning off (e.g. thinking: {type: "disabled"} for Claude models). An effort level can accompany false (reasoning false, effort: :low) for APIs such as Anthropic Messages, where effort also limits token spend without thinking; APIs whose only reasoning control is effort ignore it and keep reasoning off.

reasoning :medium                   # portable effort level
reasoning budget_tokens: 8_000      # explicit thinking budget, where supported
reasoning true                      # reasoning on, depth left to the API
reasoning false                     # reasoning off
reasoning false, effort: :low       # reasoning off, with an effort level on Messages
reasoning nil                       # no reasoning configuration (removes an inherited value)

The reader returns the normalized session.think options ({effort: "medium"}, {budget_tokens: 8000}, {}, {enabled: false}, or nil).

extra sets provider-specific extra data on every session the agent builds. The recognized keys depend on the serializer used for the request — for example, the Bedrock Converse serializer maps guardrail_config, stop_sequences, additional_model_request_fields, and performance_config into the request payload (see the prompt_builder docs for each serializer's keys). Redeclaring extra in a subclass replaces the whole hash, and continue re-applies the agent's current value to restored sessions. A per-request ask(extra: {...}) replaces the declaration for that request; to add to it instead, merge explicitly: MyAgent.ask(msg, extra: MyAgent.extra.merge(...)). Session options such as extra: are only accepted when ask builds the session — passing them to continue, or together with a session:, raises ArgumentError.

Every scalar declaration (provider, model, system, instructions, temperature, max_output_tokens, reasoning, max_tool_iterations, and extra) also accepts a block in lieu of the argument, or a callable (anything responding to call) as the argument. The block or callable is evaluated each time the agent builds or sends a request, so the value can be generated dynamically at runtime:

class ResearchAgent < PatientLLM::Agent
  extra { LLMConfiguration.extra_hash }
  temperature -> { AppConfig.llm_temperature }
end

Validation and coercion (extra's Hash check, provider's symbol coercion, reasoning's normalization) apply to the block's result each time it is evaluated. A reasoning block can return any value the static form accepts — an effort level, a Hash such as {budget_tokens: 2048}, false to turn reasoning off, or nil for no configuration:

class SupportAgent < PatientLLM::Agent
  model { LLMExperiment.model }
  reasoning { LLMExperiment.reasoning_effort }   # e.g. :low, :high, or false
  max_output_tokens { LLMExperiment.max_output_tokens }
end

Per-request session options override the declarations for one request. The reasoning: option is normalized the same way as the declaration, so reasoning: :low, reasoning: {budget_tokens: 2048}, and reasoning: false produce the same request as the equivalent reasoning declaration, and reasoning: nil sends no reasoning configuration for that request. Any other Hash (e.g. {summary: "auto"}) is set as the session's raw reasoning configuration for provider-specific settings:

SupportAgent.ask(message, reasoning: false, max_output_tokens: 1_000)

Subclasses inherit every declaration from their parent agent class, including tools and the output schema — so a base agent can hold shared configuration while subclasses specialize. Inheritance is live: getters fall back to the parent class, so declarations added to the parent later are visible to existing subclasses.

Override a setting by redeclaring it, or remove an inherited scalar setting by passing an explicit nil (tools and the output schema cannot be removed this way):

class BaseAgent < PatientLLM::Agent
  provider :openai
  model "gpt-5"
  temperature 0.2

  tool :search, "Search the knowledge base" do
    param :query, :string, required: true
  end
end

class CreativeAgent < BaseAgent
  temperature 0.9     # override
end

class DefaultTemperatureAgent < BaseAgent
  temperature nil     # remove: use the provider's default temperature
end

Tools merge by name — redeclaring a tool in a subclass replaces the inherited declaration for that subclass only.

Declare the schema and define the handler together. The handler is the instance method with the tool's name, receiving the LLM-provided arguments as keywords:

tool :search, "Search the knowledge base" do
  param :query, :string, "The search query", required: true
  param :limit, :integer
end

def search(query:, limit: nil)
  KnowledgeBase.search(query, limit: limit || 10)   # String or JSON-able return value
end

A raw JSON Schema hash is also accepted: tool :search, "...", parameters: {...}.

When the model responds with tool calls, the gem automatically executes the matching methods, appends the results to the session, and re-issues the request — until the model returns a plain text response or the iteration cap is reached. Your completed hook only fires for the final response; define tool_round(response) to observe intermediate rounds. Raise PatientLLM::HaltError.new(content: "...") from a tool method to stop the loop and surface the content as the final response.

Each hook receives a single object bundling the whole invocation:

• completed(response) and tool_round(response) receive a PatientLLM::Agent::Response exposing text, object (parsed structured output; completed only — an intermediate tool_round response has none, and calling object there raises StructuredOutputError), state (the serializable session), usage, model, tool_calls, session, context (what you passed to ask/continue), http_response (the PatientHttp::Response — status, headers, body, duration), and http_request_id. In completed the HTTP exchange is the final request's; in tool_round it is that round's.
• failed(failure) receives a PatientLLM::Agent::Failure exposing the error (with error_type, message, and error_class delegated for convenience) alongside the same session, state, context, http_response, and http_request_id. The http_response is nil for non-HTTP errors such as timeouts and connection failures.

Both objects support [] as a shorthand for reading a context value, so response[:trip_id] is the same as response.context[:trip_id] (and raises KeyError if the key was not passed to ask/continue).

def completed(response)
  Trip.find(response[:trip_id]).update!(itinerary: response.object)
end

A tool method that declares a context: keyword receives the context too:

def search(query:, context:)
  KnowledgeBase.for_user(context[:user_id]).search(query)
end

Inside hooks and tool methods the agent also exposes session and provider instance readers.

Pass callback: to ask, continue, or ask! to send the hooks to another class instead of the agent. The agent still supplies the provider, model, tools, and output schema — only the hooks move:

class TripCallbacks
  def completed(response)
    Trip.find(response[:trip_id]).update!(itinerary: response.object)
  end

  def failed(failure)
    ErrorReporter.notify(failure.error, trip_id: failure[:trip_id])
  end
end

TripPlannerAgent.ask("Plan a weekend in NYC", context: {trip_id: trip.id}, callback: TripCallbacks)

The class is instantiated fresh in the worker for each invocation and may implement any subset of completed, failed, and tool_round; a hook it doesn't implement falls back to the agent's own. Only the class name travels through the queue, so it must be a named, resolvable class — an anonymous class raises ArgumentError, and one implementing none of the three hooks raises ArgumentError when ask is called.

output do
  field :answer, :string, required: true
  field :confidence, :number
  field :sources, array: :object do
    field :title, :string
    field :url, :string
  end
end

def completed(response)
  response.object   # => parsed Hash matching the schema
end

response.object raises PatientLLM::StructuredOutputError (with the raw text attached) if the response cannot be parsed. A raw JSON Schema hash is also accepted: output schema: {...}.

response.state is a JSON-native hash of the full conversation. Persist it anywhere, then continue:

def completed(response)
  Conversation.find(response.context[:conversation_id]).update!(state: response.state)
end

# Later:
ResearchAgent.continue(conversation.state, "Tell me more about that", context: {conversation_id: conversation.id})

continue re-applies the agent's current model, system, tools, and output schema to the restored session, so deploying changes to an agent cleanly affects older conversations.

Passing an existing session: to ask is different: the session is sent as-is and the agent's declarations (model, system, tools, output schema) are not applied to it. Use continue for restored conversations.

ask! runs the request (and the whole tool loop) synchronously and returns the final response — no job system required. The completed/failed hooks still run, so tests exercise the real code paths:

response = ResearchAgent.ask!("What is the capital of France?")
response.text   # => "Paris."

preview_request builds the request that ask would send — through the same resolution logic — without sending anything. It returns a PatientLLM::RequestPreview with the resolved url, headers, and JSON payload. Header values that reference registered secrets are replaced with placeholders, and request preprocessors (which run at send time) are not applied:

preview = ResearchAgent.preview_request("What is the capital of France?")
preview.url       # => "https://api.openai.com/v1/chat/completions"
preview.payload   # => {"model" => "gpt-4o", "messages" => [...], ...}

Agents compile down to this API; use it directly when you need full control.

Create a callback class with on_complete and on_error methods. Callbacks receive keyword arguments, and you only declare the ones you need — the dispatcher inspects your method signature and passes just those values (or everything if you declare **kwargs):

class LLMCallback
  def on_complete(session:, provider:, llm_response:, callback_args:, http_response:, request_id:)
    # session       - the PromptBuilder::Session with the response already added
    # provider      - the provider name (String)
    # llm_response  - a PromptBuilder::Response with the assistant's response
    # callback_args - a PatientHttp::CallbackArgs containing data you passed in the `ask` call
    # http_response - the raw PatientHttp::Response
    # request_id    - the original request id (stable across tool-call iterations)

    puts llm_response.text
    puts "Tokens: #{llm_response.usage.input_tokens} in / #{llm_response.usage.output_tokens} out"

    save_session_state(callback_args[:user_id], session.to_h)
  end

  def on_error(session:, provider:, callback_args:, error:, http_response:, request_id:)
    # error is a PatientHttp::RequestError, ClientError (HTTP 4xx),
    # or ServerError (HTTP 5xx). All respond to:
    #   error.error_type  - :timeout, :connection, :ssl, :http_error, etc.
    #   error.message     - human-readable message
    #   error.error_class - the original exception class (for RequestError)
    #   error.request_id
    log_error(error.error_type, error.message)
  end
end

Each callback may declare any subset of the keywords below, in any order. PatientLLM.ask validates your callback's signatures up front and raises an ArgumentError if a method uses an unsupported name or a positional parameter, or if on_error omits its required error keyword.

Create a PromptBuilder::Session and call PatientLLM.ask to make an async request:

session = PromptBuilder::Session.new(model: "gpt-4o")
session.system("You are a helpful assistant.")
session.user("What is the capital of France?")

PatientLLM.ask(session, provider: :openai, callback: LLMCallback, callback_args: {
  user_id: current_user.id
})

PromptBuilder::Session supports the full range of generation options, plus helpers for structured output, reasoning, and tools:

session.temperature = 0.7
session.max_output_tokens = 1000
session.json_output(my_json_schema)      # structured output; parse with llm_response.parsed_json
session.think(effort: :medium)           # portable reasoning configuration
session.use_tools("weather", "search")   # attach tools registered on PromptBuilder.tool_registry

PatientLLM.ask accepts per-request overrides:

PatientLLM.ask(session,
  provider: :openai,
  callback: LLMCallback,
  url: "http://localhost:1234",            # Override the provider's base URL
  serializer: :messages,                   # Override the API format
  path: "/chat/completions",               # Override the endpoint path
  headers: {"X-Custom" => "value"},        # Additional HTTP headers
  params: {max_completion_tokens: 1000},   # Additional request parameters
  preprocessors: :aws_sigv4,               # Replace the provider's request preprocessors
  timeout: 600,                            # Request timeout in seconds
  stream: true,                            # Stream the response (see "Streaming responses")
  read_timeout: 120,                       # Maximum wait for data on a streamed request
  max_response_size: 16 * 1024 * 1024,     # Maximum response body size in bytes
  max_tool_iterations: 3,                  # Tool loop cap for this request
  processor: :llm                          # Named PatientHttp processor for this request
)

headers and params are merged on top of the provider's configured values, while the other options replace the provider defaults. All overrides are preserved across automatic tool-loop iterations.

PatientLLM.preview_request accepts the same arguments (minus callback:) and returns the request that ask would send — a PatientLLM::RequestPreview with the resolved url, headers (secret values redacted), and JSON payload — without enqueuing or executing anything:

preview = PatientLLM.preview_request(session, provider: :openai)
preview.payload   # => {"model" => "gpt-4o", "messages" => [...]}

Tools can also be registered on the global PromptBuilder.tool_registry with their handler, then attached to sessions by name — no schema duplication:

PromptBuilder.register_tool(
  "weather",
  description: "Get the current weather for a location",
  parameters: {type: "object", properties: {location: {type: "string"}}, required: ["location"]}
) do |args|
  WeatherService.lookup(args["location"])
end

session.use_tools("weather")
PatientLLM.ask(session, provider: :openai, callback: LLMCallback)

The automatic tool loop works the same as with agents. When the cap is exceeded, on_error receives a PatientHttp::RequestError whose error_type is :max_tool_iterations.

Run any request synchronously with PatientLLM.inline:

PatientLLM.inline do
  PatientLLM.ask(session, provider: :openai, callback: LLMCallback)
end

Or register the PatientHttp inline handler globally for a console or test process with PatientHttp.inline!.

The full request URL is built by concatenating the base URL (from the provider registry or the url: option) with the path. When you don't set path, it defaults to the path for the active serializer (v1/chat/completions for :chat_completion, v1/responses for :open_responses, v1/messages for :messages, model/{model}/converse for :converse, v1beta/models/{model}:generateContent for :gemini). A {model} placeholder in the path is replaced with the session's model, percent-encoded as a single path segment, at dispatch time. If your base URL already includes a /v1 prefix, override the path to avoid duplication:

PatientLLM.ask(session,
  provider: :openai,
  callback: LLMCallback,
  url: "https://my-gateway.internal/openai/v1",
  path: "chat/completions"
)

Sessions serialize to JSON for storage and later restoration:

# In your callback, save the state (the response is already in the session):
def on_complete(session:, callback_args:, **)
  save_to_database(callback_args[:conversation_id], session.to_h)
end

# Later, restore and continue:
session = PromptBuilder::Session.from_h(load_from_database(conversation_id))
session.user("Tell me more about that.")
PatientLLM.ask(session, provider: :openai, callback: LLMCallback)

The serialized session travels through the job queue with each request. For conversations that can get large (e.g. with file attachments), configure automatic offloading to a PatientHttp payload store:

PatientLLM.configure do |config|
  config.session_offload payload_store: :s3, threshold: 65_536
end

Sessions above the threshold are written to the store and passed by reference. Offloaded payloads are not deleted after use (so job retries keep working); use a store with managed expiration such as a Redis TTL or an S3 lifecycle rule.

This gem is not yet published to RubyGems. Add it from GitHub:

gem "patient_llm", github: "bdurand/patient_llm"

Then execute:

$ bundle

Open a pull request on GitHub.

Please use the standardrb syntax and lint your code with standardrb --fix before submitting.

The gem is available as open source under the terms of the MIT License.