inquirex-llm

LLM integration verbs for the Inquirex questionnaire engine.

Extends the core DSL with two server-side verbs: extract (alias: clarify), which turns free-text answers into structured data, and summarize, which closes a flow with a prose summary of the whole session. Ships with a pluggable adapter interface and a NullAdapter for testing. (describe and detour are temporarily parked.)

inquirex is a pure Ruby, declarative, rules-driven questionnaire engine for building conditional intake forms, qualification wizards, and branching surveys.

require "inquirex-llm" injects the LLM verbs into the core Inquirex.define DSL.

No separate entry point needed.

require "inquirex"
require "inquirex-llm"

definition = Inquirex.define id: "tax-intake-2026", version: "1.0.0" do
  start :description

  ask :description do
    type :text
    question "Describe your business in a few sentences."
    transition to: :extracted
  end

  extract :extracted do
    from :description
    prompt "Extract structured business information from the description."
    schema industry:          :string,
           entity_type:       :string,
           employee_count:    :integer,
           estimated_revenue: :currency
    model :claude_sonnet
    temperature 0.2
    transition to: :done
  end
  
  say :done do
    text "Thank you! We'll be in touch."
  end
end

All core verbs (ask, say, header, btw, warning, confirm) and widget hints work alongside LLM verbs in the same Inquirex.define block.

Since inquirex 0.7.0, Inquirex.load_dsl validates source against a default-deny allowlist before evaluating it — a word nobody declared is a violation, not an oversight. Requiring this gem registers its own vocabulary (extract, clarify, summarize, and the methods legal inside their blocks), so a flow stored in a database or authored in a visual builder loads normally:

Inquirex.load_dsl(customer.flow_dsl)   # validates, then evaluates

fallback is the one deliberate exception: it takes a Ruby block, which is exactly what static validation cannot vet, so it is excluded from the allowlist rather than permitted. Flows that need it must be loaded from your own source with unsafe: true.

Registration is skipped on inquirex versions predating the allowlist, so the gem still works against them.

Extract structured data from a free-text answer. Requires from (or from_all), prompt, and schema. The stored/serialized verb is always "extract"; clarify is a DSL-only alias.

extract :business_extracted do
  from :business_description
  prompt "Extract structured business information."
  schema industry: :string, employee_count: :integer, revenue: :currency
  model :claude_sonnet
  temperature 0.2
  max_tokens 1024
  transition to: :next_step
end

Closes a flow with a multi-paragraph prose summary of the whole session, written for the user to read and keep.

Inquirex.define id: "depreciation-help" do
  start :intro

  say(:intro) { text "Depreciation spreads an asset's cost over its useful life."; transition to: :asset }
  ask(:asset) { type :enum; question "What kind of asset?"; options vehicle: "A vehicle", building: "A building" }

  summarize :wrap_up do
    temperature 0.4
  end
end

Two things make it different from every other verb.

It reads the transcript, not the answers. The core gem's text accumulators record everything the user was shown and every answer they gave. That is what summarize is given. It matters because a flow that mostly explains things — a help widget, an explainer — collects almost no answers, so a summary built from the answers hash would have nothing to say. Declaring summarize adds a :transcript accumulator automatically if the flow declares no text accumulator of its own.

The gem owns the prompt. Inquirex::LLM::Prompts::SUMMARIZE is a constant, and adapters read it from there rather than from node.prompt. A flow author who could replace it could make the summary assert things the session never established, over a signature that still reads as the application's own words — so prompt is rejected at definition time rather than ignored at runtime, and a hand-forged or tampered node cannot substitute its own instructions either.

The prompt constrains the model to what the transcript establishes, and to the markdown subset a renderer can lay out and print: headings, lists, blockquotes, emphasis, inline code, and fenced code blocks.

Each raises Errors::DefinitionError naming the reason.

temperature, model, and max_tokens remain available: they change how the summary is generated, not what it is allowed to say.

summarize uses its own adapter method, because nothing about the call resembles an extraction — the prompt is the gem's, the input is the transcript, and the result is a markdown String rather than a schema-shaped Hash:

adapter = Inquirex::LLM::AnthropicAdapter.new(api_key: ENV["ANTHROPIC_API_KEY"])
markdown = adapter.summarize(engine.current_step, engine.text(:transcript))

An adapter that implements only #call keeps working for extract and raises NotImplementedError the first time a flow asks it to summarize — loudly, rather than returning something that looks like a summary and is not. An empty transcript raises Errors::AdapterError rather than asking a model to invent one.

The inquirex-widget renders the result with Close and Print buttons; markdown is sanitized against a strict element allowlist before it reaches the page.

Most extract schemas exist to pre-fill questions asked later in the same flow. Declaring those fields twice — once in the schema, once in the question — invites drift, and worse: a hand-typed income_types: :multi_enum gives the LLM no idea which values are legal, so its answers won't match the question's options.

Instead, pass the schema as a list of question ids:

extract :extracted do
  from :description
  prompt "Extract the client's filing status, dependents, and income types."
  schema :filing_status, :dependents, :income_types
  transition to: :filing_status
end

ask :filing_status do
  type :enum
  question "Filing status?"
  options({ "single" => "Single", "mfj" => "Married Filing Jointly" })
  transition to: :dependents
end
# ...

Each symbol is resolved against the flow at definition time — references may point forward to questions defined after the extract step. The gem looks up the question's declared type, and for :enum / :multi_enum questions folds the exhaustive list of allowed option values into the JSON schema sent to the LLM. The adapters then instruct the model to answer using only those values, so extracted answers always match the downstream question's options (and Engine#prefill! can skip the question).

A symbol that matches no ask/confirm step in the flow fails validation with Inquirex::LLM::Errors::DefinitionError — as do references to display-only steps and other LLM steps.

Both forms compose. Use keywords for output fields that have no corresponding question:

schema :filing_status, :income_types, confidence: :decimal

When the schema is built from question references, the schema already tells the LLM the field names, types, and allowed values — the main thing a hand-written prompt still adds is the questions' own wording. prompt :auto generates exactly that at definition time:

extract :extracted do
  from :description
  prompt :auto
  schema :filing_status, :dependents, :income_types
  transition to: :filing_status
end

The generated prompt enumerates each referenced question's text ("- filing_status: What is your filing status for 2025?" …), lists explicit keyword fields by name and type, and instructs the model to leave unsupported fields empty. Generation happens at build time, so the wire format and adapters always see a concrete prompt string — :auto never leaves the DSL. It requires at least one question reference; with only explicit key: :type fields there is no question wording to generate from, and validation fails.

Write the prompt by hand when you need domain framing the questions don't carry ("for tax filing purposes", "map S-Corp to s_corp") — an explicit prompt always wins.

summarize rejects the content-shaping methods rather than ignoring them — see summarize for each refusal and its reason.

The engine treats LLM steps as collecting steps. The server adapter processes the LLM call and feeds the result back:

engine = Inquirex::Engine.new(definition)

engine.answer("I run an LLC with 15 employees, ~$2M revenue.")
# engine.current_step_id => :extracted

# Server-side: adapter calls the LLM
adapter = MyLlmAdapter.new
result = adapter.call(engine.current_step, engine.answers)
# => { industry: "Technology", employee_count: 15, revenue: 2_000_000.0 }

engine.answer(result)
# engine.current_step_id => :summary

The built-in adapters return an Inquirex::LLM::Extraction for an extract step. It is the Hash above (a Hash subclass, so prefill!, == and JSON serialization see only the values) carrying two more things from the one LLM call that produced it: raw, the JSON exactly as the model sent it before enum values were canonicalized (raw[:entity_type] # => "LLC" where result[:entity_type] # => "llc"), and usage, the tokens and, when the provider reports it, the cost.

For testing, use NullAdapter which returns schema-conformant placeholder values without any API calls:

adapter = Inquirex::LLM::NullAdapter.new
result = adapter.call(engine.current_step)
# => { industry: "", employee_count: 0, revenue: 0.0 }

Model ids, the provider and its credentials are set in one place, Inquirex::LLM.configuration. Each setting is read from its environment variable when that variable is set and not blank, and otherwise takes the default below. The adapters fall back on it whenever no api_key: or model: is passed and a node sets no max_tokens or temperature of its own.

An invalid value (an unknown provider, a token count that is not a positive integer, a temperature outside 0 to 2) raises Inquirex::LLM::Errors::ConfigurationError naming the variable. to_h and inspect print every *_api_key as [REDACTED].

ENV is read once, on first use. Code can override any setting afterwards; nil restores the default:

Inquirex::LLM.configure do |c|
  c.provider        = :anthropic
  c.anthropic_model = "claude-opus-5-5"
  c.max_tokens      = 4096
end

adapter = Inquirex::LLM.adapter            # built from the configuration
Inquirex::LLM.reset_configuration!         # discard overrides and read ENV again

Inquirex::LLM.adapter picks the adapter, first match wins:

OPENROUTER_API_KEY and TYPESAFE_API_KEY never select an adapter on their own: a host that has one of those keys for something else keeps the adapter it had, and Jev cannot summarize, so both must be asked for by name.

OpenRouter and Jev are never auto-detected; select them with INQUIREX_LLM_ADAPTER or build them directly.

All adapters use net/http (stdlib, no extra dependency), inject the declared schema into the system prompt as a strict JSON contract, and raise Inquirex::LLM::Errors::AdapterError on HTTP / parse failures and SchemaViolationError when the model's output is missing declared fields.

adapter = Inquirex::LLM::AnthropicAdapter.new   # key and model from the configuration
adapter = Inquirex::LLM::AnthropicAdapter.new(  # or name them; arguments win
  api_key: "sk-ant-...",
  model:   "claude-opus-5-5"                    # or pass the short symbol in the DSL
)

The current model ids live in one constant, Inquirex::LLM::Models::ANTHROPIC (also reachable as AnthropicAdapter::MODELS): claude-fable-5-1, claude-opus-5-5, claude-sonnet-5-5 and claude-haiku-5-5. The default is Models::ANTHROPIC_DEFAULT, claude-sonnet-5-5, unless INQUIREX_LLM_ANTHROPIC_MODEL names another. Recognized model :symbol values in the DSL: :claude_sonnet, :claude_haiku, :claude_opus and :claude_fable, each mapped to its family's id in MODELS. :claude_fable is Anthropic-only; the OpenAI adapter does not map it.

These models reject sampling parameters, so the adapter does not send temperature to them. A temperature in the DSL still reaches a model you name by an older id, such as model: "claude-sonnet-4-6".

adapter = Inquirex::LLM::OpenAIAdapter.new     # key and model from the configuration
adapter = Inquirex::LLM::OpenAIAdapter.new(api_key: "sk-...", model: "gpt-4.1")

Uses Chat Completions with response_format: { type: "json_object" } so the model is constrained to return valid JSON. Recognized DSL symbols: :gpt_4o, :gpt_4o_mini, :gpt_4_1, :gpt_4_1_mini. For cross-provider portability, the adapter also accepts the Claude symbols (:claude_sonnet → gpt-4o etc.) so a flow file that says model :claude_sonnet runs unchanged against either provider.

adapter = Inquirex::LLM::OpenRouterAdapter.new   # key and model from the configuration
adapter = Inquirex::LLM::OpenRouterAdapter.new(model: "google/gemini-2.5-flash", effort: "low")

Any model OpenRouter hosts, with the OpenAI adapter's prompts and parsing. The Claude and GPT symbols map to OpenRouter ids (:claude_haiku → anthropic/claude-haiku-5.5); any other model is sent as written. effort: (none … max) becomes OpenRouter's reasoning.effort; without it the model decides.

adapter = Inquirex::LLM::JevAdapter.new                       # key and model from the configuration
adapter = Inquirex::LLM::JevAdapter.new(min_confidence: 0.6)  # less sure than 0.6 → nil
extraction = adapter.call(node, answers)                      # the values, as every adapter returns them
extraction.raw.dig("filing_status", "confidence")             # how sure Jev was, from TypeSafe's answer

Jev, TypeSafe's System One model, answers typed questions with calibrated probabilities instead of writing JSON. The adapter turns each schema field into questions and sends them all in one request: a Choice per enum (plus "not stated"), a yes / no / not-stated Choice per boolean, a presence Choice plus a Noul per value for multi_enum, and for integer a Choice among the numbers the text states (digits or words, Inquirex::LLM::Numbers) and the counts 0 to 12. It retries 429 and 529 twice with backoff.

Jev chooses; it cannot write. A schema with a string, text, date, email, phone or other free-form field raises AdapterError before anything is sent (JevAdapter.unsupported_fields(schema) checks a flow up front), and summarize always raises. Route those to a text model.

A common use case: ask one open-ended question, let the LLM extract answers for many downstream questions, and only prompt the user for what the LLM couldn't determine. This is what the core engine's Engine#prefill! is for:

definition = Inquirex.define id: "tax-intake" do
  start :describe

  ask :describe do
    type :text
    question "Describe your 2025 tax situation."
    transition to: :extracted
  end

  extract :extracted do
    from :describe
    prompt "Extract: filing_status, dependents, income_types, state_filing."
    schema filing_status: :string,
           dependents:    :integer,
           income_types:  :multi_enum,
           state_filing:  :string
    model :claude_sonnet
    transition to: :filing_status
  end

  ask :filing_status do
    type :enum
    question "Filing status?"
    options %w[single married_filing_jointly head_of_household]
    skip_if not_empty(:filing_status)     # ← the whole trick
    transition to: :dependents
  end

  ask :dependents do
    type :integer
    question "How many dependents?"
    skip_if not_empty(:dependents)
    transition to: :income_types
  end
  # …and so on for every field in the extract schema
end

engine  = Inquirex::Engine.new(definition)
adapter = Inquirex::LLM::OpenAIAdapter.new  # or AnthropicAdapter

engine.answer("I'm MFJ with two kids in California, W-2 plus some crypto.")
result = adapter.call(engine.current_step, engine.answers)
engine.answer(result)         # stored under :extracted
engine.prefill!(result)       # splats into top-level answers

# Every downstream step whose skip_if rule now evaluates true gets
# auto-skipped by the engine. engine.current_step_id jumps straight to
# whichever field the LLM couldn't fill in.

Engine#prefill! is non-destructive (won't clobber an answer the user already gave), ignores nil/empty values so they don't spuriously trigger not_empty, and auto-advances past any step whose skip_if now evaluates true. See examples/09_tax_preparer_llm.rb for a complete runnable flow, or the repo-level demo_llm_intake.rb for a scripted end-to-end walkthrough.

LLM steps serialize with "requires_server": true so the JS widget knows to round-trip to the server. LLM metadata lives under an "llm" key:

{
  "verb": "extract",
  "requires_server": true,
  "transitions": [{ "to": "next_step", "requires_server": true }],
  "llm": {
    "prompt": "Extract structured business information.",
    "schema": {
      "industry": "string",
      "employee_count": "integer",
      "revenue": "currency",
      "income_types": {
        "type": "multi_enum",
        "values": ["W2", "business", "crypto"]
      }
    },
    "from_steps": ["business_description"],
    "model": "claude_sonnet",
    "temperature": 0.2,
    "max_tokens": 1024
  }
}

Unconstrained fields serialize as a plain type string; fields resolved from :enum / :multi_enum questions serialize as { "type": ..., "values": [...] } so any consumer (the JS widget, a server adapter) sees the full contract. Fallback procs are stripped from JSON (server-side only).

Subclass Inquirex::LLM::Adapter and implement #call(node, answers):

class MyLlmAdapter < Inquirex::LLM::Adapter
  def call(node, answers)
    source = source_answers(node, answers)
    response = my_llm_client.complete(
      node.prompt,
      context: source,
      model: node.model,
      temperature: node.temperature
    )
    result = parse_response(response)
    validate_output!(node, result)
    result
  end
end

The base class provides #source_answers (gathers relevant answers) and #validate_output! (checks schema conformance).

Generate natural-language text from structured data. Requires from and prompt. No schema needed.

describe :business_narrative do
  from :business_extracted
  prompt "Write a brief narrative of this business for the intake report."
  transition to: :next_step
end

Dynamically generate follow-up questions based on an answer. The server adapter handles presenting the generated questions and collecting responses. Requires from, prompt, and schema.

detour :followup do
  from :description
  prompt "Generate 2-3 follow-up questions to clarify the tax situation."
  schema questions: :array, answers: :hash
  transition to: :next_step
end

Which model extracts best, measured with this gem's own prompts: ten flows, a complete and a vague text each, run on agent-evals. The lineup and the data live in evals/; the adapter and the scorer are Inquirex::LLM::Evals, loaded only by require "inquirex/llm/evals".

How a run works:

1. Verify (offline): each dataset's schema.rb must match the extract step in its dsl.rb, and every expected.json must fit that schema.
2. Expand the lineup: every model under vendors: in evals/.agent-eval.config.yml, times each effort/temperature it lists, becomes one contender. Each contender runs every case.
3. Call the model: one OpenRouter chat completion per case, built from this gem's production prompt and the step's schema.
4. Score the raw reply field by field: correct 1.0, missed 0.2, hallucinated 0.15, incorrect 0.1, invalid 0. A case's score is the mean over its fields. The rules are in evals/README.md.
5. Report: a leaderboard on screen and a JSON log under log/agent-evals/. If BRAINTRUST_API_KEY or LOGFIRE_TOKEN is set, each case is also sent to that one tracker; --local turns that off.

just evals -n                       # the whole matrix, calling nothing
just evals                          # every configured model on every dataset
just evals --local -d 05-managed-it-services   # one dataset, no remote tracker
just evals-verify                   # step 1 only

bundle exec agent-evals cases       # the 20 cases
bundle exec agent-evals run -m openai/gpt-4o-mini   # one model, per-case breakdown

just evals needs OPENROUTER_API_KEY and the :evals bundle group. It is a paid run: 14 contenders × 20 cases = 280 calls.

bundle install
bundle exec rspec
bundle exec rubocop

© 2026 Konstantin Gredeskoul.

Distributed under the MIT License.

See LICENSE.txt for details.