Nitro Intelligence

The entrypoint to everything AI.

This component aims to consolidate and standardize AI features implemented in the host application.

Configuration

NitroIntelligence is configured via an initializer in the host application. All external dependencies (e.g. inference API credentials, observability settings) must be injected at boot time:

NitroIntelligence.configure do |config|
  # Standard Rails integrations
  config.logger         = Rails.logger           # Logger instance
  config.environment    = Rails.env              # e.g. "production", "test"
  config.cache_provider = Rails.cache            # ActiveSupport cache store

  # Inference (LLM) settings
  config.inference_api_key  = "..."              # API key for the inference service
  config.inference_base_url = "https://..."      # Base URL for the inference service

  # Observability (Langfuse) settings
  config.observability_base_url = "https://..."  # Base URL for the observability service
  config.observability_projects = [              # Array of project credential hashes
    {
      "slug"       => "my-feature-project",
      "id"         => "project-id",
      "public_key" => "pk-...",
      "secret_key" => "sk-...",
    },
  ]

  # Agent server settings (optional)
  config.agent_server_config = {}                # Hash of AgentServer keyword arguments

  # Model configuration
  config.model_config = {
    "default_audio_transcription_model" => "gpt-4o-transcribe",
    "default_text_model" => "gpt-4o-mini",
    "default_image_model" => "nano-banana-2",
    "default_text_to_speech_model" => "gpt-4o-mini-tts",
    "models" => [
      {
        "name" => "gpt-4o-mini",
        "type" => "text"
      },
      {
        "name" => "gpt-4o-transcribe",
        "type" => "audio_transcription"
      },
      {
        "name" => "nano-banana-2",
        "type" => "image",
        "aspect_ratios" => ["1:1", "2:3", "3:2", "3:4", "4:3"],
        "resolutions" => ["512", "1K", "2K"],
        "omit_output_fields" => ["provider_specific_fields.thought_signatures"]
      },
      {
        "name" => "gpt-4o-mini-tts",
        "type" => "text_to_speech",
        "default_voice" => "marin",
        "default_response_format" => "mp3",
        "voices" => ["echo", "nova", "marin", "cedar"],
        "response_formats" => ["mp3", "wav"]
      }
    ]
  }
end

Configuration Keys

Key	Type	Default	Description
`logger`	`Logger`	`Logger.new($stdout)`	Logger used for diagnostic output
`environment`	`String`	`"test"`	Runtime environment name
`cache_provider`	cache store	`NullCache`	ActiveSupport-compatible cache store
`inference_api_key`	`String`	`""`	API key for the LLM inference service
`inference_base_url`	`String`	`""`	Base URL for the LLM inference service
`observability_base_url`	`String`	`""`	Base URL for the Langfuse observability service
`observability_projects`	`Array<Hash>`	`[]`	Langfuse project credentials (slug, id, public_key, secret_key)
`agent_server_config`	`Hash`	`{}`	Credentials for `AgentServer.new`. Expected keys: `base_url` (String) — HTTP base URL of the agent server; `api_key` (String) — bearer token; `user_id` (String, default: `"default-user"`) — caller identity
`model_config`	`Hash`	`{}`	Model defaults and per-model settings. Top-level keys: `default_text_model`, `default_audio_transcription_model`, `default_image_model`, `default_text_to_speech_model`, and `models` (array of per-model hashes keyed by `name` and `type`, with type-specific options like `aspect_ratios`/`resolutions` for images or `voices`/`response_formats` for TTS)

Basic Usage

OpenAI API/LLM Requests

A simple LLM call can be invoked as follows:

client = NitroIntelligence::Client.new
client.chat(message: "Why is the sky blue?")
content = result.choices.first&.message&.content

This component handles setting defaults for most things, such as model, host, keys, etc.

You may also use openai-ruby compatible syntax with this wrapper by passing the parameters keyword in. For example:

client = NitroIntelligence::Client.new
client.chat(parameters: { model: "meta-llama/Llama-3.1-8B-Instruct", messages: [{ role: "user", content: "Why is the sky blue?" }]})

Providing Parameters

Parameters such as 'max_tokens' and 'temperature' can be passed in under the parameters key.

client = NitroIntelligence::Client.new
client.chat(parameters: { model: "meta-llama/Llama-3.1-8B-Instruct", max_tokens: 1000, temperature: 0.7, messages: [{ role: "user", content: "Why is the sky blue?" }]})

For a full list of supported parameters, see the API reference here.

Audio Transcription

Nitro Intelligence can be used to transcribe audio from a file into text.

Basic examples of usage:

client = NitroIntelligence::Client.new
audio = File.open('sample.m4a', 'rb')
result = client.transcribe_audio(audio_file: audio)

result is a OpenAI::Models::Audio::Transcription object:

<OpenAI::Models::Audio::Transcription:0x2fdc8 {:text=>"Hello, how are you doing today?", :usage=>{:input_tokens=>28, :output_tokens=>10, :total_tokens=>38, :type=>:tokens, :input_token_details=>{:audio_tokens=>28, :text_tokens=>0}}}>

To use a prompt, simply initialize your client with the observability_project_slug and pass prompt_name into parameters:

client = NitroIntelligence::Client.new(observability_project_slug: "my-test-project")
audio = File.open('sample.m4a', 'rb')
result = client.transcribe_audio(audio_file: audio, parameters: {prompt_name: "Spanish Converter"})

puts result

# <OpenAI::Models::Audio::Transcription:0x2fd8c {:text=>"Hola, ¿cómo estás hoy?", :usage=>{:input_tokens=>37, :output_tokens=>9, :total_tokens=>46, :type=>:tokens, :input_token_details=>{:audio_tokens=>28, :text_tokens=>9}}}>

Text-to-Speech

Nitro Intelligence can be used to create spoken-word audio files from text.

Basic example of usage:

client = NitroIntelligence::Client.new
tts = client.text_to_speech(message: 'Hello, this is Power Home Remodeling Group.')

# tts is a StringIO object, we can write the file to disk
File.binwrite('tts.mp3', tts.string)

Supplying custom parameters:

client = NitroIntelligence::Client.new
tts = client.text_to_speech(
  message: 'Hello, this is Power Home Remodeling Group.',
  parameters: {
    voice: 'marin',
    response_format: 'mp3',
    instructions: 'Translate the English into Spanish before speaking.',
    speed: 1.25
  }
)

# tts is a StringIO object, we can write the file to disk
File.binwrite('tts.mp3', tts.string)

For a full list of supported parameters, see the API reference here. Note that voice and response_format are further constrained to the voices and response_formats listed for the chosen model in config.model_config.

Image Editing and Generation

Nitro Intelligence can be used for image editing and generation

Basic examples of usage:

Image Generation

client = NitroIntelligence::Client.new
result = client.generate_image(message: "Create an image of a bear installing a window.")

result is a NitroIntelligence::ImageGeneration object, and the generated image can be accessed via result.generated_image which returns a NitroIntelligence::Image.

You may write the file to disk:

File.binwrite("my_generated_image.#{result.generated_image.file_extension}", result.generated_image.byte_string)

Image Editing and Uploading Reference Images

To edit an image, provide your source image as a byte string, along with any references you would like to include.

client = NitroIntelligence::Client.new
house = File.binread("./house.jpg")
siding = File.binread("./siding.png")
result = client.generate_image(message: "Replace the siding in the image of the house with the new siding I have provided.", target_image: house, reference_images: [siding])

Using Prompts

See Observability[## Observability] for more details. Basic usage looks like:

client = NitroIntelligence::Client.new(observability_project_slug: "sample-project-slug")
house = File.binread("./house.jpg")
siding = File.binread("./siding.png")
result = client.generate_image(target_image: house, reference_images: [siding], parameters: {prompt_name: "Sample Prompt Name"})

Image Configuration

You can specify parameters such as model to use, aspect ratio and resolution via the parameters key:

client = NitroIntelligence::Client.new
result = client.generate_image(message: "Create an image of a bear installing a window.", parameters: {aspect_ratio: "4:3", resolution: "512"})

Observability

Setup

If your feature is setup for observability, simply pass the observability project and desired prompt name in when invoking requests. It is also preferable to pass in a source and any other useful metadata for later debugging. For example:

client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
client.chat(
  message: "Why is the sky blue?",
  parameters: {
    prompt_name: "My Prompt",
    # prompt_label: "debug",
    # prompt_version: "v2",
    metadata: {
      source: self.class,
    },
  }
)

If no prompt_label or prompt_version is provided, the 'production' prompt is used by default.

Custom Trace Names

To provide custom trace names to the observability platform, you can pass 'trace_name' in parameters. Example:

client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
client.chat(
  message: "Why is the sky blue?",
  parameters: {
    prompt_name: "My Prompt",
    metadata: {
      source: self.class,
    },
    trace_name: "custom-trace-name"
  }
)

Custom Trace IDs

To generate a deterministic trace ID in the observability platform, you can pass trace_seed as a parameter. This is useful when you want a stable identifier that is derived from a specific domain value (e.g., document ID). The same trace_seed will produce the same trace ID, making it easier to correlate multiple events.

document_id = "document-123"
client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
client.chat(
  message: "summarize the document",
  parameters: {
    trace_seed: document_id,
  }
)

Scoring

You can use NitroIntelligence::Reporter to evaluate existing traces. Calling NitroIntelligence::Reporter#score lets you attach metrics to a trace in the observability platform.

By Trace ID

reporter = NitroIntelligence::Reporter.new(observability_project_slug: "fake-feature-project")
reporter.score(name: "precision", value: 0.5, trace_id: "2377b0b38acf0f11b95504344fad6152")

By Trace Seed

Knowing the trace seed, you can use NitroIntelligence::Trace.create_id to get the trace ID.

document_id = "document-123"
reporter = NitroIntelligence::Reporter.new(observability_project_slug: "fake-feature-project")
trace_id = NitroIntelligence::Trace.create_id(seed: document_id)
reporter.score(name: "precision", value: 0.5, trace_id:)

Prompt Variables and Config

Prompts are often created with "variables". These variables can be supplied and compiled into the prompt. For example:

client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
client.chat(
  message: "Where is the appointment?",
  parameters: {
    prompt_name: "My Prompt With Variables",
    prompt_variables: {
      appointment_id: "1234",
    },
    metadata: {
      source: self.class,
    },
  }
)

Prompts can be created with a "config" JSON object. This config stores structured data such as model parameters (like model name, temperature), function/tool parameters, or JSON schemas.

The prompt config is merged into your chat request by default. The prompt config object is merged into the "parameters" hash, overriding any existing keys.

Consider this prompt config:

{
  "model": "gpt-4o-mini"
}

Invoking this request would result in "gpt-4o-mini" being used as the model, even if supplied manually:

client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
client.chat(
  message: "Where is the appointment?",
  parameters: {
    model: "meta-llama/Llama-3.1-8B-Instruct", # Will not be used, will be overridden by config "gpt-4o-mini"
    prompt_name: "My Prompt With Variables",
    prompt_variables: {
      appointment_id: "1234",
    },
    metadata: {
      source: self.class,
    },
  }
)

To disable the prompt config entirely, which may be useful for debugging/testing, you can supply the "prompt_config_disabled" keyword. For example:

client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
client.chat(
  message: "Where is the appointment?",
  parameters: {
    model: "meta-llama/Llama-3.1-8B-Instruct", # This will now be used since "prompt_config_disabled" is true
    prompt_name: "My Prompt With Variables",
    prompt_variables: {
      appointment_id: "1234",
    },
    prompt_config_disabled: true,
    metadata: {
      source: self.class,
    },
  }
)

Agent Server

The Agent Server is Nitro Intelligence's lightweight SDK for working with hosted agent threads, runs, and human review flows. It is mainly used to initialize conversation threads, trigger agent runs, inspect agent tool calls pending human approval, and resume interrupted threads after human reviews.

For the full Agent Server guide, see AGENT_SERVER.md.