llm_meta_client

A Rails Engine for integrating multiple LLM providers into your application. Provides scaffold and authentication generators to quickly build LLM-powered chat applications with support for OpenAI, Anthropic, Google, and Ollama.

This gem is a Rails frontend client that delegates all LLM provider interactions to an external backend service. The gem itself does not implement any LLM provider SDKs directly — instead, it communicates with the external service via REST API to manage API keys, models, and chat completions.

┌─────────────────────┐       REST API         ┌─────────────────────┐
│  Your Rails App     │ ───────────────────>   │  External LLM       │
│  + llm_meta_client  │ <───────────────────   │  Service Backend    │
└─────────────────────┘                        └─────────────────────┘
                                                  │
                                                  ├── OpenAI API
                                                  ├── Anthropic API
                                                  ├── Google API
                                                  └── Ollama (local)

• Scaffold generator — Generates models, controllers, views, and JavaScript controllers for a full chat UI with Turbo Stream support
• Authentication generator — Sets up Devise with Google OAuth2 for user authentication
• Core modules — ServerQuery, ServerResource, Helpers, ChatManageable, HistoryManageable, and custom exception classes
• Hotwire integration — Real-time UI updates via Turbo Stream with Stimulus controllers
• Guest user support — Ollama can be used without authentication; Google Sign-In is optional
• Conversation branching — Branch from a previous message to explore alternative conversation paths
• CSV export — Download chat history as CSV (single chat or all chats)
• Auto-generated titles — Chat titles are automatically generated by the LLM from the conversation content
• Context summarization — Conversation history is automatically summarized to keep context concise

• Ruby >= 3.4
• Rails >= 8.1.1
• An external LLM service backend (see External LLM Service)

Add this line to your application's Gemfile:

gem "llm_meta_client"

And then execute:

$ bundle install

$ rails generate llm_meta_client:scaffold

This creates:

• Models: Chat, Message
• Controllers: ChatsController, PromptsController
• Views: Chat views (new, edit, Turbo Stream responses), message partials, shared form fields (_family_field, _api_key_field, _model_field), layout with header and sidebar
• JavaScript: Stimulus controllers (llm_selector, chats_form, chat_title_edit), popover
• Initializer: config/initializers/llm_service.rb
• Migrations: create_chats, create_messages
• Routes: Chats resource with clear, start_new, download_all_csv, download_csv, update_title actions

If you want user authentication with Google OAuth2:

$ rails generate llm_meta_client:authentication

This creates:

• Model: User
• Controllers: Users::OmniauthCallbacksController, Users::SessionsController
• Initializers: config/initializers/devise.rb, config/initializers/omniauth.rb
• Locale: config/locales/devise.en.yml
• Migration: create_users
• Routes: Devise routes with OmniAuth callbacks

This generator also adds the following gems to your Gemfile:

• devise
• omniauth
• omniauth-google-oauth2
• omniauth-rails_csrf_protection

Run bundle install again after the authentication generator.

$ rails db:migrate

This gem uses Rails credentials (rails credentials:edit) instead of environment variables for configuration management.

$ EDITOR="vim" bin/rails credentials:edit

Add the following entries to your credentials file:

# Required
llm_service:
  base_url: "http://localhost:3000"  # URL of your external LLM service

  # Optional
  summarize_conversation_count: 10  # Number of recent messages for context (default: 10)

# Required only if using the authentication generator
google:
  client_id: "your-google-client-id"
  client_secret: "your-google-client-secret"

Ensure your external LLM service is running, then:

$ rails server -p 3001

All configuration values are managed via Rails credentials (rails credentials:edit).

These values are referenced in the generated initializers config/initializers/llm_service.rb and config/initializers/devise.rb.

This gem requires an external LLM service backend that provides the following REST API endpoints:

GET /api/llms

{
  "llms": [
    {
      "uuid": "...",
      "description": "Ollama Local",
      "family": "ollama",
      "llm_type": "ollama",
      "available_models": ["llama3", "mistral"]
    }
  ]
}

GET /api/llm_api_keys

{
  "llm_api_keys": [
    {
      "uuid": "...",
      "description": "My OpenAI Key",
      "llm_type": "openai",
      "available_models": ["gpt-4", "gpt-3.5-turbo"]
    }
  ]
}

POST /api/llm_api_keys/:uuid/models/:model_id/chats

Request body:

{
  "prompt": "Context:..., User Prompt: ..."
}

Response body:

{
  "response": {
    "message": "The assistant's response text"
  }
}

Sends chat prompts to the external LLM service and returns responses. Used internally by the generated Chat model.

query = LlmMetaClient::ServerQuery.new
response = query.call(jwt_token, api_key_uuid, model_id, context, user_content)
# => "The assistant's response text"

• HTTP timeout: 5 minutes (300 seconds)
• Responses are synchronous (no streaming)

Fetches available LLM providers and models from the external service.

# Get a flat list of available LLM options
options = LlmMetaClient::ServerResource.available_llm_options(jwt_token)
# => [{uuid:, description:, llm_type:, available_models:}, ...]

# Get options grouped by provider family
families = LlmMetaClient::ServerResource.available_llm_families(jwt_token)
# => [{name: "OpenAI", llm_type: "openai", api_keys: [...]}, ...]

• Guest users (jwt_token is nil or blank) receive only Ollama options
• Logged-in users receive their API keys plus Ollama (if available)

Each time a user sends a message, the following happens:

1. A PromptExecution record and a user Message are created
2. If conversation history exists, it is summarized by the LLM to keep context concise
3. The summarized context + user prompt are sent to the external LLM service
4. The assistant response is saved as a new Message
5. A chat title is auto-generated by the LLM (if not already set)
6. Turbo Stream updates the UI in real-time

This means each user message may trigger up to 3 HTTP requests to the external LLM service (context summarization, main response, title generation), each with a 5-minute timeout.

Note: Streaming is not currently supported. All LLM responses are synchronous.

1. Fork the repository
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a Pull Request

The gem is available as open source under the terms of the MIT License.