Pedicab

A lightweight Ruby framework for conversational AI interfaces using local LLM models. Pedicab provides a clean, intuitive API for building AI-powered applications with context management and structured responses.

Pedicab is designed to be "the fastest LLMs in the west" - a streamlined interface for local language model inference. It focuses on the essential mechanics of conversational AI while maintaining simplicity and performance.

• Local LLM Integration: Uses GGUF model files with Python llama-cpp backend
• Context Management: Automatic conversation context tracking and management
• Elegant Syntax: Intuitive [] and << operators for different interaction patterns
• State Management: Persistent conversation state with reset capabilities
• Performance Tracking: Built-in benchmarking and timing information
• Conditional Logic: Support for boolean conditions and flow control

1. Ruby >= 2.6.0
2. Python with llama-cpp-python package
3. Local LLM Models in GGUF format

1. Install the gem:

gem install pedicab

2. Install Python dependencies:

pip install llama-cpp-python

3. Download LLM models to /models/ directory:

# Example: Download a model
wget https://example.com/model.gguf -O /models/qwen.gguf

4. Set up the Python backend (included with the gem)

require 'pry'
require "bundler/setup"
# 1. include the Pedicab module
require 'pedicab'

# 2. create local Pedicab
@me = Pedicab["me"]

if File.exist? 'me.txt'
  # 3. use the me.txt file to predefine context.
  @me[File.read('me.txt')]
end

# 4. start interactive session
Pry.start
# ... that's it.

require 'pedicab'
puts Pedicab << "What is the meaning of life?"

require 'pedicab'

# Create a new conversation instance
ai = Pedicab['my_assistant']

# Start a fresh conversation
response = ai["Hello, how are you?"]
puts response.out
# => "I'm doing well, thank you for asking!"

# Continue the conversation with context
response = ai["Tell me about Ruby programming"]
puts response.out
# => "Ruby is a dynamic, object-oriented programming language..."

# Reset conversation context
ai.reset!

# The [] operator starts fresh
ai["What's the capital of France?"]  # New conversation

# The << operator continues with context
ai["What about Germany?"]           # Knows we're discussing countries

# Define custom response handling
ai.handle do |response|
  puts "AI Response: #{response.out}"
  puts "Processing time: #{response.took}s"
  response
end

ai["Explain quantum computing"]
# => AI Response: Quantum computing is a revolutionary...
# => Processing time: 1.23s

The main interface for AI conversations.

• initialize(id) - Create new conversation instance
• reset! - Clear conversation context and state
• [](prompt) - Start fresh conversation with prompt
• <<(prompt) - Continue conversation with context
• handle(&block) - Set custom response handler
• life() - Total processing time for all requests

• took - Time taken for last request
• prompt - Last prompt sent
• response - Array of all responses
• out - Last response content
• last - Array of previous prompts
• thoughts - LLM thinking process (if available)
• context - Current conversation context
• handler - Response handler lambda

Low-level LLM interface for direct model interaction.

• go(prompt, &block) - Send prompt to model, optionally process each line
• if?(condition, &block) - Evaluate boolean condition, optionally execute block
• reset! - Clear ride state
• [](key) - Access state value
• []=(key, value) - Set state value
• to_h - Get state as hash

The ride maintains internal state including:

• :content - Model response content
• :thought - Model thinking process
• :action - Current action description
• :yes - Boolean result for conditions
• :took - Processing time

• Pedicab.[](id) - Create new conversation instance
• Pedicab.models - List available models in /models/
• Pedicab.ride(id) - Create new ride instance

ai = Pedicab['conditional_bot']

# Use conditional logic
if ai.ride.if?("the user wants programming help")
  ai["I'll help you with programming!"]
else
  ai["How can I assist you today?"]
end

# Process each line of a response
ai.ride.go("List 3 programming languages") do |line|
  puts "Language: #{line[:for]}"
end

ride = Pedicab.ride['stateful_bot']

# Store custom state
ride[:user_preference] = "detailed"
ride[:last_topic] = "Ruby"

# Access state later
puts ride[:user_preference]  # => "detailed"
puts ride.to_h              # => { user_preference: "detailed", last_topic: "Ruby" }

ai = Pedicab['monitored_bot']

ai["First question"]     # Processes
puts ai.took            # Time for last request: 0.85s

ai["Follow up question"] # Processes
puts ai.life            # Total time: 1.67s

• MODEL - Default model name (default: 'qwen')
• DEBUG - Debug level (0=off, 1=basic, 2=verbose)

# List available models
puts Pedicab.models
# => ["qwen", "llama2", "mistral"]

# Models should be placed in /models/ directory as .gguf files
# /models/qwen.gguf
# /models/llama2.gguf
# /models/mistral.gguf

1. Ruby Interface (lib/pedicab.rb)

   • Main API and conversation management
   • Context tracking and state management
   • Elegant operator interface

2. Ride Layer (lib/pedicab/ride.rb)

   • Low-level LLM communication
   • JSON-based protocol with Python backend
   • State management and error handling

3. Python Backend (lib/pedicab.py)

   • llama-cpp-python integration
   • Model loading and inference
   • Grammar-based constrained outputs

Ruby P Class -> Ride Class -> Python Backend -> LLM Model -> Response

1. Ruby sends JSON request to Python process
2. Python processes with llama-cpp-python
3. Response includes content and thinking process
4. Ruby parses and manages state

begin
  ai = Pedicab['test_bot']
  response = ai["Some question"]
rescue Pedicab::Error => e
  puts "Pedicab error: #{e.message}"
rescue => e
  puts "Unexpected error: #{e.message}"
end

Enable debug output:

# In your shell
export DEBUG=1  # Basic debugging
export DEBUG=2  # Verbose debugging

# Or in Ruby
ENV['DEBUG'] = '1'

Debug output includes:

• Action descriptions
• Processing times
• Full state information
• Error details

1. Model Selection: Choose appropriate model size for your use case
2. Context Management: Reset conversations when starting new topics
3. Batch Processing: Use the iteration methods for list responses
4. Monitoring: Track processing times with took and life methods

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

MIT License - see LICENSE file for details.

• Simplified core architecture
• Removed auxiliary modules for focus on essential mechanics
• Improved performance and reduced dependencies
• Enhanced error handling and debugging capabilities

See git history for detailed changelog.

For issues and questions:

• GitHub Issues: https://github.com/xorgnak/pedicab/issues
• Documentation: https://github.com/xorgnak/pedicab

Pedicab: A better way to go with local LLMs