POML Ruby Gem

A Ruby implementation of the POML (Prompt Oriented Markup Language) interpreter.

This is a Ruby port of the original POML library developed by Microsoft, which was originally implemented in JavaScript/TypeScript and Python. This Ruby gem is designed to be fully compatible with the original POML specification and has been synchronized with version 0.0.9 of the original library to maintain complete feature parity.

✅ Full Implementation Complete: The Ruby implementation is now fully aligned with the original POML library structure, including correct tools positioning at the top level and proper handling of all component types. 100% of test suite passing (399 tests, 2812 assertions) with all functionality fully operational.

For comprehensive documentation, tutorials, and examples, please refer to the original POML library documentation:

• 📚 Main Repository: https://github.com/microsoft/poml
• 📖 Documentation: Complete language reference and guides
• 💡 Examples: Extensive collection of POML examples
• 🎯 Use Cases: Real-world applications and patterns

The original documentation is an excellent resource for learning POML concepts, syntax, and best practices that apply to this Ruby implementation as well.

gem install poml

Or add to your Gemfile:

gem 'poml'

After installation, check out the comprehensive tutorial for examples and integration patterns.

require 'poml'

# Basic usage
result = Poml.process(markup: '<poml><role>Assistant</role><task>Help users</task></poml>')
puts result['content']

# With template variables
markup = '<poml><role>{{role_name}}</role><task>{{task_name}}</task></poml>'
context = { 'role_name' => 'Data Analyst', 'task_name' => 'Analyze data' }
result = Poml.process(markup: markup, context: context)

# Different output formats
result = Poml.process(markup: markup, format: 'openai_chat')  # OpenAI chat format
result = Poml.process(markup: markup, format: 'raw')          # Raw text
result = Poml.process(markup: markup, format: 'dict')         # Dictionary (default)

# With custom stylesheet
stylesheet = { 'role' => { 'captionStyle' => 'bold' } }
result = Poml.process(markup: markup, stylesheet: stylesheet)

# Disable chat mode for single prompts
result = Poml.process(markup: markup, chat: false)

📖 For comprehensive examples and advanced usage patterns, see the POML Tutorial

# OpenAI API integration
messages = Poml.process(markup: prompt_template, context: vars, format: 'openai_chat')
response = openai_client.chat.completions.create(model: 'gpt-4', messages: messages)

# Rails service integration  
class PromptService
  def self.generate(template_name, context = {})
    markup = File.read("app/prompts/#{template_name}.poml")
    Poml.process(markup: markup, context: context, format: 'raw')
  end
end

# Error handling
begin
  result = Poml.process(markup: user_input)
rescue Poml::Error => e
  puts "POML Error: #{e.message}"
end

# Process a POML file
poml examples/101_explain_character.poml

# Process markup directly
poml "<poml><role>Test</role><task>Say hello</task></poml>" --format raw

# With context variables
poml markup.poml --context '{"name": "John", "role": "developer"}'

# Different output formats
poml markup.poml --format openai_chat
poml markup.poml --format raw

• -f, --format FORMAT: Output format (raw, dict, openai_chat, openaiResponse, langchain, pydantic)
  • raw: Plain text output with message boundaries (like ===== system =====)
  • dict: JSON object with content and metadata
  • openai_chat: Array of messages in OpenAI Chat Completion API format
  • openaiResponse: Standardized AI response structure with content, type, and metadata
  • langchain: Object with both messages array and raw content
  • pydantic: Enhanced Python interoperability with strict JSON schema support
• -c, --context JSON: Context variables as JSON
• --no-chat: Disable chat mode
• -s, --stylesheet JSON: Stylesheet as JSON
• -o, --output FILE: Output to file
• -h, --help: Show help
• -v, --version: Show version

Note: While this Ruby implementation aims for compatibility with the original POML library, the output format structures may differ slightly from the original TypeScript/Python implementations. The format names are kept consistent for API compatibility, but the Ruby gem provides its own implementation of each format suitable for Ruby applications.

During test-driven development, several critical insights were discovered about POML's dual-mode architecture:

Key Discovery: The original POML TypeScript implementation uses SimpleMarkupComponent with tagName='code' which preserves XML attributes when syntax="xml" is specified. This means:

• Tutorial formatting tests expect plain HTML: <code>content</code>
• Markup component tests expect XML with attributes: <code inline="true">content</code>
• Both behaviors are correct depending on context - the syntax="xml" mode should preserve XML structure with attributes

Critical Fix: The regex pattern /<code\b[^>]*>/ was incorrectly matching <code-block> tags, causing XML parsing failures. Fixed with negative lookahead: /<code(?!-)[^>]*>/

• Word boundaries (\b) don't prevent hyphenated extensions
• Negative lookahead ((?!-)) provides precise disambiguation
• This fix resolved multiple XML parsing test failures

The Ruby implementation now correctly handles:

1. Markdown Mode: Components render to Markdown syntax (`code`, **bold**, etc.)
2. XML Mode: Components preserve HTML/XML structure for syntax="xml" contexts
3. Attribute Preservation: XML mode maintains component attributes like inline="true"

The comprehensive test suite revealed the importance of:

• Boundary condition testing for regex patterns
• Context-aware rendering based on syntax attributes
• Dual compatibility between chat and XML syntax modes
• Progressive complexity in template and component interactions

Please refer to ROADMAP.md for understanding which features are already implemented.

The original POML library uses a sophisticated presentation system with multiple rendering modes:

• markup - Renders to markup languages (Markdown by default, configurable)
• serialize - Renders to serialized data formats (JSON, XML, etc.)
• free - Flexible rendering mode
• multimedia - Media-focused rendering

The Ruby implementation currently uses a simplified approach:

• Default rendering - Produces Markdown-like output for readability
• Output format components - Use <output format="html|json|xml|text|markdown"/> for specific formats
• XML mode - Components can render HTML when in XML context

Original Implementation:

• Uses generic <h> tags with level attributes
• Presentation mode determines output format (HTML vs Markdown)

Ruby Implementation:

• Supports both <h> and <h1>-<h6> tag syntax
• Defaults to Markdown output (# text) unless in XML mode
• Use <output format="html"/> for HTML output (<h1>text</h1>)

If migrating from the original TypeScript/JavaScript implementation:

1. Output Formats: Explicitly specify output format for HTML rendering
2. Presentation Context: Ruby gem uses output format components instead of presentation context
3. Component Mapping: Most components work identically, but output format may differ

Version 0.0.7 introduced a breaking change to align with the original POML library structure:

result = Poml.process(markup: markup)
tools = result['metadata']['tools']  # ❌ Wrong location

result = Poml.process(markup: markup)  
tools = result['tools']              # ✅ Correct location

Why this change?

• Aligns with original TypeScript/Node.js POML implementation
• Matches the CliResult interface structure: { messages, schema?, tools?, runtime? }
• Ensures full compatibility with original library behavior

Migration steps:

1. Update all code accessing result['metadata']['tools'] to use result['tools']
2. The metadata object still contains other data: chat, stylesheet, variables, response_schema
3. No other result structure changes were made

• <role>: Define the AI's role
• <task>: Define the task to perform
• <hint>: Provide hints or additional information
• <p>: Paragraph text

• <ai>: AI assistant messages
• <human>: Human user messages
• <system>: System prompts and instructions

• <Document src="file.txt">: Include external documents (.txt, .docx, .pdf)
• <img src="image.jpg" alt="description">: Include images
• <Table src="data.csv">: Include tabular data

• <cp caption="Section Title">: Captioned paragraph/section
• <list> with <item>: Create lists
• <example> with <input> and <output>: Define examples

Use {{variable}} syntax for template substitution:

<poml>
  <role>{{user_role}}</role>
  <task>Process {{count}} items</task>
</poml>

Enable XML output mode:

<poml syntax="xml">
  <role>Consultant</role>
  <cp caption="Requirements">
    <list>
      <item>Requirement 1</item>
      <item>Requirement 2</item>
    </list>
  </cp>
</poml>

Customize component appearance:

<poml>
  <stylesheet>
  {
    "role": {
      "captionStyle": "bold",
      "caption": "System Role"
    }
  }
  </stylesheet>
  <role>Assistant</role>
</poml>

• ✅ 291 tests with 1591 assertions - ALL PASSING
• 🎯 Complete test coverage with comprehensive, well-organized test suite including performance and format compatibility tests
• ✅ Template variable substitution with conditional logic and loops
• ✅ Multiple output formats (raw, dict, openai_chat, openaiResponse, langchain, pydantic)
• ✅ Document inclusion (.txt, .docx, .pdf) with robust encoding support
• ✅ Image handling with URL support (HTTP/HTTPS fetching, base64 encoding, local files)
• ✅ Inline rendering support - Components can render inline for seamless text flow
• ✅ Enhanced file operations - UTF-8 encoding with international file name support (Chinese, Arabic, etc.)
• ✅ Enhanced Pydantic integration - Python interoperability with strict JSON schema validation
• ✅ Table data processing from CSV, JSON, and JSONL sources
• ✅ XML syntax mode with full component compatibility
• ✅ Stylesheet support for component customization
• ✅ Command-line interface with comprehensive options
• ✅ Chat vs non-chat modes for different use cases
• ✅ Schema definitions - Full support for both legacy lang and new parser attribute syntax
• ✅ Tool registration - Enhanced tool definition capabilities with multiple formats
• ✅ Chat components - AI, Human, and System message components with nested formatting support
• ✅ Unknown component handling - Graceful error handling for unrecognized components
• ✅ Performance testing - Comprehensive performance benchmarks for large datasets and complex templates
• ✅ Format compatibility - Cross-format consistency validation and feature testing

The gem supports including external documents:

• Text files (.txt): Direct inclusion
• Word documents (.docx): Text extraction using system tools or zip parsing
• PDF files (.pdf): Text extraction using pdftotext (when available)

git clone https://github.com/GhennadiiMir/poml.git
cd poml
bundle install

Run tests:

# Run stable test suite (recommended)
bundle exec rake test           # 291 tests, 1591 assertions, all passing

# Run all tests including development tests  
bundle exec rake test_all       # Includes debug tests (for development)

# Run specific test files
bundle exec ruby -I lib test/test_basic_functionality.rb
bundle exec ruby -I lib test/test_template_engine.rb
bundle exec ruby -I lib test/test_performance.rb
bundle exec ruby -I lib test/test_format_compatibility.rb

Build and install locally:

gem build poml.gemspec
gem install poml-0.0.7.gem

The gem is available as open source under the terms of the MIT License.