RTOON - Ruby Token Object Oriented Notation Parser

A Ruby gem for parsing and encoding Token Object Oriented Notation (TOON) — a human-readable data format combining tabular structure with JSON-like flexibility.'

Inspired by toon-format/toon which is written in TypeScript and for TypeScript projects.

TOON is a data serialization format with schema declarations, tabular rows, and indentation-based nesting.

From the original project description:

Token-Oriented Object Notation is a compact, human-readable serialization format designed for passing structured data to Large Language Models with significantly reduced token usage. It's intended for LLM input as a lossless, drop-in representation of JSON data.

TOON's sweet spot is uniform arrays of objects – multiple fields per row, same structure across items. It borrows YAML's indentation-based structure for nested objects and CSV's tabular format for uniform data rows, then optimizes both for token efficiency in LLM contexts. For deeply nested or non-uniform data, JSON may be more efficient.

TOON achieves CSV-like compactness while adding explicit structure that helps LLMs parse and validate data reliably.

Example:

users[2]{id,name}:
  1,Ada
  2,Bob

Parses to:

{"users" => [{"id" => "1", "name" => "Ada"}, {"id" => "2", "name" => "Bob"}]}

gem install rtoon

Or add to Gemfile:

gem 'rtoon'

require 'rtoon'

# Parse TOON → Ruby
result = Rtoon.parse("users[1]{id,name}:\n  1,Ada\n")
# => {"users" => [{"id" => "1", "name" => "Ada"}]}

# Encode Ruby → TOON
data = {"users" => [{"id" => "1", "name" => "Ada"}]}
toon = Rtoon.encode(data)
# => "users[1]{id,name}:\n  1,Ada\n"

Schema Declaration:

identifier[size]{field1,field2}:
  value1,value2
  value3,value4

Field Assignment:

name: John
age: 30

Nested Structure:

company[1]{depts,location}:
  depts[2]{name,size}:
    engineering,50
    sales,20
  location: NYC

Parses TOON-formatted string to Ruby Hash/Array.

Converts Ruby data structure to TOON format.

begin
  result = Rtoon.parse(invalid_toon)
rescue Rtoon::Parser::ParseError => e
  puts "Parse error: #{e.message}"
end

# Clone and setup
git clone https://github.com/zoobean/rtoon.git
cd rtoon
bundle install

# Run tests
ruby test/toon_test.rb

# Compile grammar (if modified)
racc -o lib/rtoon/parser.tab.rb lib/rtoon/parser.y

• Values returned as strings (no automatic type conversion)
• No string escaping (commas in values not supported)
• Fixed 2-space indentation
• No comment syntax

1. Fork the repository
2. Create a feature branch
3. Add tests for new features
4. Submit a Pull Request

See ARCHITECTURE.md for implementation details.

MIT License - See LICENSE.txt

Antonio Chavez - Zoobean

Report bugs: GitHub Issues