LutaML Ruby modeller
Purpose
Lutaml::Model is the Ruby implementation of the LutaML modeling methodology, for:
-
creating information models in the LutaML language (or its Ruby DSL)
-
serializing and deserializing LutaML information models
-
accessing data instances of LutaML information models
-
documenting LutaML information models
It provides simple, flexible and comprehensive mechanisms for defining information models with attributes and types, and the serialization of them to/from serialization formats including Hash, JSON, XML, YAML, and TOML.
For serialization formats, it uses an adapter pattern to support multiple libraries for each format, providing flexibility and extensibility for your data modeling needs.
|
Note
|
The Lutaml::Model modeling Ruby DSL was originally designed to be mostly compatible with the data modeling DSL of Shale, a data modeller for Ruby. Lutaml::Model is meant to address advanced needs not currently addressed by Shale. Instructions on how to migrate from Shale to Lutaml::Model are provided in Migration steps from Shale. |
Features
-
Define models with attributes and types
-
Serialize and deserialize models to/from Hash, JSON, XML, YAML, and TOML
-
Support for multiple serialization libraries (e.g.,
toml-rb,tomlib) -
Configurable adapters for different serialization formats
-
Support for collections and default values
-
Custom serialization/deserialization methods
-
XML namespaces and mappings
-
Deep object comparison and visual diff generation with similarity scoring
-
Generate serialization schemas from model definitions (Schema generation)
-
Import serialization schemas to define models (Schema import)
-
Create custom adapters for additional data formats (see Custom serialization adapters)
-
Dynamically modify model attribute types using registers (see Custom Registers)
Data modeling in a nutshell
Data modeling is the process of creating a data model for the data to be stored in a database or used in an application. It helps in defining the structure, relationships, and constraints of the data, making it easier to manage and use.
Lutaml::Model simplifies data modeling in Ruby by allowing you to define models with attributes and serialize/deserialize them to/from various serialization formats seamlessly.
The Lutaml::Model data modelling approach is as follows:
LutaML Model
│
Has many attributes
│
▼
Attribute
│
Has type of
│
┌──────────┴──────────┐
│ │
Model Value (Leaf)
│ │
Has many attributes Contains one basic value
│ │
┌───────┴─────┐ ┌──────┴──────┐
│ │ │ │
Model Value (Leaf) String Integer
│ Date Boolean
│ Time Float
Has many attributes ... ...
│
▼
(Recursive pattern continues...)Studio (Model)
├── name (Value: String) = "Pottery Studio"
├── address (Model)
│ ├── street (Value: String) = "123 Clay St"
│ ├── city (Value: String) = "Ceramics City"
│ └── postcode (Value: String) = "12345"
├── established (Value: Date) = 2020-01-01
└── kilns (Model)
├── count (Value: Integer) = 3
└── temperature (Value: Float) = 1200.0╔═══════════════════════╗ ╔════════════════════════════╗
║ LutaML Core Model ║ ║ Serialization Models ║
╚═══════════════════════╝ ╚════════════════════════════╝
╭┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╮ ╭┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╮
┆ Model ┆ ┆ XML Model ┆
┆ │ ┆ ┌────────────────┐ ┆ │ ┆
┆ ┌────────┴──┐ ┆ │ │ ┆ ┌──────┴──────┐ ┆
┆ │ │ ┆ │ Model │ ┆ │ │ ┆
┆ Models Value Types ┆──►│ Transformation │ ┆ Models Value Types ┆
┆ │ │ ┆ │ & │ ┆ │ │ ┆
┆ │ │ ┆ │ Mapping Rules │ ┆ │ │ ┆
┆ │ ┌──────┴──┐ ┆ │ │ ┆ ┌────┴────┐ ┌─┴─┐ ┆
┆ │ │ │ ┆ └────────────────┘ ┆ │ │ │ │ ┆
┆ │ String Integer ┆ │ ┆ Element Value xs:string ┆
┆ │ Date Float ┆ │ ┆ Attribute Type xs:date ┆
┆ │ Time Boolean ┆ ├──────────►┆ xs:boolean ┆
┆ │ ┆ │ ┆ xs:anyURI ┆
┆ └──────┐ ┆ │ ╰┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╯
┆ │ ┆ │
┆ Contains ┆ │ ╭┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╮
┆ more Models ┆ │ ┆ JSON Model ┆
┆ (recursive) ┆ │ ┆ │ ┆
┆ ┆ │ ┆ ┌──────┴──────┐ ┆
╰┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╯ └──────────►┆ │ │ ┆
┆ Models Value Types ┆
┆ │ │ ┆
┆ │ │ ┆
┆ ┌────┴───┐ ┌───┴──┐ ┆
┆ │ │ │ │ ┆
┆ object array number string ┆
┆ value boolean null ┆
╰┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╯╔═══════════════════════╗ ╔══════════════════╗ ╔═══════════════════════╗
║LutaML Model Class FOO ║ ║LutaML Transformer║ ║LutaML Model Class BAR ║
╚═══════════════════════╝ ╚══════════════════╝ ╚═══════════════════════╝
╭┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╮ ╭┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╮
┆ Model ┆ ┆ Model ┆
┆ │ ┆ ┌────────────────┐ ┆ │ ┆
┆ ┌────────┴──┐ ┆ │ │ ┆ ┌────────┴──┐ ┆
┆ │ │ ┆ │ Model │ ┆ │ │ ┆
┆ Models Value Types ┆───►│ Transformation │───►┆ Models Value Types ┆
┆ │ │ ┆◄───│ & │◄───┆ │ │ ┆
┆ │ │ ┆ │ Mapping Rules │ ┆ │ │ ┆
┆ │ ┌──────┴──┐ ┆ │ │ ┆ │ ┌──────┴──┐ ┆
┆ │ │ │ ┆ └────────────────┘ ┆ │ │ │ ┆
┆ │ String Integer ┆ ┆ │ String Integer ┆
┆ │ Date Float ┆ ┆ │ Date Float ┆
┆ │ Time Boolean ┆ ┆ │ Time Boolean ┆
┆ │ ┆ ┆ │ ┆
┆ └──────┐ ┆ ┆ └──────┐ ┆
┆ │ ┆ ┆ │ ┆
┆ Contains ┆ ┆ Contains ┆
┆ more Models ┆ ┆ more Models ┆
┆ (recursive) ┆ ┆ (recursive) ┆
┆ ┆ ┆ ┆
╰┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╯ ╰┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╯Value class, transformation, and serialization formats╔═══════════════════════╗ ╔═══════════════════════╗
║LutaML Value Class FOO ║ ║ Serialization Value ║
╚═══════════════════════╝ ╚═══════════════════════╝
╭┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╮ ╭┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╮
┆ ┌───────────────┐ ┆ ┆ ┌───────────────┐ ┆
┆ │ Value │ ┆ ┌──────────────────┐ ┆ │ XML Value │ ┆
┆ └───────────────┘ ┆──►│ Value Serializer │──►┆ └───────────────┘ ┆
┆ ┌───────────────┐ ┆ └──────────────────┘ ┆ ┌───────────────┐ ┆
┆ │Primitive Types│ ┆ ┆ │XML Value Types│ ┆
┆ └───────────────┘ ┆ ┆ └───────────────┘ ┆
┆ ┌───┘ ┆ ┆ ┌───┘ ┆
┆ ├─ string ┆ ┆ ├─ xs:string ┆
┆ ├─ integer ┆ ┆ ├─ xs:integer ┆
┆ ├─ float ┆ ┆ ├─ xs:decimal ┆
┆ ├─ boolean ┆ ┆ ├─ xs:boolean ┆
┆ ├─ date ┆ ┆ ├─ xs:date ┆
┆ ├─ time_without_date ┆ ┆ ├─ xs:time ┆
┆ ├─ date_time ┆ ┆ ├─ xs:dateTime ┆
┆ ├─ time ┆ ┆ ├─ xs:decimal ┆
┆ ├─ decimal ┆ ┆ ├─ xs:anyType ┆
┆ └─ hash ┆ ┆ └─ (complex element) ┆
╰┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╯ ╰┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╯
│
▼
┌───────────────────┐
│ Value Transformer │
└───────────────────┘
│
▼
╔═══════════════════════╗
║LutaML Value Class BAR ║
╚═══════════════════════╝
╭┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╮
┆ ┌───────────────┐ ┆
┆ │ Value │ ┆
┆ └───────────────┘ ┆
┆ ┌───────────────┐ ┆
┆ │Primitive Types│ ┆
┆ └───────────────┘ ┆
┆ ┌───┘ ┆
┆ ├─ string ┆
┆ ├─ integer ┆
┆ ├─ float ┆
┆ ├─ boolean ┆
┆ ├─ date ┆
┆ ├─ time_without_date ┆
┆ ├─ date_time ┆
┆ ├─ time ┆
┆ ├─ decimal ┆
┆ └─ hash ┆
╰┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╯╔═════════════════════╗ ╔═════════════════════╗ ╔═════════════════════╗
║ Studio (Core Model) ║ ║ JSON Model ║ ║ Serialized JSON ║
╚═════════════════════╝ ╚═════════════════════╝ ╚═════════════════════╝
name: "Studio 1" ┌─► { ┌─► {
address: │ "name": "...", │ "name": "Studio 1",
├── street: "..." │ "address": { │ "address": {
└── city: "..." │ "street": "...", │ "street": "...",
kilns: ──┤ "city": "..." ──┤ "city": "..."
├── count: 3 │ }, │ },
└── temp: 1200 │ "kilnsCount": ..., │ "kilnsCount": 3,
│ "kilnsTemp": ... │ "kilnsTemp": 1200
└─► } └─► }Installation
Add this line to your application’s Gemfile:
gem 'lutaml-model'And then execute:
bundle installOr install it yourself as:
gem install lutaml-modelComponents
LutaML provides the following set of components to model information in a structured way.
-
Basic models
-
Attributes in models
-
Values assigned to attributes
-
Collections of models
Model
General
A LutaML model is used to represent a class of information, of which a model instance is a set of information representing a coherent concept.
There are two ways to define an information model in Lutaml::Model:
-
Inheriting from the
Lutaml::Model::Serializableclass -
Including the
Lutaml::Model::Serializemodule
Definition
Through inheritance
The simplest way to define a model is to create a class that inherits from
Lutaml::Model::Serializable.
The attribute class method is used to define attributes.
require 'lutaml/model'
class Kiln < Lutaml::Model::Serializable
attribute :brand, :string
attribute :capacity, :integer
attribute :temperature, :integer
endThrough inclusion
If the model class already has a super class that it inherits from, the model
can be extended using the Lutaml::Model::Serialize module.
require 'lutaml/model'
class Kiln < SomeSuperClass
include Lutaml::Model::Serialize
attribute :brand, :string
attribute :capacity, :integer
attribute :temperature, :integer
endInheritance
A model can inherit from another model to inherit all attributes and methods of the parent model, allowing for code reusability and a clear model hierarchy.
Syntax:
class Superclass < Lutaml::Model::Serializable
# attribute ...
# serialization blocks
end
class Subclass < Superclass
# attributes are additive
# serialization blocks are replaced
endAn inherited model has the following characteristics:
-
All attributes are inherited from the parent model.
-
Additional calls to
attributein the child model are additive, unless the attribute name is the same as an attribute in the parent model. -
Serialization blocks, such as
xmlandkey_valueare replaced when defined.-
In order to selectively import serialization mapping rules from the parent model, the
import_model_mappingsmethod can be used (see [import_model_mappings]).
-
Comparison and Diff
A Serialize / Serializable object can be compared with another object of the
same class using the == operator. This is implemented through the
ComparableModel module, which provides powerful comparison and diff
functionality.
Basic Comparison
Two objects are considered equal if they have the same class and all their attributes are equal. This behavior differs from the typical Ruby behavior, where two objects are considered equal only if they have the same object ID.
|
Note
|
Two Serialize objects will have the same hash value if they have the
same class and all their attributes are equal.
|
> a = Kiln.new(brand: 'Kiln 1', capacity: 100, temperature: 1050)
> b = Kiln.new(brand: 'Kiln 1', capacity: 100, temperature: 1050)
> a == b
> # true
> a.hash == b.hash
> # trueDeep Comparison
The comparison works recursively for nested objects and handles complex data structures:
class Glaze < Lutaml::Model::Serializable
attribute :color, :string
attribute :temperature, :integer
attribute :food_safe, :boolean
end
class Ceramic < Lutaml::Model::Serializable
attribute :type, :string
attribute :glaze, Glaze
end
# Nested object comparison
glaze1 = Glaze.new(color: "Blue", temperature: 1200, food_safe: true)
glaze2 = Glaze.new(color: "Blue", temperature: 1200, food_safe: true)
ceramic1 = Ceramic.new(type: "Bowl", glaze: glaze1)
ceramic2 = Ceramic.new(type: "Bowl", glaze: glaze2)
ceramic1 == ceramic2 # true - deep comparison of nested objectsCircular Reference Handling
The comparison safely handles circular references without infinite loops:
class RecursiveNode < Lutaml::Model::Serializable
attribute :name, :string
attribute :next_node, RecursiveNode
end
node1 = RecursiveNode.new(name: "A")
node2 = RecursiveNode.new(name: "B", next_node: node1)
node1.next_node = node2 # Creates circular reference
# Comparison still works without infinite loops
node1_copy = RecursiveNode.new(name: "A")
node2_copy = RecursiveNode.new(name: "B", next_node: node1_copy)
node1_copy.next_node = node2_copy
node1 == node1_copy # trueDiff Generation
The ComparableModel module provides powerful diff functionality to visualize
differences between objects and calculate similarity scores.
Understanding Diff and Score
Diff (Difference): A visual representation showing the differences between two objects:
- Removed values are marked with - (typically red)
- Added values are marked with + (typically green)
- Hierarchical structure shows nested objects and their relationships
Score: A numerical value between 0 and 1 representing how different the objects are:
- 0 = Objects are identical (no differences)
- 1 = Objects are completely different
- 0.5 = Objects are 50% different
- Convert to similarity percentage: (1 - score) * 100
The diff_with_score Method
The diff_with_score method returns two values as an array:
diff_score, diff_tree = Lutaml::Model::Serialize.diff_with_score(obj1, obj2, options)
# ↑ ↑
# Float String
# (0.0-1.0) (Visual representation)-
diff_score(Float): The numerical difference score between 0.0 and 1.0 -
diff_tree(String): The formatted visual diff showing all differences
Basic Example
# Create two different objects
ceramic1 = Ceramic.new(
type: "Bowl",
glaze: Glaze.new(color: "Blue", temperature: 1200, food_safe: true)
)
ceramic2 = Ceramic.new(
type: "Bowl",
glaze: Glaze.new(color: "Red", temperature: 1000, food_safe: false)
)
# Generate diff with similarity score
diff_score, diff_tree = Lutaml::Model::Serialize.diff_with_score(ceramic1, ceramic2)
puts "Difference Score: #{diff_score}"
puts "Similarity: #{((1 - diff_score) * 100).round(2)}%"
puts "Visual Diff:"
puts diff_treeOutput:
Difference Score: 0.25
Similarity: 75.0%
Visual Diff:
└── Ceramic
└── glaze (Glaze):
├── color (Lutaml::Model::Type::String):
│ ├── - (String) "Blue"
│ └── + (String) "Red"
├── temperature (Lutaml::Model::Type::Integer):
│ ├── - (Integer) 1200
│ └── + (Integer) 1000
└── food_safe (Lutaml::Model::Type::Boolean):
├── - (TrueClass) true
└── + (FalseClass) falseUnderstanding the Return Values Structure
The method returns an array with exactly two elements:
result = Lutaml::Model::Serialize.diff_with_score(obj1, obj2)
# result[0] => diff_score (Float between 0.0 and 1.0)
# result[1] => diff_tree (String with visual representation)
# Or using array destructuring:
score, tree = Lutaml::Model::Serialize.diff_with_score(obj1, obj2)
puts "Objects are #{(score * 100).round(1)}% different"
puts treeDiff Options
The diff functionality supports various options:
# Show unchanged attributes as well
diff_score, diff_tree = Lutaml::Model::Serialize.diff_with_score(
ceramic1,
ceramic2,
show_unchanged: true, # Show attributes that are the same
highlight_diff: false, # Don't highlight only differences
use_colors: true # Use color coding (red/green)
)
# With indentation
diff_score, diff_tree = Lutaml::Model::Serialize.diff_with_score(
ceramic1,
ceramic2,
indent: " "
)Collection Handling
The diff functionality handles collections (arrays) intelligently:
class CeramicCollection < Lutaml::Model::Serializable
attribute :name, :string
attribute :items, Ceramic, collection: true
end
collection1 = CeramicCollection.new(
name: "Blue Collection",
items: [
Ceramic.new(type: "Bowl", glaze: glaze1),
Ceramic.new(type: "Plate", glaze: glaze1)
]
)
collection2 = CeramicCollection.new(
name: "Mixed Collection",
items: [
Ceramic.new(type: "Bowl", glaze: glaze1), # Same as first
Ceramic.new(type: "Cup", glaze: glaze2) # Different
]
)
diff_score, diff_tree = Lutaml::Model::Serialize.diff_with_score(collection1, collection2)
# Shows detailed diff of collection items with indexesUse Cases
The comparison and diff functionality is particularly useful for:
-
Testing: Verify model equality in specs
-
Data Migration: Compare objects before/after transformation
-
Auditing: Track changes to model instances
-
Data Validation: Identify differences in data imports
-
Debugging: Visualize object differences during development
-
API Testing: Compare expected vs actual responses
Value types
General types
Lutaml::Model supports the following attribute value types.
Every type has a corresponding Ruby class and a serialization format type.
| Lutaml::Model::Type | Ruby class | XML | JSON | YAML | Example value |
|---|---|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
complex element |
object |
map |
|
(nil value) |
|
|
|
|
|
|
|
|
|
|
|
Decimal type
|
Warning
|
Decimal is an optional feature. |
The Decimal type is a value type that is disabled by default.
|
Note
|
The reason why the Decimal type is disabled by default is that the
BigDecimal class became optional to the standard Ruby library from Ruby 3.4
onwards. The Decimal type is only enabled when the bigdecimal library is
loaded.
|
The following code needs to be run before using (and parsing) the Decimal type:
require 'bigdecimal'If the bigdecimal library is not loaded, usage of the Decimal type will
raise a Lutaml::Model::TypeNotSupportedError.
Symbol type
The Symbol type provides support for Ruby symbols across all serialization formats.
Type Casting Behavior:
The Symbol type only accepts valid string inputs and existing symbols:
-
✅ Non-empty strings:
"active"→:active -
✅ Existing symbols:
:pending→:pending -
✅ Wrapper format:
":done:"→:done -
✅ Other types: integers, arrays, hashes, booleans → symbol (works similar to string type)
-
❌ Empty strings:
""→nil
Since not all serialization formats natively support symbols (XML, JSON, TOML don’t), the Symbol type uses format-specific serialization strategies:
-
YAML: Uses native symbol format (
:symbol) -
XML/JSON/TOML: Uses wrapper format (
":symbol:") for compatibility
The Symbol type automatically handles conversion between these formats when parsing and serializing.
class Task < Lutaml::Model::Serializable
attribute :status, :symbol
attribute :priority, :symbol
xml do
root "task"
map_element "status", to: :status
map_element "priority", to: :priority
end
json do
map "status", to: :status
map "priority", to: :priority
end
end
task = Task.new(status: :in_progress, priority: :high)
# XML serialization uses wrapper format
task.to_xml
# => <task><status>:in_progress:</status><priority>:high:</priority></task>
# JSON serialization uses wrapper format
task.to_json
# => {"status":":in_progress:","priority":":high:"}
# YAML serialization uses native symbols
task.to_yaml
# => ---
# => status: :in_progress
# => priority: :high
# All formats parse back to Ruby symbols correctly
Task.from_xml(task.to_xml).status # => :in_progress
Task.from_json(task.to_json).status # => :in_progress
Task.from_yaml(task.to_yaml).status # => :in_progressCustom types
General
A custom type is a user-defined class that extends the behavior of built-in
types. A built-in type is one that is provided by Lutaml::Model, such as
:string, :integer, or :date.
Understanding Types and Models
Lutaml::Model provides two approaches to define custom data structures:
- Types
-
Defines primitive values that represent a basic unit of information. A type cannot be further decomposed. Inherits from
Lutaml::Model::Type::Value. - Models
-
Defines objects composed of multiple attributes, of which each attribute can be a type or a model. Inherits from
Lutaml::Model::Serializable.
The key differences are described in the table below.
| Aspect | Types (Type::Value) |
Models (Serializable) |
|---|---|---|
Purpose |
Represent single primitive-like values with custom behavior |
Represent complex objects with multiple attributes and relationships |
Storage |
Contains a single |
Contains multiple attributes defined via |
Use cases |
Value transformation, validation, format-specific serialization of primitives |
Complex nested data structures, objects with multiple properties |
Required methods |
|
None (provided by framework) |
Inheritance |
|
|
Registration |
Can be registered as reusable types via |
Not registered as types, used directly as classes |
Serialization control |
Fine-grained control per format via |
Controlled via mapping blocks ( |
Quick Reference: Type vs Model
# ✅ CUSTOM TYPE - for single values with special behavior
class PostCode < Lutaml::Model::Type::String
def self.cast(value)
value.to_s.upcase.gsub(/\s/, '') # Normalize: remove spaces, uppercase
end
end
# ✅ MODEL - for complex objects with multiple attributes
class Address < Lutaml::Model::Serializable
attribute :street, :string
attribute :city, :string
attribute :postal_code, PostCode # Uses the custom type above
end
# Usage in a model
class Person < Lutaml::Model::Serializable
attribute :name, :string # Built-in type
attribute :postal_code, PostCode # Custom type (single value)
attribute :address, Address # Serializable object (multiple attributes)
endWhen to use custom types and models
Use Custom Types when:
-
You need to transform or validate a single primitive value
-
You want consistent behavior across multiple attributes of the same type
-
You need format-specific serialization of primitive data
-
You’re creating reusable value types (like currency, phone numbers, postcodes)
-
The data represents a single conceptual value, even if complex internally
Use Models when:
-
You need to model objects with multiple attributes
-
You want to define relationships between objects
-
You need complex nested structures
-
The data represents a distinct entity or concept with multiple properties
-
You need different serialization mappings for the same object structure
Creating custom types
A custom class can be used as an attribute type. The custom class must inherit
from Lutaml::Model::Type::Value or a class that inherits from it.
A class inheriting from the Value class carries the attribute value which
stores the one-and-only "true" value that is independent of serialization
formats.
The minimum requirement for a custom class is to implement the following methods:
self.cast(value)-
Assignment of an external value to the
Valueclass to be set asvalue. Casts the value to the custom type. self.serialize(value)-
Serializes the custom type to an object (e.g. a string). Takes the internal
valueand converts it into an output suitable for serialization.
class FiveDigitPostCode < Lutaml::Model::Type::String
def self.cast(value)
value = value.to_s if value.is_a?(Integer)
unless value.is_a?(::String)
raise Lutaml::Model::TypeError, "Invalid value for type 'FiveDigitPostCode'"
end
# Pad zeros to the left
value.rjust(5, '0')
end
def self.serialize(value)
value
end
end
class Studio < Lutaml::Model::Serializable
attribute :postcode, FiveDigitPostCode
endPractical examples: Type vs Model
Use a custom type when you need to handle currency with consistent formatting and validation:
# Custom Type - represents a single currency value
class Currency < Lutaml::Model::Type::Value
def self.cast(value)
case value
when String
# Remove currency symbols and convert to float
cleaned = value.gsub(/[$,]/, '')
Float(cleaned)
when Numeric
value.to_f
else
raise Lutaml::Model::TypeError, "Invalid currency value: #{value}"
end
end
def self.serialize(value)
sprintf("%.2f", value)
end
# Format-specific serialization
def to_xml
"$#{sprintf('%.2f', value)}"
end
def to_json(*_args)
value # JSON uses numbers
end
end
class Product < Lutaml::Model::Serializable
attribute :name, :string
attribute :price, Currency # Reusable custom type
attribute :wholesale_price, Currency # Same type, consistent behavior
endUse a Serializable object when you need multiple related attributes:
# Serializable object - represents a complex address with multiple attributes
class Address < Lutaml::Model::Serializable
attribute :street, :string
attribute :city, :string
attribute :postal_code, :string
attribute :country, :string
# Define how this complex object maps to different formats
xml do
root "Address"
map_element "Street", to: :street
map_element "City", to: :city
map_element "PostalCode", to: :postal_code
map_element "Country", to: :country
end
json do
map "street", to: :street
map "city", to: :city
map "postalCode", to: :postal_code
map "country", to: :country
end
end
class Studio < Lutaml::Model::Serializable
attribute :name, :string
attribute :address, Address # Complex nested object
end# GOOD: Custom Type for phone numbers (single conceptual value)
class PhoneNumber < Lutaml::Model::Type::String
def self.cast(value)
# Normalize phone number format
value.to_s.gsub(/\D/, '') # Remove non-digits
end
def to_xml
# Format for XML: +1-555-123-4567
"+1-#{value[0..2]}-#{value[3..5]}-#{value[6..9]}"
end
end
# GOOD: Serializable for contact info (multiple related attributes)
class ContactInfo < Lutaml::Model::Serializable
attribute :email, :string
attribute :phone, PhoneNumber # Uses custom type
attribute :preferred_contact_method, :string
end
# BAD: Don't use Serializable for simple values
class BadPhoneNumber < Lutaml::Model::Serializable
attribute :number, :string # Overkill for a single value
end
# BAD: Don't use Type for complex structures
class BadContactInfo < Lutaml::Model::Type::Value
# This would be difficult to manage and serialize properly
def self.cast(value)
# Complex parsing logic for multiple fields... ❌
end
endRegistering custom types
Custom types can be registered for reuse across your application using symbols:
# Register the custom type
Lutaml::Model::Type.register(:currency, Currency)
Lutaml::Model::Type.register(:phone, PhoneNumber)
# Now you can use symbols instead of class names
class Product < Lutaml::Model::Serializable
attribute :name, :string
attribute :price, :currency # Uses registered Currency type
attribute :contact_phone, :phone # Uses registered PhoneNumber type
end
# You can also look up registered types
currency_type = Lutaml::Model::Type.lookup(:currency)
# => CurrencyType casting and validation
Custom types automatically handle type casting and can include validation:
class TemperatureInCelsius < Lutaml::Model::Type::Integer
def self.cast(value)
temp = super(value) # Use parent's casting first
# Add validation
if temp < -273 || temp > 5000
raise Lutaml::Model::TypeError,
"Temperature #{temp}°C is outside valid range (-273 to 5000)"
end
temp
end
def to_xml
"#{value}°C"
end
end
class KilnSettings < Lutaml::Model::Serializable
attribute :firing_temperature, TemperatureInCelsius
end
# Usage
kiln = KilnSettings.new(firing_temperature: "1200") # String gets cast to Integer
# => #<KilnSettings @firing_temperature=1200>
# Invalid values raise errors
kiln = KilnSettings.new(firing_temperature: "-300")
# => Lutaml::Model::TypeError: Type Error: Temperature -300°C is outside valid rangeExtending built-in types
You can extend existing built-in types to add custom behavior:
# Extend String type for email validation
class EmailAddress < Lutaml::Model::Type::String
EMAIL_REGEX = /\A[\w+\-.]+@[a-z\d\-]+(\.[a-z\d\-]+)*\.[a-z]+\z/i
def self.cast(value)
email = super(value) # Use String's casting
unless email.match?(EMAIL_REGEX)
raise Lutaml::Model::TypeError, "Invalid email format: #{email}"
end
email.downcase # Normalize to lowercase
end
end
# Extend Integer type for ID validation
class PositiveInteger < Lutaml::Model::Type::Integer
def self.cast(value)
num = super(value) # Use Integer's casting
if num <= 0
raise Lutaml::Model::TypeError, "Value must be positive: #{num}"
end
num
end
end
# Extend Date type for business days only
class BusinessDate < Lutaml::Model::Type::Date
def self.cast(value)
date = super(value) # Use Date's casting
if date.saturday? || date.sunday?
raise Lutaml::Model::TypeError, "Business date cannot be weekend: #{date}"
end
date
end
endCustom type inheritance hierarchy
# Base currency type
class Currency < Lutaml::Model::Type::Value
def self.cast(value)
case value
when String
Float(value.gsub(/[^0-9.-]/, ''))
when Numeric
value.to_f
end
end
def self.serialize(value)
sprintf("%.2f", value)
end
end
# Specialized currency types
class USDollar < Currency
def to_xml
"$#{sprintf('%.2f', value)}"
end
end
class Euro < Currency
def to_xml
"€#{sprintf('%.2f', value)}"
end
end
class JPYen < Currency
def self.serialize(value)
value.to_i.to_s # No decimal places for Yen
end
def to_xml
"¥#{value.to_i}"
end
end
# Register specialized types
Lutaml::Model::Type.register(:usd, USDollar)
Lutaml::Model::Type.register(:eur, Euro)
Lutaml::Model::Type.register(:jpy, JPYen)
class InternationalProduct < Lutaml::Model::Serializable
attribute :name, :string
attribute :price_usd, :usd
attribute :price_eur, :eur
attribute :price_jpy, :jpy
endSerialization of custom types
The serialization of custom types can be made to differ per serialization format
by defining methods in the class definitions. This requires additional methods
than the minimum required for a custom class (i.e. self.cast(value) and
self.serialize(value)).
This is useful in the case when different serialization formats of the same model expect differentiated value representations.
The methods that can be overridden are named:
self.from_{format}(serialized_string)-
Deserializes a string of the serialization format and returns the object to be assigned to the
Valueclass'value. to_{format}-
Serializes the object to a string of the serialization format.
The {format} part of the method name is the serialization format in lowercase
(e.g. hash, json, xml, yaml, toml).
Suppose in XML we handle a high-precision date-time type that requires custom serialization methods, but other formats such as JSON do not support this type.
For instance, in the normal DateTime class, the serialized string is
2012-04-07T01:51:37+02:00, and the high-precision format is
2012-04-07T01:51:37.112+02:00.
We create HighPrecisionDateTime class is a custom class that inherits
from Lutaml::Model::Type::DateTime.
class HighPrecisionDateTime < Lutaml::Model::Type::DateTime
# Inherit the `self.cast(value)` and `self.serialize(value)` methods
# from Lutaml::Model::Type::DateTime
# The format looks like this `2012-04-07T01:51:37.112+02:00`
def self.from_xml(xml_string)
::DateTime.parse(xml_string)
end
# The %L adds milliseconds to the time
def to_xml
value.strftime('%Y-%m-%dT%H:%M:%S.%L%:z')
end
end
class Ceramic < Lutaml::Model::Serializable
attribute :kiln_firing_time, HighPrecisionDateTime
xml do
root 'ceramic'
map_element 'kilnFiringTime', to: :kiln_firing_time
# ...
end
endAn XML snippet with the high-precision date-time type:
<ceramic>
<kilnFiringTime>2012-04-07T01:51:37.112+02:00</kilnFiringTime>
<!-- ... -->
</ceramic>When loading the XML snippet, the HighPrecisionDateTime class will be used to
parse the high-precision date-time string.
However, when serializing to JSON, the value will have the high-precision part lost due to the inability of JSON to handle high-precision date-time.
> c = Ceramic.from_xml(xml)
> #<Ceramic:0x0000000104ac7240 @kiln_firing_time=#<HighPrecisionDateTime:0x0000000104ac7240 @value=2012-04-07 01:51:37.112000000 +0200>>
> c.to_json
> # {"kilnFiringTime":"2012-04-07T01:51:37+02:00"}Best practices in using custom types
When implementing custom types:
-
Ensure you have a clear use case: Decide between using a custom type or a model based on whether you need to represent a single value or a complex structure.
-
Inherit appropriately: Use
Lutaml::Model::Type::Valuefor completely new types, or extend built-in types likeType::String,Type::Integerfor specialized behavior. -
Implement required methods: Always implement
self.cast(value)andself.serialize(value)at minimum. -
Add validation: Use the
castmethod to validate and normalize input values. -
Handle format-specific serialization: Override
to_xml,to_json, etc. methods when different formats need different representations. -
Register reusable types: Use
Lutaml::Model::Type.register(:symbol, YourType)for types you’ll use across multiple models. -
Leverage inheritance: Create type hierarchies for related but specialized types.
-
Test thoroughly: Custom types affect both parsing and serialization, so test with all formats you plan to use.
Attributes
Basic attributes
An attribute is the basic building block of a model. It is a named value that stores a single piece of data (which may be one or multiple pieces of data).
An attribute only accepts the type of value defined in the attribute definition.
The attribute value type can be one of the following:
-
Value (inherits from Lutaml::Model::Value)
-
Model (inherits from Lutaml::Model::Serializable)
Syntax:
attribute :name_of_attribute, TypeWhere,
name_of_attribute-
The defined name of the attribute.
Type-
The type of the attribute.
attribute class method to define simple attributesclass Studio < Lutaml::Model::Serializable
attribute :name, :string
attribute :address, :string
attribute :established, :date
ends = Studio.new(name: 'Pottery Studio', address: '123 Clay St', established: Date.new(2020, 1, 1))
puts s.name
#=> "Pottery Studio"
puts s.address
#=> "123 Clay St"
puts s.established
#=> <Date: 2020-01-01>Restricting the value of an attribute
The restrict class method is used to update or refine the validation rules for an attribute that has already been defined. This allows you to apply additional or stricter constraints to an existing attribute without redefining it.
restrict class method to update the options of an existing attributeclass Studio < Lutaml::Model::Serializable
attribute :name, :string
restrict :name, collection: 1..3, pattern: /[A-Za-z]+/
endclass Document < Lutaml::Model::Serializable
attribute :status, :string
end
class DraftDocument < Document
# Only allow "draft" or "in_review" as valid statuses for drafts
restrict :status, values: %w[draft in_review]
end
class PublishedDocument < Document
# Only allow "published" or "archived" as valid statuses for published documents
restrict :status, values: %w[published archived]
end
# Usage
# Call .validate! to trigger validation and raise an error if the value is not allowed
Document.new(status: "draft").validate! # valid, there are no validation rules for `Document`
Document.new(status: "published").validate! # valid, there are no validation rules for `Document`
DraftDocument.new(status: "draft").validate! # valid
DraftDocument.new(status: "in_review").validate! # valid
DraftDocument.new(status: "published").validate! # raises error (not allowed)
PublishedDocument.new(status: "published").validate! # valid
PublishedDocument.new(status: "archived").validate! # valid
PublishedDocument.new(status: "draft").validate! # raises error (not allowed)All options that are supported by the attribute class method are also supported by the restrict method. Any unsupported option passed to restrict will result in a Lutaml::Model::InvalidAttributeOptionsError being raised.
Polymorphic attributes
General
A polymorphic attribute is an attribute that can accept multiple types of values. This is useful when the attribute defines common characteristics and behaviors among different types.
An attribute with a defined value type also accepts values that are of a class that is a subclass of the defined type.
The assigned attribute of Type accepts polymorphic classes as long as the
assigned instance is of a class that either inherits from the declared type or
matches it.
Naïve approach does not work…
A naïve polymorphic approach is to define an attribute with a superclass type and assign instances of subclasses to it.
While this approach works (somewhat) in modeling, it does not work with serialization (half) or deserialization (not at all).
The following example illustrates why such approach is naïve.
class Studio < Lutaml::Model::Serializable
attribute :name, :string
end
# CeramicStudio is a specialization of Studio
class CeramicStudio < Studio
attribute :clay_type, :string
end
class PotteryClass < Lutaml::Model::Serializable
# the :studio attribute should accept Studio and CeramicStudio
attribute :studio, Studio
end# This works
> s = Studio.new(name: 'Pottery Studio')
> p = PotteryClass.new(studio: s)
> p.studio
# => <Studio:0x0000000104ac7240 @name="Pottery Studio", @address=nil, @established=nil>
# A subclass of Studio is also valid
> s = CeramicStudio.new(name: 'Ceramic World', clay_type: 'Red')
> p = PotteryClass.new(studio: s)
> p.studio
# => <CeramicStudio:0x0000000104ac7240 @name="Ceramic World", @address=nil, @established=nil, @clay_type="Red">
> p.studio.name
# => "Ceramic World"
> p.studio.clay_type
# => "Red"So far so good. However, this approach does not work in serialization.
This is what happens when we call to_yaml on the PotteryClass instance.
> puts p.to_yaml
# => ---
# => studio:
# => name: Ceramic World
# => clay_type: RedWhen deserializing the YAML string, the studio attribute will be deserialized
as an instance of Studio, not CeramicStudio. This means that the clay_type
attribute will be lost.
> p = PotteryClass.load_yaml("---\nstudio:\n name: Ceramic World\n clay_type: Red")
> p.studio
# => <Studio:0x0000000104ac7240 @name="Ceramic World">
> p.studio.clay_type
# => ERRORProper polymorphic approaches
Lutaml::Model offers rich support for polymorphic attributes, through configuration at both attribute and serialization levels.
In polymorphism, there are the following components:
- polymorphic attribute
-
the attribute that can be assigned multiple types.
- polymorphic attribute class
-
the class that has a polymorphic attribute.
- polymorphic superclass
-
a class assigned to a polymorphic attribute that serves as the superclass for all accepted polymorphic classes.
- polymorphic subclass
-
a class that is a subclass of the polymorphic superclass and can be assigned to the polymorphic attribute. There are often more than 2 subclasses in a scenario since polymorphism is meant to apply to multiple types.
To utilize polymorphic attributes, modification to all of these components are necessary.
In serialized form, polymorphic classes are differentiated by an explicit "polymorphic class differentiator".
In key-value formats like YAML, the polymorphic class differentiator is typically a key-value pair that contains the polymorphic class name.
references:
- _class: Document # This is a DocumentReference
name: "The Tibetan Book of the Dead"
document_id: "book:tbtd"
- _class: Anchor # This is an AnchorReference
name: "Chapter 1"
anchor_id: "book:tbtd:anchor-1"In XML, the polymorphic class differentiator is typically an attribute that contains the polymorphic class name.
<references>
<!-- The "document-ref" value is a DocumentReference -->
<reference reference-type="document-ref">
<name>The Tibetan Book of the Dead</name>
<document_id>book:tbtd</document_id>
</reference>
<!-- The "anchor-ref" value is an AnchorReference -->
<reference reference-type="anchor-ref">
<name>Chapter 1</name>
<anchor_id>book:tbtd:anchor-1</anchor_id>
</reference>
</references>|
Note
|
While it is possible to determine different polymorphic classes based on the attributes they contain, such mechanism would not be able to determine the polymorphic class if serializations of two polymorphic subclasses can be identical. |
There are two basic scenarios in using polymorphic attributes:
-
Scenario 1: Setting polymorphism in the polymorphic superclass:
-
Defining the polymorphic attribute
-
Setting the differentiator in the polymorphic superclass
-
Mapping in the polymorphic superclass
-
-
Scenario 2: Setting polymorphism in the individual polymorphic subclasses:
-
Defining the polymorphic attribute
-
Setting the differentiator in the individual polymorphic subclasses
-
Mapping in the polymorphic attribute class and individual polymorphic subclasses
-
|
Note
|
Please refer to spec/lutaml/model/polymorphic_spec.rb for full examples
of implementing polymorphic attributes.
|
Defining the polymorphic attribute
The polymorphic attribute class is a class that has a polymorphic attribute.
At this level, the polymorphic option is used to specify the types that the
polymorphic attribute can accept.
class PolymorphicAttributeClass < Lutaml::Model::Serializable
attribute :attribute_name, (1)
{polymorphic-superclass-class}, (2)
{options}, (3)
polymorphic: [ (4)
polymorphic-subclass-1, (5)
polymorphic-subclass-2,
]
end-
The name of the polymorphic attribute.
-
The polymorphic superclass class.
-
Any options for the attribute.
-
The
polymorphicoption that determines the acceptable polymorphic subclasses, or justtrue. -
The polymorphic subclasses.
The polymorphic option is an array of polymorphic subclasses that the
attribute can accept.
These options enable the following scenarios.
-
If the polymorphic attribute is to only contain instances of the
polymorphic-superclass-class, not its subclasses, then thepolymorphicoption is not needed.In the following code,
ReferenceSethas an attributereferencesthat only accepts instances ofReference. Thepolymorphicoption does not apply.class ReferenceSet < Lutaml::Model::Serializable attribute :references, Reference, collection: true end
-
If the attribute (collection or not) is meant to only contain one type of polymorphic subclasses, then the
polymorphicoption is also not needed, because the polymorphic subclass can be stated as the attribute value type.In the following code,
ReferenceSethas an attributereferencesthat only accepts instances ofDocumentReference, a subclass ofReference. Thepolymorphicoption does not apply.class ReferenceSet < Lutaml::Model::Serializable attribute :references, DocumentReference, collection: true end
-
If the attribute (collection or not) is meant to contain instances belonging to any polymorphic subclass of a defined base class, then set the
polymorphic: trueoption.In the following code,
ReferenceSetis a class that has a polymorphic attributereferences. Thereferencesattribute can accept instances of any polymorphic subclass of theReferencebase class, sopolymorphic: trueis set.class ReferenceSet < Lutaml::Model::Serializable attribute :references, Reference, collection: true, polymorphic: true end
-
If the attribute (collection or not) is meant to contain instances belonging to more than one polymorphic subclass, then those acceptable polymorphic subclasses should be explicitly specified in the
polymorphic: […]option.In the following code,
ReferenceSetis a class that has a polymorphic attributereferences. Thereferencesattribute can accept instances ofDocumentReferenceandAnchorReference, both of which are subclasses ofReference.class ReferenceSet < Lutaml::Model::Serializable attribute :references, Reference, collection: true, polymorphic: [ DocumentReference, AnchorReference, ] end
Differentiating polymorphic subclasses
General
A polymorphic subclass needs an additional attribute with the
polymorphic_class option to allow Lutaml::Model for identifying itself in
serialization. This attribute is called the "polymorphic class differentiator".
There are two methods for setting the polymorphic class differentiator:
-
Setting the polymorphic class differentiator in the polymorphic superclass, as polymorphic subclasses inherit from it (relying on Inheritance).
-
Setting the polymorphic class differentiator in the individual polymorphic subclasses
Setting the differentiator in the polymorphic superclass
The polymorphic class differentiator can be set in the polymorphic superclass. This scenario fits best if there are many polymorphic subclasses and the polymorphic superclass can be modified.
Syntax:
class PolymorphicSuperclass < Lutaml::Model::Serializable
attribute :{_polymorphic_differentiator}, (1)
:string, (2)
polymorphic_class: true (3)
# ...
end-
The polymorphic differentiator is a normal attribute that can be assigned to any name.
-
The polymorphic differentiator must have a value type of
:string. -
The option for
polymorphic_classmust be set totrueto indicate that this attribute accepts subclass types.
Setting the differentiator in the individual polymorphic subclasses
The polymorphic class differentiator can be set in the individual polymorphic subclasses. This scenario fits best if there are few polymorphic subclasses and the polymorphic superclass cannot be modified.
Syntax:
# No modification to the superclass is needed.
class PolymorphicSuperclass < Lutaml::Model::Serializable
# ...
end
# The polymorphic differentiator is set in the subclass.
class PolymorphicSubclass < PolymorphicSuperclass
attribute
:{_polymorphic_differentiator}, (1)
:string, (2)
polymorphic_class: true (3)
# ...
end-
The polymorphic differentiator is a normal attribute that can be assigned to any name.
-
The polymorphic differentiator must have a value type of
:string. -
The option for
polymorphic_classmust be set totrueto indicate that this attribute accepts subclass types.
Polymorphic differentiation in serialization
General
The polymorphic attribute class needs to determine what class to use based on the serialized value of the polymorphic differentiator.
The polymorphic attribute class mapping is format-independent, allowing for differentiation of polymorphic subclasses in different serialization formats.
The mapping of the serialized polymorphic differentiator can be set in either:
-
the polymorphic superclass; or
-
the polymorphic attribute class and the individual polymorphic subclasses.
Mapping in the polymorphic superclass
This use case applies when the polymorphic superclass can be modified, and that polymorphism is intended to apply to all its subclasses.
This is done through the polymorphic_map option in the serialization blocks
inside the polymorphic attribute class.
Syntax:
class PolymorphicSuperclass < Lutaml::Model::Serializable
attribute :{_polymorphic_differentiator}, :string, polymorphic_class: true
xml do
(map_attribute | map_element) "XmlPolymorphicAttributeName", (1)
to: :{_polymorphic_differentiator}, (2)
polymorphic_map: { (3)
"xml-value-for-subclass-1" => PolymorphicSubclass1, (4)
"xml-value-for-subclass-2" => PolymorphicSubclass2,
}
end
(key_value | key_value_format) do
map "KeyValuePolymorphicAttributeName", (5)
to: :{_polymorphic_differentiator}, (6)
polymorphic_map: {
"keyvalue-value-for-subclass-1" => PolymorphicSubclass1,
"keyvalue-value-for-subclass-2" => PolymorphicSubclass2,
}
end
end
class PolymorphicSubclass1 < PolymorphicSuperclass
# ...
end
class PolymorphicSubclass2 < PolymorphicSuperclass
# ...
end
class PolymorphicAttributeClass < Lutaml::Model::Serializable
attribute :polymorphic_attribute,
PolymorphicSuperclass,
{options},
polymorphic: [
PolymorphicSubclass1,
PolymorphicSubclass2,
]
# ...
end-
The name of the XML element or attribute that contains the polymorphic differentiator.
-
The name of the polymorphic differentiator attribute defined in
attributewith thepolymorphicoption. -
The
polymorphic_mapoption that determines the class to use based on the value of the differentiator. -
The mapping of the differentiator value to the polymorphic subclass.
-
The name of the key-value element that contains the polymorphic differentiator.
-
The name of the polymorphic differentiator attribute defined in
attributewith thepolymorphicoption.
class Reference < Lutaml::Model::Serializable
attribute :_class, :string, polymorphic_class: true
attribute :name, :string
xml do
map_attribute "reference-type", to: :_class, polymorphic_map: {
"document-ref" => "DocumentReference",
"anchor-ref" => "AnchorReference",
}
map_element "name", to: :name
end
key_value do
map "_class", to: :_class, polymorphic_map: {
"Document" => "DocumentReference",
"Anchor" => "AnchorReference",
}
map "name", to: :name
end
end
class DocumentReference < Reference
attribute :document_id, :string
xml do
map_element "document_id", to: :document_id
end
key_value do
map "document_id", to: :document_id
end
end
class AnchorReference < Reference
attribute :anchor_id, :string
xml do
map_element "anchor_id", to: :anchor_id
end
key_value do
map "anchor_id", to: :anchor_id
end
end
class ReferenceSet < Lutaml::Model::Serializable
attribute :references, Reference, collection: true, polymorphic: [
DocumentReference,
AnchorReference,
]
end---
references:
- _class: Document
name: The Tibetan Book of the Dead
document_id: book:tbtd
- _class: Anchor
name: Chapter 1
anchor_id: book:tbtd:anchor-1<ReferenceSet>
<references reference-type="document-ref">
<name>The Tibetan Book of the Dead</name>
<document_id>book:tbtd</document_id>
</references>
<references reference-type="anchor-ref">
<name>Chapter 1</name>
<anchor_id>book:tbtd:anchor-1</anchor_id>
</references>
</ReferenceSet>Mapping in the polymorphic attribute class and individual polymorphic subclasses
This use case applies when the polymorphic superclass is not meant to be modified.
This is done through the polymorphic_map option in the serialization blocks
inside the polymorphic attribute class, and the polymorphic option in the
individual polymorphic subclasses.
In this scenario, similar to the previous case where the polymorphic differentiator is set at the polymorphic superclass, the following conditions must be satisifed:
-
the polymorphic differentiator attribute name must be the same across polymorphic subclasses
If the model polymorphic differentiator in one polymorphic subclass is
_ref_type, then it must be so in all other polymorphic subclasses. -
the polymorphic differentiator in the serialization formats must be identical within the polymorphic subclasses of that serialization format.
If the XML polymorphic differentiator is
reference-type, then it must be so in the XML of all polymorphic subclasses.
Syntax:
# Assume that we have no access to the base class and we need to define
# polymorphism in the sub-classes.
class PolymorphicSuperclass < Lutaml::Model::Serializable
end
class PolymorphicSubclass1 < PolymorphicSuperclass
attribute :_polymorphic_differentiator, :string
xml do
(map_attribute | map_element) "XmlPolymorphicAttributeName", (1)
to: :_polymorphic_differentiator
end
(key_value | key_value_format) do
map "KeyValuePolymorphicAttributeName", (2)
to: :_polymorphic_differentiator
end
end
class PolymorphicSubclass2 < PolymorphicSuperclass
attribute :_polymorphic_differentiator, :string
xml do
(map_attribute | map_element) "XmlPolymorphicAttributeName2",
to: :_polymorphic_differentiator
end
(key_value | key_value_format) do
map "KeyValuePolymorphicAttributeName2",
to: :_polymorphic_differentiator
end
end
class PolymorphicAttributeClass < Lutaml::Model::Serializable
attribute :polymorphic_attribute,
PolymorphicSuperclass,
{options},
polymorphic: [
PolymorphicSubclass1,
PolymorphicSubclass2,
] (3)
# ...
xml do
map_element "XmlPolymorphicElement", (4)
to: :polymorphic_attribute,
polymorphic: { (5)
# This refers to the polymorphic differentiator attribute in the polymorphic subclass.
attribute: :_polymorphic_differentiator, (6)
class_map: { (7)
"xml-i-am-subclass-1" => "PolymorphicSubclass1",
"xml-i-am-subclass-2" => "PolymorphicSubclass2",
},
}
end
(key_value | key_value_format) do
map "KeyValuePolymorphicAttributeName", (8)
to: :polymorphic_attribute,
polymorphic: { (9)
attribute: :_polymorphic_differentiator, (10)
class_map: { (11)
"keyvalue-i-am-subclass-1" => "PolymorphicSubclass1",
"keyvalue-i-am-subclass-2" => "PolymorphicSubclass2",
},
}
end
end-
The name of the XML element or attribute that contains the polymorphic differentiator.
-
The name of the key-value element that contains the polymorphic differentiator.
-
Definition of the polymorphic attribute and the polymorphic subclasses in the polymorphic attribute class.
-
The name of the XML element that contains the polymorphic attributes. This must be an element as a polymorphic attribute must be a model.
-
The
polymorphicoption on a mapping defines necessary information for polymorphic serialization. -
The
attribute:name of the polymorphic differentiator attribute defined in the polymorphic subclass. -
The
class_map:option that determines the polymorphic subclass to use based on the value of the differentiator. -
The name of the key-value format key that contains the polymorphic attributes.
-
Same as <5>, but for the key-value format.
-
Same as <6>, but for the key-value format.
-
Same as <7>, but for the key-value format.
class Reference < Lutaml::Model::Serializable
attribute :name, :string
end
class DocumentReference < Reference
attribute :_class, :string
attribute :document_id, :string
xml do
map_element "document_id", to: :document_id
map_attribute "reference-type", to: :_class
end
key_value do
map "document_id", to: :document_id
map "_class", to: :_class
end
end
class AnchorReference < Reference
attribute :_class, :string
attribute :anchor_id, :string
xml do
map_element "anchor_id", to: :anchor_id
map_attribute "reference-type", to: :_class
end
key_value do
map "anchor_id", to: :anchor_id
map "_class", to: :_class
end
end
class ReferenceSet < Lutaml::Model::Serializable
attribute :references, Reference, collection: true, polymorphic: [
DocumentReference,
AnchorReference,
]
xml do
root "ReferenceSet"
map_element "reference", to: :references, polymorphic: {
# This refers to the attribute in the polymorphic model, you need
# to specify the attribute name (which is specified in the sub-classed model).
attribute: "_class",
class_map: {
"document-ref" => "DocumentReference",
"anchor-ref" => "AnchorReference",
},
}
end
key_value do
map "references", to: :references, polymorphic: {
attribute: "_class",
class_map: {
"Document" => "DocumentReference",
"Anchor" => "AnchorReference",
},
}
end
end---
references:
- _class: Document
name: The Tibetan Book of the Dead
document_id: book:tbtd
- _class: Anchor
name: Chapter 1
anchor_id: book:tbtd:anchor-1<ReferenceSet>
<reference reference-type="document-ref">
<name>The Tibetan Book of the Dead</name>
<document_id>book:tbtd</document_id>
</reference>
<reference reference-type="anchor-ref">
<name>Chapter 1</name>
<anchor_id>book:tbtd:anchor-1</anchor_id>
</reference>
</ReferenceSet>Collection attributes
Define attributes as collections (arrays or hashes) to store multiple values
using the collection option.
When defining a collection attribute, it is important to understand the default initialization behavior and how to customize it.
By default, collections are initialized as nil. However, if you want the collection to be initialized as an empty array, you can use the initialize_empty: true option.
collection can be set to:
true-
The attribute contains an unbounded collection of objects of the declared class.
{min}..{max}-
The attribute contains a collection of objects of the declared class with a count within the specified range. If the number of objects is out of this numbered range, a
CollectionCountOutOfRangeErrorwill be raised.When set to
0..1, it means that the attribute is optional, it could be empty or contain one object of the declared class.When set to
1..(equivalent to1..Infinity), it means that the attribute must contain at least one object of the declared class and can contain any number of objects.When set to 5..10` means that there is a minimum of 5 and a maximum of 10 objects of the declared class. If the count of values for the attribute is less then 5 or greater then 10, the
CollectionCountOutOfRangeErrorwill be raised.
Syntax:
attribute :name_of_attribute, Type, collection: true
attribute :name_of_attribute, Type, collection: {min}..{max}
attribute :name_of_attribute, Type, collection: {min}..collection option to define a collection attributeclass Studio < Lutaml::Model::Serializable
attribute :location, :string
attribute :potters, :string, collection: true
attribute :address, :string, collection: 1..2
attribute :hobbies, :string, collection: 0..
end> Studio.new
> # address count is `0`, must be between 1 and 2 (Lutaml::Model::CollectionCountOutOfRangeError)
> Studio.new({ address: ["address 1", "address 2", "address 3"] })
> # address count is `3`, must be between 1 and 2 (Lutaml::Model::CollectionCountOutOfRangeError)
> Studio.new({ address: ["address 1"] }).potters
> # []
> Studio.new({ address: ["address 1"] }).address
> # ["address 1"]
> Studio.new(address: ["address 1"], potters: ['John Doe', 'Jane Doe']).potters
> # ['John Doe', 'Jane Doe']# Default to `nil`
class SomeModel < Lutaml::Model::Serializable
attribute :coll, :string, collection: true
xml do
root "some-model"
map_element 'collection', to: :coll
end
key_value do
map 'collection', to: coll
end
end
puts SomeModel.new.coll
# => nil
puts SomeModel.new.to_xml
# =>
# <some-model xsi:xmlns="..."><collection xsi:nil="true"/></some-model>
puts SomeModel.new.to_yaml
# =>
# ---
# coll: null# Default to empty array
class SomeModel < Lutaml::Model::Serializable
attribute :coll, :string, collection: true, initialize_empty: true
xml do
map_element 'collection', to: :coll
end
key_value do
map 'collection', to: coll
end
end
puts SomeModel.new.coll
# => []
puts SomeModel.new.to_xml
# =>
# <some-model><collection/></some-model>
puts SomeModel.new.to_yaml
# =>
# ---
# coll: []Derived attributes
A derived attribute has a value computed dynamically on evaluation of an instance method.
It is defined using the method: option along with a mandatory type specification. If the return value is not of the type, it will be casted to the specified type.
Syntax:
attribute :name_of_attribute, Type, method: :instance_method_nameclass Invoice < Lutaml::Model::Serializable
attribute :subtotal, :float
attribute :tax, :float
attribute :total, :float, method: :total_value
def total_value
subtotal + tax
end
end
i = Invoice.new(subtotal: 100.0, tax: 12.0)
i.total
#=> 112.0
puts i.to_yaml
#=> ---
#=> subtotal: 100.0
#=> tax: 12.0
#=> total: 112.0Choice attributes
The choice directive allows specifying that elements from the specified range are included.
|
Note
|
Attribute-level definitions are supported. This can be used with both
key_value and xml mappings.
|
Syntax:
choice(min: {min}, max: {max}) do
{block}
endWhere,
min-
The minimum number of elements that must be included. The minimum value can be
0. max-
The maximum number of elements that can be included. The maximum value can go up to
Float::INFINITY. block-
The block of elements that must be included. The block can contain multiple
attributeandchoicedirectives.
choice directive to define a set of attributes with a rangeclass Studio < Lutaml::Model::Serializable
choice(min: 1, max: 3) do
choice(min: 1, max: 2) do
attribute :prefix, :string
attribute :forename, :string
end
attribute :completeName, :string
end
endThis means that the Studio class must have at least one and at most three
attributes.
-
The first choice must have at least one and at most two attributes.
-
The second attribute is the
completeName. -
The first choice can have either the
prefixandforenameattributes or just theforenameattribute. -
The last attribute
completeNameis optional.
Choice and sequence can be used together to create complex structures.
choice (model-level) and sequence (XML-level) directives togetherclass Person < Lutaml::Model::Serializable
attribute :first_name, :string
attribute :last_name, :string
choice do
attribute :age, :integer
attribute :dob, :string
end
choice(min: 1, max: 2) do
attribute :email, :string, collection: 0..2
attribute :phone, :string, collection: 0..2
attribute :address, :string, collection: true
end
xml do
root "Person", mixed: true
sequence do
map_element :FirstName, to: :first_name
map_element :LastName, to: :last_name
map_element :Age, to: :age
map_element :Dob, to: :dob
map_element :Email, to: :email
map_element :Phone, to: :phone
map_element :Address, to: :address
end
end
endChoosing between custom types and transform procs
When deciding how to implement value transformations, consider:
Use Custom Value Type classes when:
-
Bidirectional transformations are needed across formats
-
Format-specific representations are required (XML wants YYYYMMDD, JSON wants DDMMYYYY)
-
The logic will be reused across multiple attributes or models
-
Complex parsing or calculation is involved (e.g., ISO week date calculations)
-
Type safety and encapsulation are important
Use Attribute-level transform procs when:
-
Same simple transformation applies to ALL serialization formats uniformly
-
Logic is specific to one attribute in one model (non-reusable)
-
Quick inline modification is sufficient
-
No format-specific behavior is needed
Use Mapping-level transform procs when:
-
Different transformation needed per serialization format
-
One-off, non-reusable transformation
-
Combined with attribute-level transforms for multi-stage processing
See Value Transformations Guide for complete decision matrix and examples.
|
Note
|
The choice directive can be used with import_model_attributes. For more details, see Using import_model_attributes inside a choice block.
|
Importable models for reuse
An importable model is a model that can be imported into another model using the
import_* directive.
This feature works both with XML and key-value formats.
-
The import order determines how elements and attributes are overwritten.
-
An importable model with XML serialization mappings requires setting the model’s XML serialization configuration with the
no_rootdirective.
The model can be imported into another model using the following directives:
import_model-
imports both attributes and mappings.
import_model_attributes-
imports only attributes.
import_model_mappings-
imports only mappings.
|
Note
|
Models with no_root can only be parsed through parent models.
Direct calling NoRootModel.from_xml will raise a NoRootMappingError.
|
|
Note
|
Namespaces are not currently supported in importable models.
If namespace is defined with no_root, NoRootNamespaceError will be raised.
|
class GroupOfItems < Lutaml::Model::Serializable
attribute :name, :string
attribute :type, :string
attribute :code, :string
xml do
no_root
sequence do
map_element "name", to: :name
map_element "type", to: :type, namespace: "http://www.example.com", prefix: "ex1"
end
map_attribute "code", to: :code
end
end
class ComplexType < Lutaml::Model::Serializable
attribute :tag, AttributeValueType
attribute :content, :string
attribute :group, :string
import_model_attributes GroupOfItems
xml do
root "GroupOfItems"
map_attribute "tag", to: :tag
map_content to: :content
map_element :group, to: :group
import_model_mappings GroupOfItems
end
end
class SimpleType < Lutaml::Model::Serializable
import_model GroupOfItems
end
class GenericType < Lutaml::Model::Serializable
import_model_mappings GroupOfItems
end<GroupOfItems xmlns:ex1="http://www.example.com">
<name>Name</name>
<ex1:type>Type</ex1:type>
</GroupOfItems>> parsed = GroupOfItems.from_xml(xml)
> # Lutaml::Model::NoRootMappingError: "GroupOfItems has `no_root`, it allowed only for reusable models"Using import_model_mappings inside a sequence
You can use import_model_mappings within a sequence block to include the element mappings from another model. This is useful for composing complex XML structures from reusable model components.
The element mappings will be imported inside this specific sequence block that calls the import method, rest of the mappings like content, attributes, etc. will be inserted at the class level.
|
Note
|
import_model and import_model_attributes are not supported inside a sequence block.
|
class Address < Lutaml::Model::Serializable
attribute :street, :string
attribute :city, :string
attribute :zip, :string
xml do
no_root
map_element :street, to: :street
map_element :city, to: :city
map_element :zip, to: :zip
end
end
class Person < Lutaml::Model::Serializable
attribute :name, :string
import_model_attributes Address
xml do
root "Person"
map_element :name, to: :name
sequence do
import_model_mappings Address
end
end
end
# Example XML output:
valid_xml = <<~XML
<Person>
<name>John Doe</name>
<street>123 Main St</street>
<city>Metropolis</city>
<zip>12345</zip>
</Person>
XML
invalid_xml = <<~XML
<Person>
<name>John Doe</name>
<street>123 Main St</street>
<zip>12345</zip>
</Person>
XML
Person.from_xml(valid_xml) # #<Person:0x00000002d56b3988 @city="Metropolis", @name="John Doe", @street="123 Main St", @zip="12345">
Person.from_xml(invalid_xml) # raises `Element `zip` does not match the expected sequence order element `city` (Lutaml::Model::IncorrectSequenceError)`Using import_model_attributes inside a choice block
You can use import_model_attributes within a choice block to allow a model to accept one or more sets of attributes from other models, with flexible cardinality. This is especially useful when you want to allow a user to provide one or more alternative forms of information (e.g., contact methods) in your model.
For example, suppose you want a Person model that can have either an email, a phone, or both as contact information. You can define ContactEmail and ContactPhone as importable models, and then use import_model_attributes for both, inside a choice block in the Person model.
|
Note
|
The import_model_attributes method is used to import the attributes from the other model into the current model. The imported attributes will be associated to the choice block that calls the import method.
|
class ContactEmail < Lutaml::Model::Serializable
attribute :email, :string
xml do
no_root
map_element :email, to: :email
end
end
class ContactPhone < Lutaml::Model::Serializable
attribute :phone, :string
xml do
no_root
map_element :phone, to: :phone
end
end
class Person < Lutaml::Model::Serializable
# Allow either or both contact methods, but at least one must be present
choice(min: 1, max: 2) do
import_model_attributes ContactEmail
import_model_attributes ContactPhone
end
xml do
root "Person"
map_element :email, to: :email
map_element :phone, to: :phone
end
end
valid_xml = <<~XML
<Person>
<email>john.doe@example.com</email>
<phone>1234567890</phone>
</Person>
XML
Person.from_xml(valid_xml).validate! # #<Person:0x00000002d0e27fe8 @email="john.doe@example.com", @phone="1234567890">
invalid_xml = <<~XML
<Person></Person>
XML
Person.from_xml(invalid_xml).validate! # raises `Lutaml::Model::ValidationError` errorUsing register functionality
The register functionality is useful when you want to reference or reuse a model by a symbolic name (e.g., across files or in dynamic scenarios), rather than by direct class reference.
Register
register = Lutaml::Model::Register.new(:importable_model)
register.register_model(GroupOfItems, id: :group_of_items)The id: :group_of_items assigns a symbolic name to the registered model, which can then be used in import_model :group_of_items.
class GroupOfSubItems < Lutaml::Model::Serializable
import_model :group_of_items
endThe import_model :group_of_items will behave the same as import_model GroupOfItems except the class is resolved from the provided register.
|
Note
|
All the import_* methods support the use of register functionality.
|
|
Note
|
For more details on registers, see Custom Registers. |
Attribute value transform
An attribute value transformation is used when the value of an attribute needs to be transformed around assignment.
There are occasions where the value of an attribute is to be transformed during assignment and retrieval, such that when the external usage of the value differs from the internal model representation.
|
Note
|
Value transformation can be applied at the attribute-level or at the serialization-mapping level. They can also be applied together. |
Given a model that stores a measurement composed of a numerical value and a unit, where the numerical value is used for calculations inside the model, but the external representation of that value is a string (across all serialization formats).
-
Internal:
number: 10.20,unit: cm. -
External:
"10.20 cm"
The transform option at the attribute method is used to define a
transformation Proc for the attribute value.
Syntax:
class SomeObject < Lutaml::Model::Serializable
attribute :attribute_name, {attr_type}, transform: {
export: ->(value) { ... },
import: ->(value) { ... }
}
endThe transform option also support collection attributes.
Where,
attribute_name-
The name of the attribute.
attr_type-
The type of the attribute.
transform-
The option to define a transformation for the attribute value.
export-
The transformation
Procfor the value when it is being retrieved from the model. import-
The transformation
Procfor the value when it is being assigned to the model.
class Ceramic < Lutaml::Model::Serializable
attribute :name, :string, transform: {
export: ->(value) { value.upcase },
import: ->(value) { value.downcase }
}
end> c = Ceramic.new(name: "Celadon")
> c.name
> # "CELADON"
> c.instance_attribute_get(:@name)
> # "Celadon"
> Ceramic.new(name: "Celadon").name = "Raku"
> # "RAKU"Value validation
General
There are several mechanisms to validate attribute values in Lutaml::Model.
Values of an enumeration
An attribute can be defined as an enumeration by using the values directive.
The values directive is used to define acceptable values in an attribute. If
any other value is given, a Lutaml::Model::InvalidValueError will be raised.
Syntax:
attribute :name_of_attribute, Type, values: [value1, value2, ...]The values set inside the values: option can be of any type, but they must
match the type of the attribute. The values are compared using the == operator,
so the type must implement the == method.
Also, If all the elements in values directive are strings then lutaml-model add some enum convenience methods, for each of the value the following three methods are added
-
value1: will return value if set -
value1?: will return true if value is set, false otherwise -
value1=: will set the value ofname_of_attributeequal tovalue1if truthy value is given, and remove it otherwise.
values directive to define acceptable values for an attribute (basic types)class GlazeTechnique < Lutaml::Model::Serializable
attribute :name, :string, values: ["Celadon", "Raku", "Majolica"]
end> GlazeTechnique.new(name: "Celadon").name
> # "Celadon"
> GlazeTechnique.new(name: "Raku").name
> # "Raku"
> GlazeTechnique.new(name: "Majolica").name
> # "Majolica"
> GlazeTechnique.new(name: "Earthenware").name
> # Lutaml::Model::InvalidValueError: Invalid value for attribute 'name'The values can be Serialize objects, which are compared using the ==
and the hash methods through the Lutaml::Model::ComparableModel module.
values directive to define acceptable values for an attribute (Serializable objects)class Ceramic < Lutaml::Model::Serializable
attribute :type, :string
attribute :firing_temperature, :integer
end
class CeramicCollection < Lutaml::Model::Serializable
attribute :featured_piece,
Ceramic,
values: [
Ceramic.new(type: "Porcelain", firing_temperature: 1300),
Ceramic.new(type: "Stoneware", firing_temperature: 1200),
Ceramic.new(type: "Earthenware", firing_temperature: 1000),
]
end> CeramicCollection.new(featured_piece: Ceramic.new(type: "Porcelain", firing_temperature: 1300)).featured_piece
> # Ceramic:0x0000000104ac7240 @type="Porcelain", @firing_temperature=1300
> CeramicCollection.new(featured_piece: Ceramic.new(type: "Bone China", firing_temperature: 1300)).featured_piece
> # Lutaml::Model::InvalidValueError: Invalid value for attribute 'featured_piece'Serialize provides a validate method that checks if all its attributes have
valid values. This is necessary for the case when a value is valid at the
component level, but not accepted at the aggregation level.
If a change has been made at the component level (a nested attribute has
changed), the aggregation level needs to call the validate method to verify
acceptance of the newly updated component.
validate method to check if all attributes have valid values> collection = CeramicCollection.new(featured_piece: Ceramic.new(type: "Porcelain", firing_temperature: 1300))
> collection.featured_piece.firing_temperature = 1400
> # No error raised in changed nested attribute
> collection.validate
> # Lutaml::Model::InvalidValueError: Invalid value for attribute 'featured_piece'String values restricted to patterns
An attribute that accepts a string value accepts value validation using regular expressions.
Syntax:
attribute :name_of_attribute, :string, pattern: /regex/pattern option to restrict the value of an attributeIn this example, the color attribute takes hex color values such as #ccddee.
A regular expression can be used to validate values assigned to the attribute.
In this case, it is /^#([A-Fa-f0-9]{6}|[A-Fa-f0-9]{3})$/.
class Glaze < Lutaml::Model::Serializable
attribute :color, :string, pattern: /\A#([A-Fa-f0-9]{6}|[A-Fa-f0-9]{3})\z/
end> Glaze.new(color: '#ff0000').color
> # "#ff0000"
> Glaze.new(color: '#ff000').color
> # Lutaml::Model::InvalidValueError: Invalid value for attribute 'color'Attribute value defaults
Specify default values for attributes using the default option.
The default option can be set to a value or a lambda that returns a value.
Syntax:
attribute :name_of_attribute, Type, default: -> { value }default option to set a default value for an attributeclass Glaze < Lutaml::Model::Serializable
attribute :color, :string, default: -> { 'Clear' }
attribute :temperature, :integer, default: -> { 1050 }
end> Glaze.new.color
> # "Clear"
> Glaze.new.temperature
> # 1050The "default behavior" (pun intended) is to not render a default value if the current value is the same as the default value.
Attribute as raw string
An attribute can be set to read the value as raw string for XML, by using the raw: true option.
Syntax:
attribute :name_of_attribute, :string, raw: trueraw option to read raw value for an XML attributeclass Person < Lutaml::Model::Serializable
attribute :name, :string
attribute :description, :string, raw: true
endFor the following XML snippet:
<Person>
<name>John Doe</name>
<description>
A <b>fictional person</b> commonly used as a <i>placeholder name</i>.
</description>
</Person>> Person.from_xml(xml)
> # <Person:0x0000000107a3ca70
@description="\n A <b>fictional person</b> commonly used as a <i>placeholder name</i>.\n ",
@element_order=["text", "name", "text", "description", "text"],
@name="John Doe",
@ordered=nil>Collections
General
Collections are used to represent a contained group of multiple instances of models.
Typically, a collection represents an "Array" or a "Set" in information modeling and programming languages. In LutaML, a collection represents an array of model instances.
Models in a collection may be:
-
constrained to be of a single kind;
-
constrained to be of multiple kinds sharing common characteristics;
-
unbounded of any kind.
LutaML Model provides the Lutaml::Model::Collection class for defining
collections of model instances.
Configuration
All formats
The instances directive defined at the Collection class level is used to
define the collection attribute and the model type of the collection elements.
Syntax:
class MyCollection < Lutaml::Model::Collection
instances {attribute}, {ModelType}
endWhere,
attribute-
The name of the attribute that contains the collection.
ModelType-
The model type of the collection elements.
Mapping instances: key-value formats only
The map_instances directive is only used in the key_value block.
Syntax:
class MyCollection < Lutaml::Model::Collection
instances {attribute}, ModelType
key_value do
map_instances to: {attribute}
end
endWhere,
attribute-
The name of the attribute that contains model instances.
This directive maps individual array elements to the defined instances
attribute. These are the items considered part of the Collection and reflected
as Enumerable elements.
Mapping instances: XML only
In the xml block, the map_element, map_attribute directives are used instead.
These directives map individual array elements to the defined instances
attribute. These are the items considered part of the Collection and reflected
as Enumerable elements.
Syntax for an element collection:
class MyCollection < Lutaml::Model::Collection
instances {attribute}, ModelType
xml do
map_element "element-name", to: {attribute}
end
endWhere,
element-name-
The name of the XML element of each model instance.
Syntax for an attribute collection:
class MyCollection < Lutaml::Model::Collection
instances {attribute}, ModelType
xml do
map_attribute "attribute-name", to: {attribute}
end
endWhere,
attribute-name-
The name of the XML attribute that contains all model instances.
XML attribute collections with delimited values
Lutaml::Model supports handling collections realized as XML attributes.
This feature allows you to serialize and deserialize multiple values stored in a single XML attribute, separated by a delimiter.
There are two approaches for handling delimited attribute values:
-
Using the
delimiter:option: provides simple splitting and joining with a fixed delimiter; -
Using the
as_list:option: provides custom import and export logic.
The delimiter option is a straightforward way to handle
attribute values that are delimited strings. It is ideal for simple cases where
you need basic string splitting and joining with a fixed delimiter. It
automatically splits the string during import and joins
the array during export.
class TitleDelimiterCollection < Lutaml::Model::Collection
instances :items, :string
xml do
root "titles"
map_attribute "title", to: :items, delimiter: "; " (1)
end
end-
The delimiter used to split and join the string values.
The as_list option provides full control over how values are split during
import and joined during export. This is useful when you need custom parsing
logic. This allows for more complex transformations beyond simple splitting and
joining that is achieved through using delimiters.
class TitleCollection < Lutaml::Model::Collection
instances :items, :string
xml do
root "titles"
map_attribute "title", to: :items, as_list: {
import: ->(str) { str.split("; ") }, (1)
export: ->(arr) { arr.join("; ") }, (2)
}
end
end-
Custom logic to split the string into an array during import.
-
Custom logic to join the array into a string during export.
delimiter and as_list options for XML attribute collectionsBoth approaches work with the same XML format:
<titles title="Title One; Title Two; Title Three"/># Both collections work identically for basic use cases
collection = TitleCollection.from_xml(xml)
collection.items # => ["Title One", "Title Two", "Title Three"]
collection.to_xml # => '<titles title="Title One; Title Two; Title Three"/>'
# Same result with delimiter option
delimiter_collection = TitleDelimiterCollection.from_xml(xml)
delimiter_collection.items # => ["Title One", "Title Two", "Title Three"]
delimiter_collection.to_xml # => '<titles title="Title One; Title Two; Title Three"/>'Collection types
A LutaML collections is used for a number of scenarios:
-
Root collections (for key-value formats)
-
Named collections
-
Keyed collections (for key-value formats)
-
Attribute collections
-
Nested collections
Root collections (key-value formats only)
General
TODO: This case needs to be fixed for JSON.
A root collection is a collection that is not contained within a parent collection.
Root collections only apply to key-value serialization formats. The XML format does not support root collections.
|
Note
|
The XML standard mandates the existence of a non-repeated "root element" in an XML document. This means that a valid XML document must have a root element, and all elements in an XML document must exist within the root. This is why an XML document cannot be a "root collection". |
|
Note
|
A root collection cannot be represented using a non-collection model. |
Root collections store multiple instances of the same model type at the root level. In other words, these are model instances that do not have a defined container at the level of the LutaML Model.
There are two kinds of root collections depending on the type of the instance value:
- "Root value collection"
-
the value is a "primitive type"
- "Root object collection"
-
the value is a "model instance"
Regardless of the type of root collection, the instance in a collection is always a LutaML model instance.
Root value collections
A root value collection is a collection that directly contains values of a primitive type.
---
- Item One
- Item Two
- Item Three[
"Item One",
"Item Two",
"Item Three"
]Syntax:
class MyCollection < Lutaml::Model::Collection
instances :items, ModelType
end
class ModelType < Lutaml::Model::Serializable
attribute :name, :string
endCode:
class Title < Lutaml::Model::Serializable
attribute :content, :string
end
class TitleCollection < Lutaml::Model::Collection
instances :titles, Title
key_value do
no_root # default
map_instances to: :titles
end
endData:
---
- Title One
- Title Two
- Title Three[
"Title One",
"Title Two",
"Title Three"
]Usage:
titles = TitleCollection.from_yaml(yaml_data)
titles.count
# => 3
titles.first.content
# => "Title One"Root object collections
A root object collection is a collection that directly contains model instances, each containing at least one serialized attribute.
name
---
- name: Item One
- name: Item Two
- name: Item Three[
{"name": "Item One"},
{"name": "Item Two"},
{"name": "Item Three"}
]Code:
class Title < Lutaml::Model::Serializable
attribute :content, :string
end
class TitleCollection < Lutaml::Model::Collection
instances :titles, Title
key_value do
no_root # default
map_instances to: :titles
end
endData:
---
- content: Title One
- content: Title Two
- content: Title Three[
{"content": "Title One"},
{"content": "Title Two"},
{"content": "Title Three"}
]Usage:
titles = TitleCollection.from_yaml(yaml_data)
titles.count
# => 3
titles.first.content
# => "Title One"Named collections
General
Named collections are collections wrapped inside a name or a key. The "name" of the collection serves as the container root of its contained model instances.
The named collection setup applies to XML and key-value serialization formats.
In a named collection setup, the collection is defined as a Lutaml::Model::Collection class, and each instance is defined as a Lutaml::Model::Serializable class.
There are two kinds of named collections depending on the type of the instance value:
- "Named value collection"
-
the value is a "primitive type"
- "Named object collection"
-
the value is a "model instance"
Regardless of the name of root collection, the instance in a collection is always a LutaML model instance.
Named value collections
A named value collection is a collection that contains values of a primitive type.
<names>
<name>Item One</name>
<name>Item Two</name>
<name>Item Three</name>
</names>---
names:
- Item One
- Item Two
- Item ThreeSyntax:
class MyCollection < Lutaml::Model::Collection
instances :items, ModelType
xml do
root "name-of-xml-container-element"
end
key_value do
root "name-of-key-value-container-element"
end
end
class ModelType < Lutaml::Model::Serializable
attribute :name, :string
endA named collection can alternatively be implemented as a non-collection model ("Model class with an attribute") that contains the collection of instances. In this case, the attribute will be an Array object, which does not contain additional attributes and methods.
class Title < Lutaml::Model::Serializable
attribute :title, :string
xml do
root "title"
map_content to: :title
end
end
class DirectTitleCollection < Lutaml::Model::Collection
instances :items, Title
xml do
root "titles"
map_element "title", to: :items
end
key_value do
map_instances to: :items
end
end<titles>
<title>Title One</title>
<title>Title Two</title>
<title>Title Three</title>
</titles>---
titles:
- Title One
- Title Two
- Title Three{
"titles": [
"Title One",
"Title Two",
"Title Three"
]
}titles = DirectTitleCollection.from_yaml(yaml_data)
titles.count
# => 3
titles.first.title
# => "Title One"
titles.last.title
# => "Title Three"Named object collections
A named object collection is a collection that contains model instances, each containing at least one serialized attribute.
|
Note
|
A named object collection can alternatively be implemented as a non-collection model ("Model class with an attribute") that contains the collection of instances. In this case, the attribute will be an Array object, which does not contain additional attributes and methods. |
<names>
<name><content>Item One</content></name>
<name><content>Item Two</content></name>
<name><content>Item Three</content></name>
</names>---
names:
- name: Item One
- name: Item Two
- name: Item ThreeData:
<titles>
<title><content>Title One</content></title>
<title><content>Title Two</content></title>
<title><content>Title Three</content></title>
</titles>---
titles:
- title: Title One
- title: Title Two
- title: Title Three{
"titles": [
{"title": "Title One"},
{"title": "Title Two"},
{"title": "Title Three"}
]
}Code:
class Title < Lutaml::Model::Serializable
attribute :title, :string
xml do
root "title"
map_element "content", to: :title
end
key_value do
map "title", to: :title
end
end
class TitleCollection < Lutaml::Model::Collection
instances :items, Title
xml do
root "titles"
map_element 'title', to: :items
end
key_value do
root "titles"
map_instances to: :items
end
endUsage:
titles = TitleCollection.from_yaml(yaml_data)
titles.count
# => 3
titles.first.title
# => "Title One"
titles.last.title
# => "Title Three"Attribute collection class
A model attribute that is a collection can be contained within a custom collection class.
A custom collection class can be defined to provide custom behavior for the
collection inside a non-collection model, with attributes using
collection: true.
Syntax:
class MyModel < Lutaml::Model::Serializable
attribute {model-attribute}, ModelType, collection: MyCollection
end
class MyCollection < Lutaml::Model::Collection
instances {instance-name}, ModelType
# Custom behavior for the collection
def custom_method
# Custom logic here
end
endData:
<titles>
<title>Title One</title>
<title>Title Two</title>
<title>Title Three</title>
</titles>titles:
- title: Title One
- title: Title Two
- title: Title Threeclass StringParts < Lutaml::Model::Collection
instances :parts, :string
def to_s
parts.join(' -- ')
end
end
class BibliographicItem < Lutaml::Model::Serializable
attribute :title_parts, :string, collection: StringParts
xml do
root "titles"
map_element "title", to: :title_parts
end
key_value do
root "titles"
map_instances to: :title_parts
end
def render_title
title_parts.to_s
end
end> bib_item = BibliographicItem.from_xml(xml_data)
> bib_item.title_parts
> # StringParts:0x0000000104ac7240 @parts=["Title One", "Title Two", "Title Three"]
> bib_item.render_title
> # "Title One -- Title Two -- Title Three"Nested collections
TODO: This case needs to be fixed.
Collections can be nested within other models and define their own serialization rules.
Nested collections can be defined in the same way as root collections, but they are defined within the context of a parent model.
Data:
<titles>
<title-group>
<artifact>
<content>Title One</content>
</artifact>
<artifact>
<content>Title Two</content>
</artifact>
<artifact>
<content>Title Three</content>
</artifact>
</title-group>
</titles>---
titles:
title-group:
- artifact:
content: Title One
- artifact:
content: Title Two
- artifact:
content: Title Threeclass Title < Lutaml::Model::Serializable
attribute :content, :string
end
class TitleCollection < Lutaml::Model::Collection
instances :items, Title
xml do
root "title-group"
map_element "artifact", to: :items
end
end
class BibItem < Lutaml::Model::Serializable
attribute :titles, TitleCollection
xml do
root "bibitem"
# This overrides the collection's root "title-group"
map_element "titles", to: :titles
end
endKeyed collections (key-value serialization formats only)
General
In key-value serialization formats, a key can be used to uniquely identify each instance. This usage allows for enforcing uniqueness in the collection.
A collection that contains keyed objects as its instances is commonly called a "keyed collection". A keyed object in a serialization format is an object identified with a unique key.
|
Note
|
The concept of keyed collections does not typically apply to XML collections. |
There are two kinds of keyed collections depending on the type of the keyed value:
- "keyed value collection"
-
the value is a "primitive type"
- "keyed object collection"
-
the value is a "model instance"
Regardless of the type of keyed collections, the instance in a collection is always a LutaML model instance.
map_key method
The map_key method specifies that the unique key is to be moved into an
attribute belonging to the instance model.
Syntax:
key_value do
map_key to_instance: {instance-attribute-name}
endWhere,
to_instance-
Refers to the attribute name in the instance that contains the key.
{key_attribute}-
The attribute name in the instance that contains the key.
map_value method
The map_value method specifies that the value (the object referenced by the
unique key) is to be moved into an attribute belonging to the instance model.
Syntax:
key_value do
# basic pattern
map_value {operation}: [*argument]
# Mapping the value object to a full instance through `to_instance`
map_value to_instance: {instance-attribute-name}
# Mapping the value object to an attribute as_instance
map_value as_attribute: {instance-attribute-name}
endKeyed value collections
A keyed value collection is a collection where the keyed item in the serialization format is a primitive type (e.g. string, integer, etc.).
The instance item inside the collection is a model instance that contains both the serialized key and serialized value both as attributes inside the model.
All three map_key, map_value, and map_instances methods need to be used to
define how instances are mapped in a keyed value collection.
class AuthorAvailability < Lutaml::Model::Serializable
attribute :id, :string
attribute :available, :boolean
end
class AuthorCollection < Lutaml::Model::Collection
instances :authors, AuthorAvailability
key_value do
map_key to_instance: :id # This refers to 'authors[].id'
map_value as_attribute: :available # This refers to 'authors[].available'
map_instances to: :authors
end
end---
author_01: true
author_02: false
author_03: trueauthors = AuthorCollection.from_yaml(yaml_data)
authors.first.id
# => "author_01"
authors.first.available
# => trueKeyed object collections
A keyed object collection is a collection where the keyed item in the serialization format contains multiple attributes.
The instance item inside the collection is a model instance that contains the serialized key as one attribute, and the serialized value attributes are all attributes inside the model.
Both the map_key and map_instances are used to define how instances are
mapped in a keyed object collection.
class Author < Lutaml::Model::Serializable
attribute :id, :string
attribute :name, :string
end
class AuthorCollection < Lutaml::Model::Collection
instances :authors, Author
key_value do
map_key to_instance: :id # This refers to 'authors[].id'
map_instances to: :authors
end
end---
author_01:
name: Author One
author_02:
name: Author Two
author_03:
name: Author Threeauthors = AuthorCollection.from_yaml(yaml_data)
authors.first.id
# => "author_01"
authors.first.name
# => "Author One"Nested keyed object collection
A nested keyed object collection is a keyed collection that contain other keyed collections. This case is simply a more complex arrangement of the principles applied to keyed object collections.
This pattern can extend to multiple levels of nesting, where each level contains a keyed object collection that can have its own key and value mappings.
Depends on whether a custom collection class is needed, the following mechanisms are available:
-
When using a Lutaml::Model::Serializable class for a keyed collection, use the
child_mappingsoption to map attributes. -
When using a Lutaml::Model::Collection class for a keyed collection, there are two options:
-
use the
map_key,map_value, andmap_instancesmethods to map attributes; or -
use the
root_mappingsoption to map attributes.
This example provides a two-layer nested structure where:
-
The first layer keys pieces by type (
bowls,vases). -
The second layer keys glazes by finish name within each piece type.
-
Each glaze finish contains detailed attributes like temperature.
# Third layer represents glaze finishes.
class GlazeFinish < Lutaml::Model::Serializable
attribute :name, :string
attribute :temperature, :integer
key_value do
map "name", to: :name
map "temperature", to: :temperature
end
end
# Second layer represents ceramic pieces each with multiple finishes.
class CeramicPiece < Lutaml::Model::Serializable
attribute :piece_type, :string
attribute :glazes, GlazeFinish, collection: true
key_value do
map "piece_type", to: :piece_type
map "glazes", to: :glazes, child_mappings: {
name: :key,
temperature: :temperature
}
end
end
# Uppermost layer represents the collection of ceramic pieces.
class StudioInventory < Lutaml::Model::Collection
instances :pieces, CeramicPiece
key_value do
map to: :pieces, root_mappings: {
piece_type: :key,
glazes: :value,
}
end
end---
bowls:
matte_finish:
name: Earth Matte
temperature: 1240
glossy_finish:
name: Ocean Blue
temperature: 1260
crackle_finish:
name: Antique Crackle
temperature: 1220
vases:
metallic_finish:
name: Bronze Metallic
temperature: 1280
crystalline_finish:
name: Ice Crystal
temperature: 1300inventory = StudioInventory.from_yaml(yaml_data)
# Access nested data through the hierarchy
puts inventory.pieces.bowls.matte_finish.name
# => "Earth Matte"
puts inventory.pieces.bowls.matte_finish.temperature
# => 1240
# Iterate through all pieces and their glazes
inventory.pieces.each do |piece_type, piece|
puts "#{piece_type.capitalize}:"
piece.glazes.each do |glaze_name, glaze|
puts " #{glaze_name}: #{glaze.name} (#{glaze.temperature}°C)"
end
endBehavior
Enumerable interface
Collections implement the Ruby Enumerable interface, providing standard
collection operations.
Collections allow the following sample Enumerable methods:
-
each- Iterate over collection items -
map- Transform collection items -
select- Filter collection items -
find- Find items matching criteria -
reduce- Aggregate collection items
# Filter items
filtered = collection.filter { |item| item.id == "1" }
# Reject items
rejected = collection.reject { |item| item.id == "1" }
# Select items
selected = collection.select { |item| item.id == "1" }
# Map items
mapped = collection.map { |item| item.name }
# Count items
count = collection.countInitialization
Collections can be initialized with an array of items or through individual item addition.
# Empty collection
collection = ItemCollection.new
# From an array of items
collection = ItemCollection.new([item1, item2, item3])
# From an array of hashes
collection = ItemCollection.new([
{ id: "1", name: "Item 1" },
{ id: "2", name: "Item 2" }
])
# Adding items later
collection << Item.new(id: "3", name: "Item 3")Ordering
TODO: This case needs to be fixed.
Collections that maintain a specific ordering of elements.
Syntax:
class MyCollection < Lutaml::Model::Collection
instances {instances-name}, ModelType
ordered by: {attribute-of-instance-or-proc}, order: {:asc | :desc}
endWhere,
{instances-name}-
name of the instances accessor within the collection
ModelType-
The model type of the collection elements.
{attribute-of-instance-or-proc}-
How model instances are to be ordered by. Values supported are:
{attribute-of-instance}-
Attribute name of an instance to be ordered by.
{proc}-
Proc that returns a value to order by (same as
sort_by), given the instance as input. order-
Order direction of the value:
:asc-
Ascending order (default).
:desc-
Descending order.
|
Note
|
When a proc is provided for ordering and order: :desc is specified, the collection is first sorted using the proc (as with Ruby’s sort_by), and the resulting array is then reversed to achieve descending order.
|
Data:
<items>
<item id="3" name="Item Three"/>
<item id="1" name="Item One"/>
<item id="2" name="Item Two"/>
</items>---
- id: 3
name: Item Three
- id: 1
name: Item One
- id: 2
name: Item Twoclass Item < Lutaml::Model::Serializable
attribute :id, :string
attribute :name, :string
xml do
map_attribute "id", to: :id
map_attribute "name", to: :name
end
end
class OrderedItemCollection < Lutaml::Model::Collection
instances :items, Item
ordered by: :id, order: :desc
xml do
root "items"
map_element "item", to: :items
end
key_value do
no_root
map_instances to: :items
end
end> collection = OrderedItemCollection.from_xml(xml_data)
> collection.map(&:id)
> # ["3", "2", "1"]
> collection = OrderedItemCollection.from_yaml(yaml_data)
> collection.map(&:id)
> # ["3", "2", "1"]class ProcOrderedItemCollection < Lutaml::Model::Collection
instances :items, Item
# Multi-level ordering: first by name length, then by name alphabetically
ordered by: ->(item) { [item.name.length, item.name] }, order: :asc
xml do
root "items"
map_element "item", to: :items
end
end> items_data = [
> { id: "1", name: "Zebra" },
> { id: "2", name: "Alpha" },
> { id: "3", name: "Beta" }
> ]
> complex_collection = ProcOrderedItemCollection.new(items_data)
> complex_collection.map(&:name)
> # ["Beta", "Alpha", "Zebra"] # Sorted by name length then alphabeticallyPolymorphic collections
Collections can contain instances of different model classes that share a common base class.
The polymorphic option for instances allows you to specify which subclasses are accepted:
class ReferenceSet < Lutaml::Model::Collection
# Accepts any subclass of Reference
instances :references, Reference, polymorphic: true
end
class ReferenceSet < Lutaml::Model::Collection
# Accepts only DocumentReference and AnchorReference
instances :references, Reference, polymorphic: [
DocumentReference,
AnchorReference,
]
endPolymorphic collection mapping
To serialize/deserialize polymorphic collections, use the polymorphic option in your mapping blocks. This allows you to specify how to differentiate subclasses in XML, YAML, or JSON.
class ReferenceSet < Lutaml::Model::Collection
instances :references, Reference, polymorphic: [
DocumentReference,
AnchorReference,
]
xml do
root "ReferenceSet"
map_instances to: :references, polymorphic: {
attribute: "_class",
class_map: {
"document-ref" => "DocumentReference",
"anchor-ref" => "AnchorReference",
},
}
end
key_value do
map_instances to: :references, polymorphic: {
attribute: "_class",
class_map: {
"Document" => "DocumentReference",
"Anchor" => "AnchorReference",
},
}
end
endThis allows round-trip serialization like:
---
references:
- _class: Document
name: The Tibetan Book of the Dead
document_id: book:tbtd
- _class: Anchor
name: Chapter 1
anchor_id: book:tbtd:anchor-1<ReferenceSet>
<references reference-type="document-ref">
<name>The Tibetan Book of the Dead</name>
<document_id>book:tbtd</document_id>
</references>
<references reference-type="anchor-ref">
<name>Chapter 1</name>
<anchor_id>book:tbtd:anchor-1</anchor_id>
</references>
</ReferenceSet>Polymorphic mapping with subclass differentiator
If you cannot modify the polymorphic superclass, define the differentiator attribute in each subclass, and use the polymorphic mapping option in the collection:
class Reference < Lutaml::Model::Serializable
attribute :name, :string
end
class DocumentReference < Reference
attribute :_class, :string
attribute :document_id, :string
xml do
map_element "document_id", to: :document_id
map_attribute "reference-type", to: :_class
end
key_value do
map "document_id", to: :document_id
map "_class", to: :_class
end
end
class AnchorReference < Reference
attribute :_class, :string
attribute :anchor_id, :string
xml do
map_element "anchor_id", to: :anchor_id
map_attribute "reference-type", to: :_class
end
key_value do
map "anchor_id", to: :anchor_id
map "_class", to: :_class
end
end
class ReferenceSet < Lutaml::Model::Collection
instances :references, Reference, polymorphic: [
DocumentReference,
AnchorReference,
]
xml do
root "ReferenceSet"
map_instances to: :references, polymorphic: {
attribute: "_class",
class_map: {
"document-ref" => "DocumentReference",
"anchor-ref" => "AnchorReference",
},
}
end
key_value do
map_instances to: :references, polymorphic: {
attribute: "_class",
class_map: {
"Document" => "DocumentReference",
"Anchor" => "AnchorReference",
},
}
end
endSerialization model mappings
General
Lutaml::Model allows you to translate a data model into serialization models of various serialization formats.
Depending on the serialization format, different methods are supported for defining serialization and deserialization mappings.
A serialization model mapping is defined using a format-specific DSL block in this syntax:
class Example < Lutaml::Model::Serializable
{format-short-name} do (1)
# ...
end
end-
{format-short-name}is the serialization format short name.
There are two kinds of serialization models:
-
Represents a singular model (maps to a Lutaml::Model::Serializable)
-
Represents a group/collection of models (maps to Lutaml::Model::Collection)
A collection contains instances of singular models, and therefore is always inextricably linked to an underlying serialization format for singular models. For instance, JSONL represents a collection (itself being invalid JSON) that uses JSON for singular models.
The supported serialization formats and their short names are defined as follows:
- Model serialization formats
-
xml-
XML
hsh-
Hash
NoteYes a 3-letter abbreviation for Hash! json-
JSON
yaml-
YAML
toml-
TOML
key_value-
Key-value format, a shorthand for all key-value formats (including JSON, YAML and TOML).
- Collection serialization formats
-
jsonl-
JSONL (JSON Lines)
yamls-
YAML Stream (multi-document format)
xml, hsh, json, yaml, toml and key_value blocks to define serialization mappingsclass Example < Lutaml::Model::Serializable
xml do
# ...
end
hsh do
# ...
end
json do
# ...
end
yaml do
# ...
end
toml do
# ...
end
key_value do
# ...
end
endjsonl block to define serialization mappings to a collectionclass Example < Lutaml::Model::Collection
jsonl do
# ...
end
endXML
Setting root element name
The root method sets the root element tag name of the XML document.
If root is not given, then the snake-cased class name will be used as the
root.
<example> in XML <example>…</example>.
Syntax:
xml do
root 'xml_element_name'
endexample
class Example < Lutaml::Model::Serializable
xml do
root 'example'
end
end> Example.new.to_xml
> #<example></example>Ommiting root element
The root element can be omitted by using the no_root method.
When no_root is used, only map_element can be used because without a root
element there cannot be attributes.
Syntax:
xml do
no_root
endclass NameAndCode < Lutaml::Model::Serializable
attribute :name, :string
attribute :code, :string
xml do
no_root
map_element "code", to: :code
map_element "name", to: :name
end
end<name>Name</name>
<code>ID-001</code>> parsed = NameAndCode.from_xml(xml)
> # <NameAndCode:0x0000000107a3ca70 @code="ID-001", @name="Name">
> parsed.to_xml
> # <code>ID-001</code><name>Name</name>Mapping all XML content
The map_all tag in XML mapping captures and maps all content within an XML
element into a single attribute in the target Ruby object.
The use case for map_all is to tell Lutaml::Model to not parse the content of
the XML element at all, and instead handle it as an XML string.
|
Note
|
The corresponding method for key-value formats is at Mapping all key-value content. |
|
Warning
|
Notice that usage of mapping all will lead to incompatibility between serialization formats, i.e. the raw string content will not be portable as objects are across different formats. |
This is useful in the case where the content of an XML element is not to be handled by a Lutaml::Model::Serializable object.
This feature is commonly used with custom methods or a custom model object to handle the content.
This includes:
-
nested tags
-
attributes
-
text nodes
The map_all tag is exclusive and cannot be combined with other mappings
(map_element, map_content) except for map_attribute for the same element,
ensuring it captures the entire inner XML content.
|
Note
|
An error is raised if map_all is defined alongside any other mapping in
the same XML mapping context.
|
Syntax:
xml do
map_all to: :name_of_attribute
endmap_all
class ExampleMapping < Lutaml::Model::Serializable
attribute :description, :string
xml do
map_all to: :description
end
end<ExampleMapping>Content with <b>tags</b> and <i>formatting</i>.</ExampleMapping>> parsed = ExampleMapping.from_xml(xml)
> puts parsed.all_content
# "Content with <b>tags</b> and <i>formatting</i>."Mapping elements
The map_element method maps an XML element to a data model attribute.
<name> tag in <example><name>John Doe</name></example>.
The value will be set to John Doe.
Syntax:
xml do
map_element 'xml_element_name', to: :name_of_attribute
endname tag to the name attributeclass Example < Lutaml::Model::Serializable
attribute :name, :string
xml do
root 'example'
map_element 'name', to: :name
end
end<example><name>John Doe</name></example>> Example.from_xml(xml)
> #<Example:0x0000000104ac7240 @name="John Doe">
> Example.new(name: "John Doe").to_xml
> #<example><name>John Doe</name></example>If an element is mapped to a model object with the XML root tag name set, the
mapped tag name will be used as the root name, overriding the root name.
class RecordDate < Lutaml::Model::Serializable
attribute :content, :string
xml do
root "recordDate"
map_content to: :content
end
end
class OriginInfo < Lutaml::Model::Serializable
attribute :date_issued, RecordDate, collection: true
xml do
root "originInfo"
map_element "dateIssued", to: :date_issued
end
end> RecordDate.new(date: "2021-01-01").to_xml
> #<recordDate>2021-01-01</recordDate>
> OriginInfo.new(date_issued: [RecordDate.new(date: "2021-01-01")]).to_xml
> #<originInfo><dateIssued>2021-01-01</dateIssued></originInfo>Mapping attributes
The map_attribute method maps an XML attribute to a data model attribute.
Syntax:
xml do
map_attribute 'xml_attribute_name', to: :name_of_attribute
endmap_attribute to map the value attributeThe following class will parse the XML snippet below:
class Example < Lutaml::Model::Serializable
attribute :value, :integer
xml do
root 'example'
map_attribute 'value', to: :value
end
end<example value="12"><name>John Doe</name></example>> Example.from_xml(xml)
> #<Example:0x0000000104ac7240 @value=12>
> Example.new(value: 12).to_xml
> #<example value="12"></example>The map_attribute method does not inherit the root element’s namespace.
To specify a namespace for an attribute, please explicitly declare the
namespace and prefix in the map_attribute method.
The following class will parse the XML snippet below:
class Attribute < Lutaml::Model::Serializable
attribute :value, :integer
xml do
root 'example'
map_attribute 'value', to: :value, namespace: "http://www.tech.co/XMI", prefix: "xl"
end
end<example xl:value="20" xmlns:xl="http://www.tech.co/XMI"></example>> Attribute.from_xml(xml)
> #<Attribute:0x0000000109436db8 @value=20>
> Attribute.new(value: 20).to_xml
> #<example xmlns:xl=\"http://www.tech.co/XMI\" xl:value=\"20\"/>Mapping content
Content represents the text inside an XML element, inclusive of whitespace.
The map_content method maps an XML element’s content to a data model
attribute.
Syntax:
xml do
map_content to: :name_of_attribute
endmap_content to map content of the description tagThe following class will parse the XML snippet below:
class Example < Lutaml::Model::Serializable
attribute :description, :string
xml do
root 'example'
map_content to: :description
end
end<example>John Doe is my moniker.</example>> Example.from_xml(xml)
> #<Example:0x0000000104ac7240 @description="John Doe is my moniker.">
> Example.new(description: "John Doe is my moniker.").to_xml
> #<example>John Doe is my moniker.</example>CDATA nodes
CDATA is an XML feature that allows the inclusion of text that may contain characters that are unescaped in XML.
While CDATA is not preferred in XML, it is sometimes necessary to handle CDATA nodes for both input and output.
|
Note
|
The W3C XML Recommendation explicitly encourages escaping characters over usage of CDATA. |
Lutaml::Model supports the handling of CDATA nodes in XML in the following behavior:
-
When an attribute contains a CDATA node with no text:
-
On reading: The node (CDATA or text) is read as its value.
-
On writing: The value is written as its native type.
-
-
When an XML mapping sets
cdata: trueonmap_elementormap_content:-
On reading: The node (CDATA or text) is read as its value.
-
On writing: The value is written as a CDATA node.
-
-
When an XML mapping sets
cdata: falseonmap_elementormap_content:-
On reading: The node (CDATA or text) is read as its value.
-
On writing: The value is written as a text node (string).
-
Syntax:
xml do
map_content to: :name_of_attribute, cdata: (true | false)
map_element :name, to: :name, cdata: (true | false)
endcdata to map CDATA contentThe following class will parse the XML snippet below:
class Example < Lutaml::Model::Serializable
attribute :name, :string
attribute :description, :string
attribute :title, :string
attribute :note, :string
xml do
root 'example'
map_element :name, to: :name, cdata: true
map_content to: :description, cdata: true
map_element :title, to: :title, cdata: false
map_element :note, to: :note, cdata: false
end
end<example><name><![CDATA[John]]></name><![CDATA[here is the description]]><title><![CDATA[Lutaml]]></title><note>Careful</note></example>> Example.from_xml(xml)
> #<Example:0x0000000104ac7240 @name="John" @description="here is the description" @title="Lutaml" @note="Careful">
> Example.new(name: "John", description: "here is the description", title: "Lutaml", note: "Careful").to_xml
> #<example><name><![CDATA[John]]></name><![CDATA[here is the description]]><title>Lutaml</title><note>Careful</note></example>Example for mapping
The following class will parse the XML snippet below:
class Ceramic < Lutaml::Model::Serializable
attribute :name, :string
attribute :description, :string
attribute :temperature, :integer
xml do
root 'ceramic'
map_element 'name', to: :name
map_attribute 'temperature', to: :temperature
map_content to: :description
end
end<ceramic temperature="1200"><name>Porcelain Vase</name> with celadon glaze.</ceramic>> Ceramic.from_xml(xml)
> #<Ceramic:0x0000000104ac7240 @name="Porcelain Vase", @description=" with celadon glaze.", @temperature=1200>
> Ceramic.new(name: "Porcelain Vase", description: " with celadon glaze.", temperature: 1200).to_xml
> #<ceramic temperature="1200"><name>Porcelain Vase</name> with celadon glaze.</ceramic>Namespaces
Namespace at root
The namespace method in the xml block sets the namespace for the root
element.
Syntax:
xml do
namespace 'http://example.com/namespace'
endxml do
namespace 'http://example.com/namespace', 'prefix'
endnamespace method to set the namespace for the root elementclass Ceramic < Lutaml::Model::Serializable
attribute :type, :string
attribute :glaze, :string
xml do
root 'Ceramic'
namespace 'http://example.com/ceramic'
map_element 'Type', to: :type
map_element 'Glaze', to: :glaze
end
end<Ceramic xmlns='http://example.com/ceramic'><Type>Porcelain</Type><Glaze>Clear</Glaze></Ceramic>> Ceramic.from_xml(xml_file)
> #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze="Clear">
> Ceramic.new(type: "Porcelain", glaze: "Clear").to_xml
> #<Ceramic xmlns="http://example.com/ceramic"><Type>Porcelain</Type><Glaze>Clear</Glaze></Ceramic>namespace method to set a prefixed namespace for the root elementclass Ceramic < Lutaml::Model::Serializable
attribute :type, :string
attribute :glaze, :string
xml do
root 'Ceramic'
namespace 'http://example.com/ceramic', 'cer'
map_element 'Type', to: :type
map_element 'Glaze', to: :glaze
end
end<cer:Ceramic xmlns='http://example.com/ceramic'><cer:Type>Porcelain</cer:Type><cer:Glaze>Clear</cer:Glaze></cer:Ceramic>> Ceramic.from_xml(xml_file)
> #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze="Clear">
> Ceramic.new(type: "Porcelain", glaze: "Clear").to_xml
> #<cer:Ceramic xmlns="http://example.com/ceramic"><cer:Type>Porcelain</cer:Type><cer:Glaze>Clear</cer:Glaze></cer:Ceramic>Namespace on attribute
If the namespace is defined on a model attribute that already has a namespace, the mapped namespace will be given priority over the one defined in the class.
Syntax:
xml do
map_element 'xml_element_name', to: :name_of_attribute,
namespace: 'http://example.com/namespace',
prefix: 'prefix'
endnamespace-
The XML namespace used by this element
prefix-
The XML namespace prefix used by this element (optional)
namespace option to set the namespace for an elementIn this example, glz will be used for Glaze if it is added inside the
Ceramic class, and glaze will be used otherwise.
class Ceramic < Lutaml::Model::Serializable
attribute :type, :string
attribute :glaze, Glaze
xml do
root 'Ceramic'
namespace 'http://example.com/ceramic'
map_element 'Type', to: :type
map_element 'Glaze', to: :glaze, namespace: 'http://example.com/glaze', prefix: "glz"
end
end
class Glaze < Lutaml::Model::Serializable
attribute :color, :string
attribute :temperature, :integer
xml do
root 'Glaze'
namespace 'http://example.com/old_glaze', 'glaze'
map_element 'color', to: :color
map_element 'temperature', to: :temperature
end
end<Ceramic xmlns='http://example.com/ceramic'>
<Type>Porcelain</Type>
<glz:Glaze xmlns='http://example.com/glaze'>
<color>Clear</color>
<temperature>1050</temperature>
</glz:Glaze>
</Ceramic>> # Using the original Glaze class namespace
> Glaze.new(color: "Clear", temperature: 1050).to_xml
> #<glaze:Glaze xmlns="http://example.com/old_glaze"><color>Clear</color><temperature>1050</temperature></glaze:Glaze>
> # Using the Ceramic class namespace for Glaze
> Ceramic.from_xml(xml_file)
> #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze=#<Glaze:0x0000000104ac7240 @color="Clear", @temperature=1050>>
> Ceramic.new(type: "Porcelain", glaze: Glaze.new(color: "Clear", temperature: 1050)).to_xml
> #<Ceramic xmlns="http://example.com/ceramic"><Type>Porcelain</Type><glz:Glaze xmlns="http://example.com/glaze"><color>Clear</color><temperature>1050</temperature></glz:Glaze></Ceramic>Namespace with inherit option
The inherit option is used at the element level to inherit the namespace from
the root element.
Syntax:
xml do
map_element 'xml_element_name', to: :name_of_attribute, namespace: :inherit
endinherit option to inherit the namespace from the root elementIn this example, the Type element will inherit the namespace from the root.
class Ceramic < Lutaml::Model::Serializable
attribute :type, :string
attribute :glaze, :string
attribute :color, :string
xml do
root 'Ceramic'
namespace 'http://example.com/ceramic', 'cera'
map_element 'Type', to: :type, namespace: :inherit
map_element 'Glaze', to: :glaze
map_attribute 'color', to: :color, namespace: 'http://example.com/color', prefix: 'clr'
end
end<cera:Ceramic
xmlns:cera='http://example.com/ceramic'
xmlns:clr='http://example.com/color'
clr:color="navy-blue">
<cera:Type>Porcelain</cera:Type>
<Glaze>Clear</Glaze>
</cera:Ceramic>> Ceramic.from_xml(xml_file)
> #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze="Clear", @color="navy-blue">
> Ceramic.new(type: "Porcelain", glaze: "Clear", color: "navy-blue").to_xml
> #<cera:Ceramic xmlns:cera="http://example.com/ceramic"
# xmlns:clr='http://example.com/color'
# clr:color="navy-blue">
# <cera:Type>Porcelain</cera:Type>
# <Glaze>Clear</Glaze>
# </cera:Ceramic>Mixed content
In XML there can be tags that contain content mixed with other tags and where whitespace is significant, such as to represent rich text.
<description><p>My name is <bold>John Doe</bold>, and I'm <i>28</i> years old</p></description>To map this to Lutaml::Model we can use the mixed option in either way:
-
when defining the model;
-
when referencing the model.
|
Note
|
This feature is not supported by Shale. |
To specify mixed content, the mixed: true option needs to be set at the
xml block’s root method.
Syntax:
xml do
root 'xml_element_name', mixed: true
endmixed to treat root as mixed contentclass Paragraph < Lutaml::Model::Serializable
attribute :bold, :string, collection: true # allows multiple bold tags
attribute :italic, :string
xml do
root 'p', mixed: true
map_element 'bold', to: :bold
map_element 'i', to: :italic
end
end> Paragraph.from_xml("<p>My name is <bold>John Doe</bold>, and I'm <i>28</i> years old</p>")
> #<Paragraph:0x0000000104ac7240 @bold="John Doe", @italic="28">
> Paragraph.new(bold: "John Doe", italic: "28").to_xml
> #<p>My name is <bold>John Doe</bold>, and I'm <i>28</i> years old</p>Ordered content
ordered: true maintains the order of XML Elements, while mixed: true preserves the order of XML Elements and Content.
|
Note
|
When both options are used, mixed: true takes precedence.
|
To specify ordered content, the ordered: true option needs to be set at the
xml block’s root method.
Syntax:
xml do
root 'xml_element_name', ordered: true
endordered to treat root as ordered contentclass RootOrderedContent < Lutaml::Model::Serializable
attribute :bold, :string
attribute :italic, :string
attribute :underline, :string
xml do
root "RootOrderedContent", ordered: true
map_element :bold, to: :bold
map_element :italic, to: :italic
map_element :underline, to: :underline
end
end<RootOrderedContent>
<underline>Moon</underline>
<italic>384,400 km</italic>
<bold>bell</bold>
</RootOrderedContent>> instance = RootOrderedContent.from_xml(xml)
> #<RootOrderedContent:0x0000000104ac7240 @bold="bell", @italic="384,400 km", @underline="Moon">
> instance.to_xml
> #<RootOrderedContent><underline>Moon</underline><italic>384,400 km</italic><bold>bell</bold></RootOrderedContent>Without Ordered True:
class RootOrderedContent < Lutaml::Model::Serializable
attribute :bold, :string
attribute :italic, :string
attribute :underline, :string
xml do
root "RootOrderedContent"
map_element :bold, to: :bold
map_element :italic, to: :italic
map_element :underline, to: :underline
end
end<RootOrderedContent>
<underline>Moon</underline>
<italic>384,400 km</italic>
<bold>bell</bold>
</RootOrderedContent>> instance = RootOrderedContent.from_xml(xml)
> #<RootOrderedContent:0x0000000104ac7240 @bold="bell", @italic="384,400 km", @underline="Moon">
> instance.to_xml
> #<RootOrderedContent>\n <bold>bell</bold>\n <italic>384,400 km</italic>\n <underline>Moon</underline>\n</RootOrderedContent>Sequence
The sequence directive specifies that the defined attributes must appear in a
specified order in XML.
|
Note
|
Sequence only supports map_element mappings.
|
Syntax:
xml do
sequence do
map_element 'xml_element_name_1', to: :name_of_attribute_1
map_element 'xml_element_name_2', to: :name_of_attribute_2
# Add more map_element lines as needed to establish a complete sequence
end
endThe appearance of the elements in the XML document must match the order defined
in the sequence block. In this case, the <xml_element_name_1> element
should appear before the <xml_element_name_2> element.
sequence keyword to define a set of elements in desired order.class Kiln < Lutaml::Model::Serializable
attribute :id, :string
attribute :name, :string
attribute :type, :string
attribute :color, :string
xml do
sequence do
map_element :id, to: :id
map_element :name, to: :name
map_element :type, to: :type
map_element :color, to: :color
end
end
end
class KilnCollection < Lutaml::Model::Serializable
attribute :kiln, Kiln, collection: 1..2
xml do
root "collection"
map_element "kiln", to: :kiln
end
end<collection>
<kiln>
<id>1</id>
<name>Nick</name>
<type>Hard</type>
<color>Black</color>
</kiln>
<kiln>
<id>2</id>
<name>John</name>
<type>Soft</type>
<color>White</color>
</kiln>
</collection>> parsed = Kiln.from_xml(xml)
# => [
#<Kiln:0x0000000104ac7240 @id="1", @name="Nick", @type="Hard", @color="Black">,
#<Kiln:0x0000000104ac7240 @id="2", @name="John", @type="Soft", @color="White">
#]
> bad_xml = <<~HERE
<collection>
<kiln>
<name>Nick</name>
<id>1</id>
<color>Black</color>
<type>Hard</type>
</kiln>
</collection>
HERE
> parsed = Kiln.from_xml(bad_xml)
# => Lutaml::Model::ValidationError: Element 'name' is out of order in 'kiln' element|
Note
|
For importing model mappings inside a sequence block, refer to Importing model mappings inside a sequence.
|
Automatic support of xsi:schemaLocation
The
W3C "XMLSchema-instance"
namespace describes a number of attributes that can be used to control the
behavior of XML processors. One of these attributes is xsi:schemaLocation.
The xsi:schemaLocation attribute locates schemas for elements and attributes
that are in a specified namespace. Its value consists of pairs of a namespace
URI followed by a relative or absolute URL where the schema for that namespace
can be found.
Usage of xsi:schemaLocation in an XML element depends on the declaration of
the XML namespace of xsi, i.e.
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance". Without this namespace
LutaML will not be able to serialize the xsi:schemaLocation attribute.
|
Note
|
It is most commonly attached to the root element but can appear further down the tree. |
The following snippet shows how xsi:schemaLocation is used in an XML document:
<cera:Ceramic
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:cera="http://example.com/ceramic"
xmlns:clr='http://example.com/color'
xsi:schemaLocation=
"http://example.com/ceramic http://example.com/ceramic.xsd
http://example.com/color http://example.com/color.xsd"
clr:color="navy-blue">
<cera:Type>Porcelain</cera:Type>
<Glaze>Clear</Glaze>
</cera:Ceramic>LutaML::Model supports the xsi:schemaLocation attribute in all XML
serializations by default, through the schema_location attribute on the model
instance object.
xsi:schemaLocation attribute in XML serializationIn this example, the xsi:schemaLocation attribute will be automatically
supplied without the explicit need to define in the model, and allows for
round-trip serialization.
class Ceramic < Lutaml::Model::Serializable
attribute :type, :string
attribute :glaze, :string
attribute :color, :string
xml do
root 'Ceramic'
namespace 'http://example.com/ceramic', 'cera'
map_element 'Type', to: :type, namespace: :inherit
map_element 'Glaze', to: :glaze
map_attribute 'color', to: :color, namespace: 'http://example.com/color', prefix: 'clr'
end
end
xml_content = <<~HERE
<cera:Ceramic
xmlns:cera="http://example.com/ceramic"
xmlns:clr="http://example.com/color"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
clr:color="navy-blue"
xsi:schemaLocation="
http://example.com/ceramic http://example.com/ceramic.xsd
http://example.com/color http://example.com/color.xsd
">
<cera:Type>Porcelain</cera:Type>
<Glaze>Clear</Glaze>
</cera:Ceramic>
HERE> c = Ceramic.from_xml(xml_content)
=>
#<Ceramic:0x00000001222bdd60
...
> schema_loc = c.schema_location
#<Lutaml::Model::SchemaLocation:0x0000000122773760
...
> schema_loc
=>
#<Lutaml::Model::SchemaLocation:0x0000000122773760
@namespace="http://www.w3.org/2001/XMLSchema-instance",
@original_schema_location="http://example.com/ceramic http://example.com/ceramic.xsd http://example.com/color http://example.com/color.xsd",
@prefix="xsi",
@schema_location=
[#<Lutaml::Model::Location:0x00000001222bd018 @location="http://example.com/ceramic.xsd", @namespace="http://example.com/ceramic">,
#<Lutaml::Model::Location:0x00000001222bcfc8 @location="http://example.com/color.xsd", @namespace="http://example.com/color">]>
> new_c = Ceramic.new(type: "Porcelain", glaze: "Clear", color: "navy-blue", schema_location: schema_loc).to_xml
> puts new_c
# <cera:Ceramic
# xmlns:cera="http://example.com/ceramic"
# xmlns:clr="http://example.com/color"
# xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
# clr:color="navy-blue"
# xsi:schemaLocation="
# http://example.com/ceramic http://example.com/ceramic.xsd
# http://example.com/color http://example.com/color.xsd
# ">
# <cera:Type>Porcelain</cera:Type>
# <cera:Glaze>Clear</cera:Glaze>
# </cera:Ceramic>|
Note
|
For details on xsi:schemaLocation, please refer to the
W3C XML standard.
|
Character encoding
General
Lutaml::Model XML adapters use a default encoding of UTF-8 for both input and
output.
Serialization data to be parsed (deserialization) and serialization data to be exported (serialization) may be in a different character encoding than the default encoding used by the Lutaml::Model XML adapter. This mismatch may lead to incorrect data reading or incompatibilities when exporting data.
The possible values for setting character encoding to are:
-
A valid encoding value, e.g.
UTF-8,Shift_JIS,ASCII; -
nilto use the default encoding of the adapter. The behavior differs based on the adapter used.-
Nokogiri:
UTF-8. The encoding is set to the default encoding of the Nokogiri library, which isUTF-8. -
Oga:
UTF-8. The encoding is set to the default encoding of the Oga library, which usesUTF-8. -
Ox:
ASCII-8bit. The encoding is set to the default encoding of the Ox library, which usesASCII-8bit.
-
When the encoding option is not set, the default encoding of UTF-8 is
used.
Serialization character encoding (exporting)
General
There are two ways to set the character encoding of the XML document during serialization:
- Instance setting
-
Setting the instance-level
encodingoption by settingModelClassInstance.encoding('…'). This setting only affects serialization. - Per-export setting
-
Setting the
encodingoption when calling for serialization action using theModelClassInstance.to_xml(…, encoding: …)method.
Instance setting
The encoding value of an instance sets the character encoding of the XML
document during serialization.
Syntax:
ModelClassInstance.encoding = {encoding_value}Where,
ModelClassInstance-
An instance of the class that inherits from Lutaml::Model::Serializable.
{encoding_value}-
The encoding of the output data.
class JapaneseCeramic < Lutaml::Model::Serializable
attribute :glaze_type, :string
attribute :description, :string
xml do
root 'JapaneseCeramic'
map_attribute 'glazeType', to: :glaze_type
map_element 'description', to: :description
end
end# Create a new instance with UTF-8 data
> instance = JapaneseCeramic.new(glaze_type: "志野釉", description: "東京国立博物館コレクションの篠茶碗「橋本」(桃山時代)")
#=> #<JapaneseCeramic:0x0000000104ac7240 @glaze_type="志野釉", @description="東京国立博物館コレクションの篠茶碗「橋本」(桃山時代)">
# Set character encoding to Shift_JIS
> instance.encoding = "Shift_JIS"
#=> "Shift_JIS"
# Serialize the instance
> serialization_output = instance.to_xml
#=> #<JapaneseCeramic><glazeType>\x{5FD8}\x{91CE}\x{91C9}</glazeType><description>\x{6771}\x{4EAC}\x{56FD}\x{7ACB}\x{535A}\x{7269}\x{9928}\x{30B3}\x{30EC}\x{30AF}\x{30B7}\x{30E7}\x{30F3}\x{306E}\x{7BC0}\x{8336}\x{7897}\x{300C}\x{6A4B}\x{672C}\x{300D}\x{FF08}\x{6853}\x{5C71}\x{6642}\x{4EE3}\x{FF09}</description></JapaneseCeramic>
# Check character encoding of output
> serialization_output.encoding
#=> "Shift_JIS"Per-export setting
The encoding option is used in the ModelClass#to_xml(…, encoding: …)
call to set the character encoding of the XML document during serialization.
The per-export encoding setting supersedes the instance-level encoding setting.
Syntax:
ModelClassInstance.to_xml(encoding: {encoding_value})Where,
ModelClassInstance-
An instance of the class that inherits from Lutaml::Model::Serializable.
{encoding_value}-
The encoding of the output data.
The following class will parse the XML snippet below:
class Ceramic < Lutaml::Model::Serializable
attribute :potter, :string
attribute :description, :string
attribute :temperature, :integer
xml do
root 'ceramic'
map_element 'potter', to: :potter
map_content to: :description
end
end<ceramic><potter>John & Jane</potter> A ∑ series of ∏ porcelain µ vases.</ceramic># Object with attributes
> ceramic_instance = Ceramic.new(potter: "John & Jane", description: " A ∑ series of ∏ porcelain µ vases.")
> #<Ceramic:0x0000000104ac7240 @potter="John & Jane", @description=" A ∑ series of ∏ porcelain µ vases.">
# Parsing the XML snippet with the default encoding of UTF-8
> ceramic_parsed = Ceramic.from_xml(xml)
> #<Ceramic:0x0000000104ac7242 @potter="John & Jane", @description=" A ∑ series of ∏ porcelain µ vases.">
# Object with attributes is equal to the parsed object
> ceramic_parsed == ceramic_instance
> # true
# Using the default encoding of UTF-8
> ceramic_instance.to_xml
> #<ceramic><potter>John & Jane</potter> A ∑ series of ∏ porcelain µ vases.</ceramic>
# Using the default encoding of the adapter, which is UTF-8 in this case
> ceramic_instance.to_xml(encoding: nil)
> #<ceramic><potter>John & Jane</potter> A ∑ series of ∏ porcelain µ vases.</ceramic>
# Using ASCII encoding
> ceramic_instance.to_xml(encoding: "ASCII")
> #<ceramic><potter>John & Jane</potter> A ∑ series of ∏ porcelain µ vases.</ceramic>to_xml overrides instance encodingclass JapaneseCeramic < Lutaml::Model::Serializable
attribute :glaze_type, :string
attribute :description, :string
xml do
root 'JapaneseCeramic'
map_attribute 'glazeType', to: :glaze_type
map_element 'description', to: :description
end
end# Create a new instance with UTF-8 data
> instance = JapaneseCeramic.new(glaze_type: "志野釉", description: "東京国立博物館コレクションの篠茶碗「橋本」(桃山時代)")
#=> #<JapaneseCeramic:0x0000000104ac7240 @glaze_type="志野釉", @description="東京国立博物館コレクションの篠茶碗「橋本」(桃山時代)">
# Set character encoding to Shift_JIS
> instance.encoding = "Shift_JIS"
#=> "Shift_JIS"
# Serialize the instance
> serialization_output = instance.to_xml(encoding: "UTF-8")
#=> #<JapaneseCeramic><glazeType>志野釉</glazeType><description>東京国立博物館コレクションの篠茶碗「橋本」(桃山時代)</description></JapaneseCeramic>
# Check character encoding of output
> serialization_output.encoding
#=> "UTF-8"Skip Model attribute rendering (exporting)
In to_xml, the except option allows you to exclude certain model attributes from being included in the XML output.
except-
An array of attribute names.
class JapaneseCeramic < Lutaml::Model::Serializable
attribute :glaze_type, :string
attribute :description, :string
xml do
root 'JapaneseCeramic'
map_attribute 'glazeType', to: :glaze_type
map_element 'description', to: :description
end
end# Create a new instance
> instance = JapaneseCeramic.new(glaze_type: "Clear", description: "Porcelain")
#=> #<JapaneseCeramic:0x00000002e5625650 @description="Porcelain", @glaze_type="Clear">
# Serialize the instance to XML without glaze_type
> instance.to_xml(except: [:glaze_type])
#=> "<JapaneseCeramic>\n <description>Porcelain</description>\n</JapaneseCeramic>"
# Serialize the instance to XML without glaze_type and description
> instance.to_xml(except: [:glaze_type, :description])
#=> "<JapaneseCeramic/>"
# Serialize the instance to YAML without glaze_type
> instance.to_yaml(except: [:glaze_type])
#=> "---\ndescription: Porcelain\n"
# Serialize the instance to YAML without glaze_type and description
> instance.to_yaml(except: [:glaze_type, :description])
#=> "--- {}\n"Deserialization character encoding (parsing)
The character encoding of the XML document being parsed is specified using the
encoding option when the ModelClass.from_{format}(…) is called.
Syntax:
ModelClass.from_{format}(string_in_format, encoding: {encoding_value})Where,
ModelClass-
The class that inherits from Lutaml::Model::Serializable.
{format}-
The format of the input data, e.g.
xml,json,yaml,toml. string_in_format-
The input data in the specified format.
{encoding_value}-
The encoding of the input data.
encoding option during parsing data not encoded in the default encoding (UTF-8)Using the definition of JapaneseCeramic at Instance setting.
This XML snippet is in Shift-JIS.
<JapaneseCeramic>
<glazeType>\x{5FD8}\x{91CE}\x{91C9}</glazeType>
<description>\x{6771}\x{4EAC}\x{56FD}\x{7ACB}\x{535A}\x{7269}\x{9928}\x{30B3}\x{30EC}\x{30AF}\x{30B7}\x{30E7}\x{30F3}\x{306E}\x{7BC0}\x{8336}\x{7897}\x{300C}\x{6A4B}\x{672C}\x{300D}\x{FF08}\x{6853}\x{5C71}\x{6642}\x{4EE3}\x{FF09}</description>
</JapaneseCeramic># Parse the XML snippet with the encoding of Shift_JIS
> instance = JapaneseCeramic.from_xml(xml, encoding: "Shift_JIS")
#=> #<JapaneseCeramic:0x0000000104ac7240 @glaze_type="志野釉", @description="東京国立博物館コレクションの篠茶碗「橋本」(桃山時代)">
# Check character encoding of the instance
> instance.encoding
#=> "Shift_JIS"
# Serialize the instance using UTF-8
> serialization_output = instance.to_xml(encoding: "UTF-8")
#=> #<JapaneseCeramic><glazeType>志野釉</glazeType><description>東京国立博物館コレクションの篠茶碗「橋本」(桃山時代)</description></JapaneseCeramic>
> serialization_output.encoding
#=> "UTF-8"encoding option is not set, the default encoding of the adapter is usedUsing the definition of JapaneseCeramic at Instance setting.
This XML snippet is in UTF-8.
<JapaneseCeramic>
<glazeType>志野釉</glazeType>
<description>東京国立博物館コレクションの篠茶碗「橋本」(桃山時代)</description>
</JapaneseCeramic>In adapters that use a default encoding of UTF-8, the content is parsed
properly.
> instance = JapaneseCeramic.from_xml(xml, encoding: nil)
#=> #<JapaneseCeramic:0x0000000104ac7240 @glaze_type="志野釉", @description="東京国立博物館コレクションの篠茶碗「橋本」(桃山時代)">
> instance.encoding
#=> "UTF-8"
> serialization_output = instance.to_xml
#=> #<JapaneseCeramic><glazeType>志野釉</glazeType><description>東京国立博物館コレクションの篠茶碗「橋本」(桃山時代)</description></JapaneseCeramic>
> serialization_output.encoding
#=> "UTF-8"In adapters that use a default encoding of ASCII-8bit, the content becomes
malformed.
> instance = JapaneseCeramic.from_xml(xml, encoding: nil)
#=> #<JapaneseCeramic:0x0000000104ac7240 @glaze_type="�菑�", @description="�東京国立博物館コレクションの篠茶碗�橋本�桃山時代�">
> instance.encoding
#=> "ASCII-8bit"
> serialization_output = instance.to_xml
#=> #<JapaneseCeramic><glazeType>�菑�</glazeType><description>�東京国立博物館コレクションの篠茶碗�橋本�桃山時代�</description></JapaneseCeramic>
> serialization_output.encoding
#=> "ASCII-8bit"Using the definition of JapaneseCeramic at Instance setting.
This XML snippet is in UTF-8.
<JapaneseCeramic>
<glazeType>志野釉</glazeType>
<description>東京国立博物館コレクションの篠茶碗「橋本」(桃山時代)</description>
</JapaneseCeramic>> JapaneseCeramic.from_xml(xml, encoding: "Shift_JIS")
#=> #<JapaneseCeramic:0x0000000104ac7240 @glaze_type="�菑���p���P", @description="�東京国立博物館コレクションの篠茶碗�橋本�桃山時代�">Key value data models
General
Key-value data models share a similar structure where data is stored as key-value pairs.
Lutaml::Model works with these formats in a similar way.
Key-value data models supported are identified by their short name:
hsh-
Hash (Ruby
Hashclass) json-
JSON
yaml-
YAML
toml-
TOML
key_value-
A way to configure key-value mappings for all supported key-value data models.
Mapping
The map method is used to define key-value mappings.
Syntax:
{key_value_type_short} do (1)
map 'key_value_model_attribute_name', to: :name_of_attribute
end-
key_value_type_shortis the key-value data model’s short name.
json do
map :color, to: :color
map :desc, to: :description
endkey_value do
map :color, to: :color
map :desc, to: :description
endUnified mapping
The key_value method is a streamlined way to map all attributes for
serialization into key-value formats including Hash, JSON, YAML, and TOML.
If there is no definite differentiation between the key value formats, the
key_value method simplifies defining mappings and improves code readability.
map method to define the same mappings across all key-value formatsThis example shows how to define a key-value data model with the key_value
method which maps the same attributes across all key-value formats.
class CeramicModel < Lutaml::Model::Serializable
attribute :color, :string
attribute :glaze, :string
attribute :description, :string
key_value do # or any other key-value data model
map :color, to: :color
map :glz, to: :glaze
map :desc, to: :description
end
end{
"color": "Navy Blue",
"glz": "Clear",
"desc": "A ceramic with a navy blue color and clear glaze."
}color: Navy Blue
glz: Clear
desc: A ceramic with a navy blue color and clear glaze.> CeramicModel.from_json(json)
> #<CeramicModel:0x0000000104ac7240 @color="Navy Blue", @glaze="Clear", @description="A ceramic with a navy blue color and clear glaze.">
> CeramicModel.new(color: "Navy Blue", glaze: "Clear", description: "A ceramic with a navy blue color and clear glaze.").to_json
> #{"color"=>"Navy Blue", "glz"=>"Clear", "desc"=>"A ceramic with a navy blue color and clear glaze."}Specific format mappings
Specific key value formats can be mapping independently of other formats.
map method to define key-value mappings per formatclass Example < Lutaml::Model::Serializable
attribute :name, :string
attribute :value, :integer
hsh do
map 'name', to: :name
map 'value', to: :value
end
json do
map 'name', to: :name
map 'value', to: :value
end
yaml do
map 'name', to: :name
map 'value', to: :value
end
toml do
map 'name', to: :name
map 'value', to: :value
end
end{
"name": "John Doe",
"value": 28
}> Example.from_json(json)
> #<Example:0x0000000104ac7240 @name="John Doe", @value=28>
> Example.new(name: "John Doe", value: 28).to_json
> #{"name"=>"John Doe", "value"=>28}Mapping all key-value content
The map_all tag captures and maps all content within a serialization format
into a single attribute in the target Ruby object.
The use case for map_all is to tell Lutaml::Model to not parse the content at
all, and instead handle it as a raw string.
|
Note
|
The corresponding method for XML is at Mapping all XML content. |
|
Warning
|
Notice that usage of mapping all will lead to incompatibility between serialization formats, i.e. the raw string content will not be portable as objects are across different formats. |
This is useful when the content needs to be handled as-is without parsing into individual attributes.
The map_all tag is exclusive and cannot be combined with other mappings,
ensuring it captures the entire content.
|
Note
|
An error is raised if map_all is defined alongside any other mapping in
the same mapping context.
|
Syntax:
hsh | json | yaml | toml | key_value do
map_all to: :name_of_attribute
endmap_all to capture all content across different formatsclass Document < Lutaml::Model::Serializable
attribute :content, :string
hsh do
map_all to: :content
end
json do
map_all to: :content
end
yaml do
map_all to: :content
end
toml do
map_all to: :content
end
endFor JSON:
{
"sections": [
{ "title": "Introduction", "text": "Chapter 1" },
{ "title": "Conclusion", "text": "Final chapter" }
],
"metadata": {
"author": "John Doe",
"date": "2024-01-15"
}
}For YAML:
sections:
- title: Introduction
text: Chapter 1
- title: Conclusion
text: Final chapter
metadata:
author: John Doe
date: 2024-01-15The content is preserved exactly as provided:
> doc = Document.from_json(json_content)
> puts doc.content
> # "{\"sections\":[{\"title\":\"Introduction\",\"text\":\"Chapter 1\"},{\"title\":\"Conclusion\",\"text\":\"Final chapter\"}],\"metadata\":{\"author\":\"John Doe\",\"date\":\"2024-01-15\"}}"
> doc = Document.from_yaml(yaml_content)
> puts doc.content
> # "sections:\n - title: Introduction\n text: Chapter 1\n - title: Conclusion\n text: Final chapter\nmetadata:\n author: John Doe\n date: 2024-01-15\n"Nested attribute mappings
The map method can also be used to map nested key-value data models
by referring to a Lutaml::Model class as an attribute class.
class Glaze < Lutaml::Model::Serializable
attribute :color, :string
attribute :temperature, :integer
json do
map 'color', to: :color
map 'temperature', to: :temperature
end
end
class Ceramic < Lutaml::Model::Serializable
attribute :type, :string
attribute :glaze, Glaze
json do
map 'type', to: :type
map 'glaze', to: :glaze
end
end{
"type": "Porcelain",
"glaze": {
"color": "Clear",
"temperature": 1050
}
}> Ceramic.from_json(json)
> #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze=#<Glaze:0x0000000104ac7240 @color="Clear", @temperature=1050>>
> Ceramic.new(type: "Porcelain", glaze: Glaze.new(color: "Clear", temperature: 1050)).to_json
> #{"type"=>"Porcelain", "glaze"=>{"color"=>"Clear", "temperature"=>1050}}Collection with keyed elements (keyed collection)
General
|
Note
|
This feature is for key-value data model serialization and deserialization only. |
The map method with the root_mappings option is used for key-value data that
is keyed using an attribute value.
In other words, the key of a key-value pair in a collection is actually the value of an attribute that belongs to the value.
Simply put, the following two data structures are considered to have the same data:
id attribute---
vase1:
name: Imperial Vase
bowl2:
name: 18th Century Bowlid attribute value located inside each element---
- id: vase1
name: Imperial Vase
- id: bowl2
name: 18th Century BowlThere are key difference between these two data structures:
-
The keyed object (first data structure) ensures uniqueness of the
idattribute value across the collection, while the array (second data structure) does not. -
The value of the
idattribute in the first data structure exists outside of the formal structure of the data object, instead, it only exists at the collection level. On the other hand, the value exists inside the structure of the data object in the second data structure.
The map method with the root_mappings option, in practice, parses the first
data structure in the same way that you would access / manipulate the second
data structure, while retaining the serialization semantics of using an
attribute as key.
As a result, usage of lutaml-model across both types of collections are identical (except when serialized).
Syntax:
class SomeKeyedCollection < Lutaml::Model::Serializable
attribute :name_of_attribute, AttributeValueType, collection: true
hsh | json | yaml | toml | key_value do
map to: :name_of_attribute, (1)
root_mappings: { (2)
# `:key` is a reserved keyword
value_type_attribute_name_for_key: :key, (3)
# `:value` is a reserved keyword (and optional)
value_type_attribute_name_for_value: :value, (4)
# `[path name]` represents the path to access the value in the
# serialization data model to be assigned to
# `AttributeValueType.value_type_attribute_name_for_custom_type`
value_type_attribute_name_for_custom_type: [path name] (5)
}
end
end
class AttributeValueType < Lutaml::Model::Serializable
attribute :value_type_attribute_name_for_key, :string
attribute :value_type_attribute_name_for_value, :string
attribute :value_type_attribute_name_for_custom_type, CustomType
end-
The
mapoption indicates that this class represents the root of the serialization object being passed in. Thename_of_attributeis the name of the attribute that will hold the collection data. (Mandatory) -
The
root_mappingskeyword specifies what the collection key represents and and value for model. (Mandatory) -
The
keykeyword specifies the attribute name of the individual collection object type that represents its key used in the collection. (Mandatory) -
The
valuekeyword specifies the attribute name of the individual collection object type that represents its data used in the collection. (Optional, if not specified, the entire object is used as the value.) -
The
value_type_attribute_name_for_custom_typeis the name of the attribute inside the individual collection object (AttributeValueType) that will hold the value accessible in the serialization data model fetched at[path name].
The mapping syntax here is similar to that of Attribute extraction except
that the :key and :value keywords are allowed in addition to {path}.
There are 3 cases when working with a keyed collection:
-
Case 1: Only move the "key" into the collection object.
-
Case 2: Move the "key" into the collection object, override all other mappings. Maps
:keyand another attribute, then we override all the other mappings (clean slate) -
Case 3: Move the "key" into the collection object to an attribute, map the entire "value" to another attribute of the collection object.
Case 1: Only move the "key" into the collection object
In this case, the "key" of the keyed collection is moved into the collection object, and all other mappings are left as they are.
When the "key" is moved into the collection object, the following happens:
-
The "key" of the keyed collection maps to a particular attribute of the collection’s instance object.
-
The "value" of the keyed collection (with its various content) maps to the collection’s instance object following the collection’s instance object type’s default mappings.
The root_mappings option should only contain one mapping, and the mapping
must lead to the :key keyword.
Syntax:
class SomeKeyedCollection < Lutaml::Model::Serializable
attribute :name_of_attribute, AttributeValueType, collection: true
hsh | json | yaml | toml | key_value do
map to: :name_of_attribute,
root_mappings: {
value_type_attribute_name_for_key: :key, (1)
}
end
end
class AttributeValueType < Lutaml::Model::Serializable
attribute :value_type_attribute_name_for_key, :string
attribute :value_type_attribute_name_for_value, :string
attribute :value_type_attribute_name_for_custom_type, CustomType
end-
The
:keykeyword specifies that the "key" of the keyed collection maps to thevalue_type_attribute_name_for_keyattribute of the collection’s instance object (i.e.AttributeValueType).
map with root_mappings (only key) to map a keyed collection into individual modelsGiven this data:
---
vase1:
name: Imperial Vase
bowl2:
name: 18th Century BowlA model can be defined for this YAML as follows:
# This is a normal Lutaml::Model class
class Ceramic < Lutaml::Model::Serializable
attribute :ceramic_id, :string
attribute :ceramic_name, :string
key_value do
map 'id', to: :ceramic_id
map 'name', to: :ceramic_name
end
end
# This is Lutaml::Model class that represents the collection of Ceramic objects
class CeramicCollection < Lutaml::Model::Serializable
attribute :ceramics, Ceramic, collection: true
key_value do
map to: :ceramics, # All data goes to the `ceramics` attribute
root_mappings: {
# The key of an object in this collection is mapped to the ceramic_id
# attribute of the Ceramic object.
ceramic_id: :key # "key" is a reserved keyword
}
end
end# Parsing the YAML collection with dynamic data keys
> ceramic_collection = CeramicCollection.from_yaml(yaml)
> #<CeramicCollection:0x0000000104ac7240
@ceramics=
[#<Ceramic:0x0000000104ac6e30 @ceramic_id="vase1", @ceramic_name="Imperial Vase">,
#<Ceramic:0x0000000104ac58f0 @ceramic_id="bowl2", @ceramic_name="18th Century Bowl">]
# NOTE: When an individual Ceramic object is serialized, the `id` attribute is
# the original key in the incoming YAML data, and because there were no mappings defined along with the `:key`, everyting is mapped to the `Ceramic` object using the mappings defined in the `Ceramic` class.
> first_ceramic = ceramic_collection.ceramics.first
> puts first_ceramic.to_yaml
=>
# ---
# id: vase1
# name: Imperial Vase
# NOTE: When in a collection, the `ceramic_id` attribute is used to key the data,
# and it disappears from the individual object.
> puts ceramic_collection.to_yaml
=>
# ---
# vase1:
# name: Imperial Vase
# bowl2:
# name: 18th Century Bowl
# NOTE: When the collection is serialized, the `ceramic_id` attribute is used to
# key the data. This is defined through the `map` with `root_mappings` method in
# CeramicCollection.
> new_collection = CeramicCollection.new(ceramics: [
Ceramic.new(ceramic_id: "vase1", ceramic_name: "Imperial Vase"),
Ceramic.new(ceramic_id: "bowl2", ceramic_name: "18th Century Bowl")
])
> puts new_collection.to_yaml
=>
# ---
# vase1:
# name: Imperial Vase
# bowl2:
# name: 18th Century BowlCase 2: Mapping the key and complex values
In this use case, the "key" of the keyed collection is moved into the collection object, and all other mappings are overridden.
When more than one mapping rule exists in the root_mappings option, the
root_mappings option will override all other mappings in the collection object.
When the "key" is moved into the collection object, the following happens:
-
The "key" of the keyed collection maps to a particular attribute of the collection’s instance object.
-
The data of the "value" of the keyed collection have their own mappings overridden by the new mapping rules of the
root_mappingsoption.
The root_mappings option can contain more than one mapping, with one of
the mapping rules leading to the :key keyword.
Syntax:
class SomeKeyedCollection < Lutaml::Model::Serializable
attribute :name_of_attribute, AttributeValueType, collection: true
hsh | json | yaml | toml | key_value do
map to: :name_of_attribute,
root_mappings: {
value_type_attribute_name_for_key: :key, (1)
value_type_attribute_name_for_value_data_1: "serialization_format_name_1", (2)
value_type_attribute_name_for_value_data_2: "serialization_format_name_2",
value_type_attribute_name_for_value_data_3: ["path name", ...] (3)
# ...
}
end
end
class AttributeValueType < Lutaml::Model::Serializable
attribute :value_type_attribute_name_for_key, :string
attribute :value_type_attribute_name_for_value_data_1, :string
attribute :value_type_attribute_name_for_value_data_2, SomeType
attribute :value_type_attribute_name_for_value_data_3, MoreType
# ...
end-
The
:keykeyword specifies that the "key" of the keyed collection maps to thevalue_type_attribute_name_for_keyattribute of the collection’s instance object (i.e.AttributeValueType). -
The
serialization_format_name_1target specifies that theserialization_format_name_2key of the keyed collection value maps to thevalue_type_attribute_name_for_value_data_1attribute of the collection’s instance object. -
The
[path name]target specifies to fetch from[path name]in the serialization data model to be assigned to thevalue_type_attribute_name_for_value_data_3attribute of the collection’s instance object.
When the root_mappings mapping contains more than one mapping rule that is not
to :key or :value, the root_mappings mapping will override all other
mappings in the collection object. This means that unmapped attributes in
root_mappings will not be incorporated in the collection instance objects.
map with root_mappings (key and complex value) to map a keyed collection into individual models"vase1":
type: "vase"
details:
name: "Imperial Vase"
insignia: "Tang Tianbao"
urn:
primary: "urn:ceramic:vase:vase1"
"bowl2":
type: "bowl"
details:
name: "18th Century Bowl"
insignia: "Ming Wanli"
urn:
primary: "urn:ceramic:bowl:bowl2"A model can be defined for this YAML as follows:
# This is a normal Lutaml::Model class
class CeramicDetails < Lutaml::Model::Serializable
attribute :name, :string
attribute :insignia, :string
key_value do
map 'name', to: :name
map 'insignia', to: :insignia
end
end
# This is a normal Lutaml::Model class
class Ceramic < Lutaml::Model::Serializable
attribute :ceramic_id, :string
attribute :ceramic_type, :string
attribute :ceramic_details, CeramicDetails
attribute :ceramic_urn, :string
key_value do
map 'id', to: :ceramic_id
map 'type', to: :ceramic_type
map 'details', to: :ceramic_details
map 'urn', to: :ceramic_urn
end
end
# This is Lutaml::Model class that represents the collection of Ceramic objects
class CeramicCollection < Lutaml::Model::Serializable
attribute :ceramics, Ceramic, collection: true
key_value do
map to: :ceramics, # All data goes to the `ceramics` attribute
root_mappings: {
# The key of an object in this collection is mapped to the ceramic_id
# attribute of the Ceramic object.
# (e.g. `vase1`, `bowl2`)
ceramic_id: :key,
ceramic_type: :type,
ceramic_details: "details",
ceramic_urn: ["urn", "primary"]
}
end
endThe output becomes:
> ceramics_collection = CeramicCollection.from_yaml(yaml)
=> #<CeramicCollection:0x0000000107a2cf30
@ceramics=
[#<Ceramic:0x0000000107a2cf30
@ceramic_id="vase1",
@ceramic_type="vase",
@ceramic_details=
#<CeramicDetails:0x0000000107a2cf30
@name="Imperial Vase",
@insignia="Tang Tianbao">,
@ceramic_urn="urn:ceramic:vase:vase1">,
#<Ceramic:0x0000000107a2cf30
@ceramic_id="bowl2",
@ceramic_type="bowl",
@ceramic_details=
#<CeramicDetails:0x0000000107a2cf30
@name="18th Century Bowl",
@insignia="Ming Wanli">
@ceramic_urn="urn:ceramic:bowl:bowl2">]
> first_ceramic = ceramics_collection.ceramics.first
> puts first_ceramic.to_yaml
=>
# ---
# id: vase1
# type: vase
# details:
# name: Imperial Vase
# insignia: Tang Tianbao
# urn: urn:ceramic:vase:vase1
> new_collection = CeramicCollection.new(ceramics: [
Ceramic.new(ceramic_id: "vase1",
ceramic_type: "vase",
ceramic_urn: "urn:ceramic:vase:vase1",
ceramic_details: CeramicDetails.new(
name: "Imperial Vase", insignia: "Tang Tianbao")
),
Ceramic.new(ceramic_id: "bowl2",
ceramic_type: "bowl",
ceramic_urn: "urn:ceramic:vase:bowl2",
ceramic_details: CeramicDetails.new(
name: "18th Century Bowl", insignia: "Ming Wanli")
)
])
> new_collection.to_yaml
>
# ---
# vase1:
# type: vase
# details:
# name: Imperial Vase
# insignia: Tang Tianbao
# urn:
# primary: urn:ceramic:vase:vase1
# bowl2:
# type: bowl
# details:
# name: 18th Century Bowl
# insignia: Ming Wanli
# urn:
# primary: urn:ceramic:bowl:bowl2Case 3: Mapping the key and delegating value to an inner object
In this use case, the "key" of the keyed collection is moved into the collection object to an attribute, and the entire "value" of the keyed collection is mapped to another attribute of the collection object.
When the "key" is moved into the collection object, the following happens:
-
The "key" of the keyed collection maps to a particular attribute of the collection’s instance object.
-
The data of the "value" of the keyed collection will be entirely mapped into an attribute of the collection’s instance object.
-
The original mapping of the "value" attribute of the collection’s instance object is retained.
The root_mappings option should only contain two mappings, and the mappings
must lead to both the :key and :value keywords.
Syntax:
class SomeKeyedCollection < Lutaml::Model::Serializable
attribute :name_of_attribute, AttributeValueType, collection: true
hsh | json | yaml | toml | key_value do
map to: :name_of_attribute,
root_mappings: {
value_type_attribute_name_for_key: :key, (1)
value_type_attribute_name_for_value: :value (2)
}
end
end
class AttributeValueType < Lutaml::Model::Serializable
attribute :value_type_attribute_name_for_key, :string
attribute :value_type_attribute_name_for_value, SomeObject
end-
The
:keykeyword specifies that the "key" of the keyed collection maps to thevalue_type_attribute_name_for_keyattribute of the collection’s instance object (i.e.AttributeValueType). -
The
:valuekeyword specifies that the entire "value" of the keyed collection maps to thevalue_type_attribute_name_for_valueattribute of the collection’s instance object (i.e.SomeObject).
When the root_mappings mapping contains more than one mapping rule, the
root_mappings mapping will override all other mappings in the collection
object. This means that unmapped attributes in root_mappings will not be
incorporated in the collection instance objects.
map with root_mappings (key and value) to map a keyed collection into individual modelsGiven this data:
---
vase1:
name: Imperial Vase
insignia: "Tang Tianbao"
bowl2:
name: 18th Century Bowl
insignia: "Ming Wanli"A model can be defined for this YAML as follows:
# This is a normal Lutaml::Model class
class CeramicDetails < Lutaml::Model::Serializable
attribute :name, :string
attribute :insignia, :string
key_value do
map 'name', to: :name
map 'insignia', to: :insignia
end
end
# This is a normal Lutaml::Model class
class Ceramic < Lutaml::Model::Serializable
attribute :ceramic_id, :string
attribute :ceramic_details, CeramicDetails
key_value do
map 'id', to: :ceramic_id
map 'details', to: :ceramic_details
end
end
# This is Lutaml::Model class that represents the collection of Ceramic objects
class CeramicCollection < Lutaml::Model::Serializable
attribute :ceramics, Ceramic, collection: true
key_value do
map to: :ceramics, # All data goes to the `ceramics` attribute
root_mappings: {
# The key of an object in this collection is mapped to the ceramic_id
# attribute of the Ceramic object.
# (e.g. `vase1`, `bowl2`)
ceramic_id: :key,
# The value of an object in this collection is mapped to the
# ceramic_details attribute of the Ceramic object.
# (e.g. `name: 18th Century Bowl`, `insignia: "Ming Wanli"`
ceramic_details: :value
}
end
end# Parsing the YAML collection with dynamic data keys
> ceramic_collection = CeramicCollection.from_yaml(yaml)
> #<CeramicCollection:0x0000000104ac7240
@ceramics=
[#<Ceramic:0x0000000104ac6e30
@ceramic_id="vase1",
@ceramic_details=
#<CeramicDetails:0x0000000104ac6e30
@name="Imperial Vase",
@insignia="Tang Tianbao">,
#<Ceramic:0x0000000104ac58f0
@ceramic_id="bowl2",
@ceramic_details=
#<CeramicDetails:0x0000000104ac58f0
@name="18th Century Bowl",
@insignia="Ming Wanli">]
# NOTE: When an individual Ceramic object is serialized, the `id` attribute is
# the original key in the incoming YAML data.
> first_ceramic = ceramic_collection.ceramics.first
> puts first_ceramic.to_yaml
=>
# ---
# id: vase1
# details:
# name: Imperial Vase
# insignia: Tang Tianbao
# NOTE: When in a collection, the `ceramic_id` attribute is used to key the data,
# and it disappears from the individual object.
> puts ceramic_collection.to_yaml
=>
# ---
# vase1:
# name: Imperial Vase
# insignia: Tang Tianbao
# bowl2:
# name: 18th Century Bowl
# insignia: Ming Wanli
# NOTE: When the collection is serialized, the `ceramic_id` attribute is used to
# key the data. This is defined through the `map` with `root_mappings` method in
# CeramicCollection.
> new_collection = CeramicCollection.new(ceramics: [
Ceramic.new(ceramic_id: "vase1",
ceramic_details: CeramicDetails.new(
name: "Imperial Vase", insignia: "Tang Tianbao")
),
Ceramic.new(ceramic_id: "bowl2",
ceramic_details: CeramicDetails.new(
name: "18th Century Bowl", insignia: "Ming Wanli")
)
])
> puts new_collection.to_yaml
=>
# ---
# vase1:
# name: Imperial Vase
# insignia: Tang Tianbao
# bowl2:
# name: 18th Century Bowl
# insignia: Ming WanliAttribute extraction
|
Note
|
This feature is for key-value data model serialization only. |
The child_mappings option is used to extract results from a key-value
serialization data model (Hash, JSON, YAML, TOML) into a Lutaml::Model::Serializable
object (collection or not).
The values are extracted from the key-value data model using the list of keys provided.
Syntax:
class SomeObject < Lutaml::Model::Serializable
attribute :name_of_attribute, AttributeValueType, collection: true
hsh | json | yaml | toml | key_value do
map 'key_value_model_attribute_name', to: :name_of_attribute,
child_mappings: {
value_type_attribute_name_1: (1)
{path_to_value_1}, (2)
value_type_attribute_name_2:
{path_to_value_2},
# ...
}
end
end-
The
value_type_attribute_name_1is the attribute name in theAttributeValueTypemodel. The value of this attribute will be assigned the key of the hash in the key-value data model. -
The
path_to_value_1is an array of keys that represent the path to the value in the key-value serialization data model. The keys are used to extract the value from the key-value serialization data model and assign it to the attribute in theAttributeValueTypemodel.The
path_to_valueis in a nested array format with each value a symbol or a string, where each symbol represents a key to traverse down. The last key in the path is the value to be extracted.
The following JSON contains 2 keys in schema named engine and gearbox.
{
"components": {
"engine": {
"manufacturer": "Ford",
"model": "V8"
},
"gearbox": {
"manufacturer": "Toyota",
"model": "4-speed"
}
}
}The path to value for the engine schema is [:components, :engine] and for
the gearbox schema is [:components, :gearbox].
In path_to_value, the :key and :value are reserved instructions used to
assign the key or value of the serialization data respectively as the value to
the attribute.
In the following JSON content, the path_to_value for the object keys named
engine and gearbox will utilize the :key keyword to assign the key of the
object as the value of a designated attribute.
{
"components": {
"engine": { /*...*/ },
"gearbox": { /*...*/ }
}
}If a specified value path is not found, the corresponding attribute in the model
will be assigned a nil value.
nil when the path_to_value is not foundIn the following JSON content, the path_to_value of [:extras, :sunroof] and
[:extras, :drinks_cooler] at the object "gearbox" would be set to nil.
{
"components": {
"engine": {
"manufacturer": "Ford",
"extras": {
"sunroof": true,
"drinks_cooler": true
}
},
"gearbox": {
"manufacturer": "Toyota"
}
}
}child_mappings option to extract values from a key-value data modelThe following JSON contains 2 keys in schema named foo and bar.
{
"schemas": {
"foo": { (1)
"path": { (2)
"link": "link one",
"name": "one"
}
},
"bar": { (1)
"path": { (2)
"link": "link two",
"name": "two"
}
}
}
}-
The keys
fooandbarare to be mapped to theidattribute. -
The nested
path.linkandpath.namekeys are used as thelinkandnameattributes, respectively.
A model can be defined for this JSON as follows:
class Schema < Lutaml::Model::Serializable
attribute :id, :string
attribute :link, :string
attribute :name, :string
end
class ChildMappingClass < Lutaml::Model::Serializable
attribute :schemas, Schema, collection: true
json do
map "schemas", to: :schemas,
child_mappings: {
id: :key,
link: %i[path link],
name: %i[path name],
}
end
endThe output becomes:
> ChildMappingClass.from_json(json)
> #<ChildMappingClass:0x0000000104ac7240
@schemas=
[#<Schema:0x0000000104ac6e30 @id="foo", @link="link one", @name="one">,
#<Schema:0x0000000104ac58f0 @id="bar", @link="link two", @name="two">]>
> ChildMappingClass.new(schemas: [Schema.new(id: "foo", link: "link one", name: "one"), Schema.new(id: "bar", link: "link two", name: "two")]).to_json
> #{"schemas"=>{"foo"=>{"path"=>{"link"=>"link one", "name"=>"one"}}, {"bar"=>{"path"=>{"link"=>"link two", "name"=>"two"}}}}}In this example:
-
The
keyof each schema (fooandbar) is mapped to theidattribute. -
The nested
path.linkandpath.namekeys are mapped to thelinkandnameattributes, respectively.
Collection data models
General
Collection data models represent a group of models, mapping to an instance of Lutaml::Model::Collection.
Collection data models supported are identified by their short name:
jsonl-
JSONL (JSON Lines)
yamls-
YAML Stream (multi-document format)
Mapping
As with collections in general, the map method is used to define collection
mappings.
Syntax:
class MySerializedCollection < Lutaml::Model::Collection
instances {attribute}, ModelType
{collection_type_short} do
map_instances to: {attribute}
end
endWhere,
{collection_type_short}-
The short name of the collection type (e.g.
jsonl,yamls). {attribute}-
The name of the attribute in the collection that will hold the collection data.
ModelType-
The type of the model that will be used in the collection.
A singular model may also utilize collection data models in the following manner.
Syntax:
class MySerializedCollection < Lutaml::Model::Serializeable
attribute {attribute}, ModelType, collection: true
{collection_type_short} do
# Notice that there is no key_name i.e map <key_name>, to: <attribute_name>,
# This is because in a collection there are no keys. Each object needs to be
# mapped to the attribute.
map to: {attribute}
end
endWhere,
{collection_type_short}-
The short name of the collection type (e.g.
jsonl,yamls). {attribute}-
The name of the attribute in the collection that will hold the collection data.
ModelType-
The type of the model that will be used in the collection.
JSONL
JSONL (short for JSON Lines) is a serialization format where each line represents a valid JSON object. The format is meant to be efficient for large datasets such as for streaming or batch processing.
It represents a collection of JSON objects encoded one object per line.
|
Note
|
The contents of JSONL itself is not valid JSON, but each line is a valid JSON. |
Since JSONL contains JSON elements, the model specified with instances or
attribute must support JSON.
Every line in a JSONL file is also a valid JSON object. If JSONL-specific
mappings (through jsonl) are not defined in the model, the existing json
mappings are used instead as a fallback for serialization and deserialization.
class Person
attribute :name, :string
attribute :age, :integer
attribute :id, :string
end
class Directory < Lutaml::Model::Collection
instances :persons, Person
jsonl do
map_instances to: :persons
end
end
jsonl = <<~JSONL
{"name":"John","age":30,"id":"abc-123"}
{"name":"Jane","age":25,"id":"def-456"}
JSONL
jsonl = Directory.from_jsonl(jsonl)
# => <Directory:0x00007fae4b0c9b10
# @persons=[
# <Person:0x00007fae4b0c9970 @name="John", @age=30, @id="abc-123">,
# <Person:0x00007fae4b0c9838 @name="Jane", @age=25, @id="def-456">
# ]>class Person
attribute :name, :string
attribute :age, :integer
attribute :id, :string
end
class Directory < Lutaml::Model::Serializeable
attribute :persons, Person, collection: true
jsonl do
map_instances to: :persons
end
end
jsonl = <<~JSONL
{"name":"John","age":30,"id":"abc-123"}
{"name":"Jane","age":25,"id":"def-456"}
JSONL
jsonl = Directory.from_jsonl(jsonl)
# => <Directory:0x00007fae4b0c9b10
# @persons=[
# <Person:0x00007fae4b0c9970 @name="John", @age=30, @id="abc-123">,
# <Person:0x00007fae4b0c9838 @name="Jane", @age=25, @id="def-456">
# ]>class Person
attribute :name, :string
attribute :age, :integer
attribute :id, :string
json do
map "full_name", to: :name
map "age", to: :age
map "id", to: :id
end
end
class Directory < Lutaml::Model::Collection
instances :persons, Person
jsonl do
map_instances to: :persons
end
end
jsonl = <<~JSONL
{"full_name":"John Doe","age":30,"id":"abc-123"}
{"full_name":"Jane Smith","age":25,"id":"def-456"}
JSONL
jsonl = Directory.from_jsonl(jsonl)
# => <Directory:0x00007fae4b0c9b10
# @persons=[
# <Person:0x00007fae4b0c9970 @name="John Doe", @age=30, @id="abc-123">,
# <Person:0x00007fae4b0c9838 @name="Jane Smith", @age=25, @id="def-456">
# ]>YAML Stream
YAML Stream (short for YAML multi-document format) is a serialization format
where each document is separated by a document separator (---). The format is
meant to be efficient for large datasets such as for streaming or batch
processing.
It represents a collection of YAML documents encoded one document per stream.
|
Note
|
The contents of YAML Stream is valid YAML, where each document is a valid YAML document separated by document separators. |
Since YAML Stream contains YAML elements, the model specified with instances
or attribute must support YAML.
Every document in a YAML Stream file is also a valid YAML document. If YAML
Stream-specific mappings (through yamls) are not defined in the model, the
existing yaml mappings are used instead as a fallback for serialization and
deserialization.
class Person
attribute :name, :string
attribute :age, :integer
attribute :id, :string
end
class Directory < Lutaml::Model::Collection
instances :persons, Person
yamls do
map_instances to: :persons
end
end
yamls = <<~YAMLS
---
name: John
age: 30
id: abc-123
---
name: Jane
age: 25
id: def-456
YAMLS
yamls = Directory.from_yamls(yamls)
# => <Directory:0x00007fae4b0c9b10
# @persons=[
# <Person:0x00007fae4b0c9970 @name="John", @age=30, @id="abc-123">,
# <Person:0x00007fae4b0c9838 @name="Jane", @age=25, @id="def-456">
# ]>class Person
attribute :name, :string
attribute :age, :integer
attribute :id, :string
end
class Directory < Lutaml::Model::Serializeable
attribute :persons, Person, collection: true
yamls do
map_instances to: :persons
end
end
yamls = <<~YAMLS
---
name: John
age: 30
id: abc-123
---
name: Jane
age: 25
id: def-456
YAMLS
yamls = Directory.from_yamls(yamls)
# => <Directory:0x00007fae4b0c9b10
# @persons=[
# <Person:0x00007fae4b0c9970 @name="John", @age=30, @id="abc-123">,
# <Person:0x00007fae4b0c9838 @name="Jane", @age=25, @id="def-456">
# ]>class Person
attribute :name, :string
attribute :age, :integer
attribute :id, :string
yaml do
map "full_name", to: :name
map "age", to: :age
map "id", to: :id
end
end
class Directory < Lutaml::Model::Collection
instances :persons, Person
yamls do
map_instances to: :persons
end
end
yamls = <<~YAMLS
---
full_name: John Doe
age: 30
id: abc-123
---
full_name: Jane Smith
age: 25
id: def-456
YAMLS
yamls = Directory.from_yamls(yamls)
# => <Directory:0x00007fae4b0c9b10
# @persons=[
# <Person:0x00007fae4b0c9970 @name="John Doe", @age=30, @id="abc-123">,
# <Person:0x00007fae4b0c9838 @name="Jane Smith", @age=25, @id="def-456">
# ]>Format-independent mechanisms
Mapping value transformation
A mapping value transformation is used when the value of an attribute needs to be transformed around the serialization process. Collection attributes are also supported.
This is useful when the representation of the value in a serialization format differs from its internal representation in the model.
|
Note
|
Value transformation can be applied at the attribute-level or at the serialization-mapping level. They can also be applied together. |
Syntax:
class SomeObject < Lutaml::Model::Serializable
# Attribute-level transformation
attribute :attribute_name, {attr_type}, transform: { (1)
export: ->(value) { ... },
import: ->(value) { ... }
}
# Mapping-level transformation in JSON format
{key_value_formats} do
map "key", to: :attribute_name, transform: { (2)
export: ->(value) { ... },
import: ->(value) { ... }
}
end
# Mapping-level transformation in XML format
xml do
map_element "ElementName", to: :attribute_name, transform: { (3)
export: ->(value) { ... },
import: ->(value) { ... }
}
map_attribute "AttributeName", to: :attribute_name, transform: {
export: ->(value) { ... },
import: ->(value) { ... }
}
end
end-
At the attribute level, the
transformoption applied to theattributemethod is used to define the transformation for the attribute. -
At the mapping level (for
{key_value_formats}formats), thetransformoption applied to themapmethod is used to define the transformation for the mapping. -
At the mapping level (for the XML format), the
transformoption applied to themap_*methods is used to define the transformation for the mapping.
Where,
attribute_name-
The name of the attribute.
attr_type-
The type of the attribute.
- Attribute-level
transform -
The option to define a transformation for the attribute value.
- Attribute-level
export -
The transformation
Procfor the value when it is being retrieved from the model. - Attribute-level
import -
The transformation
Procfor the value when it is being assigned to the model. {key_value_formats}-
The serialization format (e.g.
hsh,json,yaml,toml,key_value) for which the mapping is defined. - Mapping-level
transform -
The option to define a transformation for the serialization mapping value. The value given to the Proc is the model attribute value that does not go through attribute-level transform.
- Mapping-level
export -
The transformation
Procfor the attribute value when it is being written to the serialization format. - Mapping-level
import -
The transformation
Procfor the value when it is being read from the serialization format and assigned to the model.
class Ceramic < Lutaml::Model::Serializable
attribute :glaze_type, :string
# Mapping-level transformation in key-value formats
json do
map "glazeType", to: :glaze_type, transform: {
export: ->(value) { "Traditional #{value}" },
import: ->(value) { value.gsub("Traditional ", "") }
}
end
# Mapping-level transformation in XML format
xml do
root "Ceramic"
map_attribute "glaze-type", to: :glaze_type, transform: {
export: ->(value) { "Traditional #{value}" },
import: ->(value) { value.gsub("Traditional ", "") }
}
end
endceramic = Ceramic.new(glaze_type: "celadon")
# Export transformation applied on defined mapping
ceramic.to_json
# => {"glazeType": "Traditional celadon"}
# Export transformation applied on defined mapping
ceramic.to_xml
# => <Ceramic glaze-type="Traditional celadon"/>
# No export transformation when no mapping exists
ceramic.to_yaml
# => glaze_type: "celadon"
# Import transformation applied on defined mapping
ceramic = Ceramic.from_json('{ "glazeType" => "Traditional celadon" }')
ceramic.glaze_type
# => "celadon"
# Import transformation applied on defined mapping
ceramic = Ceramic.from_xml('<Ceramic glaze-type="Traditional raku"/>')
ceramic.glaze_type
# => "raku"
# No import transformation when no mapping exists
ceramic = Ceramic.from_yaml('glaze_type: "Traditional celadon"')
ceramic.glaze_type
# => "Traditional celadon"Attribute-level and mapping-level transformations can be used together for the same attribute in a chained fashion.
Precedence applies to the two levels of transformation for deserialization:
-
Mapping-level transformation, if defined, occurs first
-
Attribute-level transformation, if defined, is applied to the result of the mapping-level transformation
Conversely, precedence applies in the same order for serialization:
-
Attribute-level transformation, if defined, occurs first
-
Mapping-level transformation, if defined, is applied to the result of the attribute-level transformation
This mechanism allows for flexible value transformations without needing format-specific custom methods.
╔════════════════════════════╗ ╔════════════════════════════╗
║ Serialization Format Value ║ ║ Serialization Format Value ║
╚════════════════════════════╝ ╚════════════════════════════╝
| ▲
▼ |
╔════════════════════════════╗ ╔════════════════════════════╗
║ Mapping Transform ║ ║ Mapping Transform ║
╚════════════════════════════╝ ╚════════════════════════════╝
| ▲
▼ |
╔════════════════════════════╗ ╔════════════════════════════╗
║ Attribute Transform ║ ║ Attribute Transform ║
╚════════════════════════════╝ ╚════════════════════════════╝
| ▲
▼ |
╔════════════════════════════╗ ╔════════════════════════════╗
║ Model Attribute Value ║ ║ Model Attribute Value ║
╚════════════════════════════╝ ╚════════════════════════════╝class Ceramic < Lutaml::Model::Serializable
# Attribute-level transformation
attribute :glaze_type, :string, transform: {
export: ->(value) { "Ceramic #{value}" },
import: ->(value) { value.gsub("Ceramic ", "") }
}
# Mapping-level transformation in key-value formats
json do
map "glazeType", to: :glaze_type, transform: {
export: ->(value) { "Traditional #{value}" },
import: ->(value) { value.gsub("Traditional ", "") }
}
end
# Mapping-level transformation in XML format
xml do
root "Ceramic"
map_attribute "glaze-type", to: :glaze_type, transform: {
export: ->(value) { "Traditional #{value}" },
import: ->(value) { value.gsub("Traditional ", "") }
}
end
endceramic = Ceramic.new(glaze_type: "Ceramic celadon")
# Attribute-level export transformation applied
ceramic.glaze_type
# => "Ceramic celadon"
# Internal representation
ceramic.instance_value_get(:@glaze_type)
# => "celadon"
# Mapping-level export transformation applied to attribute-level transformed value
ceramic.to_json
# => {"glazeType": "Traditional Ceramic celadon"}
# Mapping-level export transformation applied to attribute-level transformed value
ceramic.to_xml
# => <Ceramic glaze-type="Traditional Ceramic celadon"/>
# No mapping-level export transformation when no mapping exists
ceramic.to_yaml
# => glaze_type: "Ceramic celadon"
# Attribute-level import transformation applied to mapping-level transformed value
ceramic = Ceramic.from_json('{ "glazeType" => "Traditional Ceramic celadon" }')
ceramic.glaze_type
# => "Ceramic celadon"
# Attribute-level import transformation applied to mapping-level transformed value
ceramic = Ceramic.from_xml('<Ceramic glaze-type="Traditional Ceramic raku"/>')
ceramic.glaze_type
# => "Ceramic raku"
# No mapping-level import transformation when no mapping exists
ceramic = Ceramic.from_yaml('glaze_type: "Ceramic celadon"')
ceramic.glaze_type
# => "Ceramic celadon"Class-based transformers
Instead of using proc-based transformations, you can define reusable transformer
classes that inherit from Lutaml::Model::ValueTransformer.
Class-based transformers provide better organization, reusability, and testing capabilities compared to inline proc transformations.
Syntax:
class CustomTransformer < Lutaml::Model::ValueTransformer
def to_json
# Transform the value for JSON output
end
def from_json(input_value)
# Transform the input value from JSON
end
def to_xml
# Transform the value for XML output
end
def from_xml(input_value)
# Transform the input value from XML
end
# Define methods for other formats as needed:
# to_yaml, from_yaml, to_toml, from_toml, etc.
end
class SomeObject < Lutaml::Model::Serializable
# Use the transformer class directly
attribute :attribute_name, {attr_type}, transform: CustomTransformer
endWhere,
CustomTransformer-
A class that inherits from
Lutaml::Model::ValueTransformerand implements format-specific transformation methods. to_*-
Methods that transform the internal value when serializing to a format. The current value is available as
valuewithin the method. from_*-
Methods that transform the input value when deserializing from a format.
|
Note
|
If a transformer class doesn’t implement a method for a specific format, that format will not be transformed, allowing selective transformation. |
class MeasurementTransformer < Lutaml::Model::ValueTransformer
def to_json
return value if value.nil?
"#{value["value"]} #{value["unit"]}"
end
def from_json(input_value)
number, unit = input_value.split
{ "value" => number.to_f, "unit" => unit }
end
def to_xml
return value if value.nil?
"#{value["value"]} #{value["unit"]}"
end
def from_xml(input_value)
number, unit = input_value.split
{ "value" => number.to_f, "unit" => unit }
end
# YAML and TOML methods not defined - those formats won't be transformed
end
class Product < Lutaml::Model::Serializable
attribute :measurement, :hash, transform: MeasurementTransformer
endproduct = Product.new(measurement: { "value" => 10.5, "unit" => "cm" })
# JSON and XML transformations are applied
puts product.to_json
# => {"measurement":"10.5 cm"}
puts product.to_xml
# => <Product>
# <measurement>10.5 cm</measurement>
# </Product>
# YAML transformation is not applied (no to_yaml method defined)
puts product.to_yaml
# => measurement:
# value: 10.5
# unit: cm
# Deserialization works the same way
Product.from_json('{"measurement": "15.0 mm"}').measurement
# => {"value"=>15.0, "unit"=>"mm"}
Product.from_yaml("measurement:\n value: 20.0\n unit: km").measurement
# => {"value"=>20.0, "unit"=>"km"}Class-based transformers can be combined with both attribute-level and mapping-level transformations, following the same precedence rules as proc-based transformers.
class PrefixTransformer < Lutaml::Model::ValueTransformer
def to_json(*_args)
"PREFIX:#{value}"
end
def from_json(input_value)
input_value.gsub("PREFIX:", "")
end
def to_xml(*_args)
"PREFIX:#{value}"
end
def from_xml(input_value)
input_value.gsub("PREFIX:", "")
end
end
class SuffixTransformer < Lutaml::Model::ValueTransformer
def to_json(*_args)
"#{value}:SUFFIX"
end
def from_json(input_value)
input_value.gsub(":SUFFIX", "")
end
def to_xml(*_args)
"#{value}:SUFFIX"
end
def from_xml(input_value)
input_value.gsub(":SUFFIX", "")
end
end
class CombinedTransformModel < Lutaml::Model::Serializable
# Attribute-level transformer
attribute :title, :string, transform: PrefixTransformer
json do
# Mapping-level transformer (applied in addition to attribute-level)
map "title", to: :title, transform: SuffixTransformer
end
xml do
root "CombinedTransformModel"
map_element "title", to: :title, transform: SuffixTransformer
end
endmodel = CombinedTransformModel.new(title: "hello")
# For JSON output:
# 1. Attribute transformer (PrefixTransformer): "hello" -> "PREFIX:hello"
# 2. Mapping transformer (SuffixTransformer): "PREFIX:hello" -> "PREFIX:hello:SUFFIX"
puts model.to_json
# => {"title": "PREFIX:hello:SUFFIX"}
# For JSON input:
# 1. Mapping transformer (SuffixTransformer): "hello:SUFFIX" -> "hello"
# 2. Attribute transformer (PrefixTransformer): "hello" -> "hello" (stored value)
model = CombinedTransformModel.from_json('{"title": "hello:SUFFIX"}')
model.title
# => "hello"Separate data model class
The Serialize module can be used to define only serialization mappings for a
separately defined data model class (a Ruby class).
|
Note
|
This is traditionally called "custom model". |
Syntax:
class MappingClass < Lutaml::Model::Serializable
model {DataModelClass}
# ...
endWhere,
MappingClass-
The class that represents the serialization mappings. This class must be a subclass of
Lutaml::Model::Serializable. DataModelClass-
The class that represents the data model.
When using a separate data model class, it is important to remember that the
serialization methods (instance#to_*, klass.from_*, such as
instance.to_yaml, instance.to_xml or Klass.from_yaml, Klass.from_xml),
are to be called on the mapping class, not the data model instance.
model method to define serialization mappings for a separate modelclass Ceramic
attr_accessor :type, :glaze
def name
"#{type} with #{glaze}"
end
end
class CeramicSerialization < Lutaml::Model::Serializable
model Ceramic
xml do
map_element 'type', to: :type
map_element 'glaze', to: :glaze
end
end> Ceramic.new(type: "Porcelain", glaze: "Clear").name
> # "Porcelain with Clear"
> CeramicSerialization.from_xml(xml)
> #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze="Clear">
> Ceramic.new(type: "Porcelain", glaze: "Clear").to_xml
> #<Ceramic><type>Porcelain</type><glaze>Clear</glaze></Ceramic>model method to define serialization mappings for a separate model in a model hierarchyThe following class will parse the XML snippet below:
class CustomModelChild
attr_accessor :street, :city
end
class CustomModelChildMapper < Lutaml::Model::Serializable
model CustomModelChild
attribute :street, Lutaml::Model::Type::String
attribute :city, Lutaml::Model::Type::String
xml do
map_element :street, to: :street
map_element :city, to: :city
end
end
class CustomModelParentMapper < Lutaml::Model::Serializable
attribute :first_name, Lutaml::Model::Type::String
attribute :child_mapper, CustomModelChildMapper
xml do
map_element :first_name, to: :first_name
map_element :CustomModelChild,
with: { to: :child_to_xml, from: :child_from_xml }
end
def child_to_xml(model, parent, doc)
child_el = doc.create_element("CustomModelChild")
street_el = doc.create_element("street")
city_el = doc.create_element("city")
doc.add_text(street_el, model.child_mapper.street)
doc.add_text(city_el, model.child_mapper.city)
doc.add_element(child_el, street_el)
doc.add_element(child_el, city_el)
doc.add_element(parent, child_el)
end
def child_from_xml(model, value)
model.child_mapper ||= CustomModelChild.new
model.child_mapper.street = value["elements"]["street"].text
model.child_mapper.city = value["elements"]["city"].text
end
end<CustomModelParent>
<first_name>John</first_name>
<CustomModelChild>
<street>Oxford Street</street>
<city>London</city>
</CustomModelChild>
</CustomModelParent>> instance = CustomModelParentMapper.from_xml(xml)
> #<CustomModelParent:0x0000000107c9ca68 @child_mapper=#<CustomModelChild:0x0000000107c95218 @city="London", @street="Oxford Street">, @first_name="John">
> CustomModelParentMapper.to_xml(instance)
> #<CustomModelParent><first_name>John</first_name><CustomModelChild><street>Oxford Street</street><city>London</city></CustomModelChild></CustomModelParent>Rendering default values (forced rendering of default values)
By default, attributes with default values are not rendered if the current value is the same as the default value.
In certain cases, it is necessary to render the default value even if the
current value is the same as the default value. This is achieved by setting the
render_default option to true.
Syntax:
attribute :name_of_attribute, Type, default: -> { value }
xml do
map_element 'name_of_attribute', to: :name_of_attribute, render_default: true
map_attribute 'name_of_attribute', to: :name_of_attribute, render_default: true
end
hsh | json | yaml | toml | key_value do
map 'name_of_attribute', to: :name_of_attribute, render_default: true
endrender_default option to force encoding the default valueclass Glaze < Lutaml::Model::Serializable
attribute :color, :string, default: -> { 'Clear' }
attribute :opacity, :string, default: -> { 'Opaque' }
attribute :temperature, :integer, default: -> { 1050 }
attribute :firing_time, :integer, default: -> { 60 }
xml do
root "glaze"
map_element 'color', to: :color
map_element 'opacity', to: :opacity, render_default: true
map_attribute 'temperature', to: :temperature
map_attribute 'firingTime', to: :firing_time, render_default: true
end
json do
map 'color', to: :color
map 'opacity', to: :opacity, render_default: true
map 'temperature', to: :temperature
map 'firingTime', to: :firing_time, render_default: true
end
endrender_default: true are rendered when the value is identical to the default> glaze_new = Glaze.new
> puts glaze_new.to_xml
# <glaze firingTime="60">
# <opacity>Opaque</opacity>
# </glaze>
> puts glaze_new.to_json
# {"firingTime":60,"opacity":"Opaque"}render_default: true with non-default values are rendered> glaze = Glaze.new(color: 'Celadon', opacity: 'Semitransparent', temperature: 1300, firing_time: 90)
> puts glaze.to_xml
# <glaze color="Celadon" temperature="1300" firingTime="90">
# <opacity>Semitransparent</opacity>
# </glaze>
> puts glaze.to_json
# {"color":"Celadon","temperature":1300,"firingTime":90,"opacity":"Semitransparent"}Advanced attribute mapping
Mapping multiple names to a single attribute
The mapping methods support multiple names mapping to a single attribute using an array of names.
Syntax:
hsh | json | yaml | toml | key_value do
map ["name1", "name2"], to: :attribute_name
end
xml do
map_element ["name1", "name2"], to: :attribute_name
map_attribute ["attr1", "attr2"], to: :attribute_name
endWhen serializing, the first element in the array of mapped names is always used as the output name.
class CustomModel < Lutaml::Model::Serializable
attribute :full_name, Lutaml::Model::Type::String
attribute :color, Lutaml::Model::Type::String
attribute :id, Lutaml::Model::Type::String
json do
map ["name", "custom_name"], with: { to: :name_to_json, from: :name_from_json }
map ["color", "shade"], with: { to: :color_to_json, from: :color_from_json }
end
xml do
root "CustomModel"
map_element ["name", "custom-name"], with: { to: :name_to_xml, from: :name_from_xml }
map_element ["color", "shade"], with: { to: :color_to_xml, from: :color_from_xml }
map_attribute ["id", "identifier"], to: :id
end
# Custom methods for JSON
def name_to_json(model, doc)
doc["name"] = "JSON Model: #{model.full_name}"
end
def name_from_json(model, value)
model.full_name = value&.sub(/^JSON Model: /, "")
end
def color_to_json(model, doc)
doc["color"] = model.color.upcase
end
def color_from_json(model, value)
model.color = value&.downcase
end
# Custom methods for XML
def name_to_xml(model, parent, doc)
el = doc.create_element("name")
doc.add_text(el, "XML Model: #{model.full_name}")
doc.add_element(parent, el)
end
def name_from_xml(model, value)
model.full_name = value.sub(/^XML Model: /, "")
end
def color_to_xml(model, parent, doc)
el = doc.create_element("color")
doc.add_text(el, model.color.upcase)
doc.add_element(parent, el)
end
def color_from_xml(model, value)
model.color = value.downcase
end
endFor JSON:
{
"custom_name": "JSON Model: Vase",
"shade": "BLUE",
"identifier": "123"
}For XML:
<CustomModel id="123">
<name>XML Model: Vase</name>
<color>BLUE</color>
</CustomModel>> model = CustomModel.from_json(json)
> model.full_name
> # "Vase"
> model.color
> # "blue"Attribute mapping delegation
Delegate attribute mappings to nested objects using the delegate option.
Syntax:
xml | hsh | json | yaml | toml do
map 'key_value_model_attribute_name', to: :name_of_attribute, delegate: :model_to_delegate_to
enddelegate option to map attributes to nested objectsThe following class will parse the JSON snippet below:
class Glaze < Lutaml::Model::Serializable
attribute :color, :string
attribute :temperature, :integer
json do
map 'color', to: :color
map 'temperature', to: :temperature
end
end
class Ceramic < Lutaml::Model::Serializable
attribute :type, :string
attribute :glaze, Glaze
json do
map 'type', to: :type
map 'color', to: :color, delegate: :glaze
end
end{
"type": "Porcelain",
"color": "Clear"
}> Ceramic.from_json(json)
> #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze=#<Glaze:0x0000000104ac7240 @color="Clear", @temperature=nil>>
> Ceramic.new(type: "Porcelain", glaze: Glaze.new(color: "Clear")).to_json
> #{"type"=>"Porcelain", "color"=>"Clear"}|
Note
|
The corresponding keyword used by Shale is receiver: instead of
delegate:.
|
Attribute serialization with custom methods
General
Define custom methods for specific attribute mappings using the with: key for
each serialization mapping block for from and to.
XML serialization with custom methods
Syntax:
xml do
map_element 'element_name', to: :name_of_element, with: {
to: :method_name_to_serialize,
from: :method_name_to_deserialize
}
map_attribute 'attribute_name', to: :name_of_attribute, with: {
to: :method_name_to_serialize,
from: :method_name_to_deserialize
}
map_content, to: :name_of_content, with: {
to: :method_name_to_serialize,
from: :method_name_to_deserialize
}
endwith: key to define custom serialization methods for XMLThe following class will parse the XML snippet below:
class Metadata < Lutaml::Model::Serializable
attribute :category, :string
attribute :identifier, :string
end
class CustomCeramic < Lutaml::Model::Serializable
attribute :name, :string
attribute :size, :integer
attribute :description, :string
attribute :metadata, Metadata
xml do
map_element "Name", to: :name, with: { to: :name_to_xml, from: :name_from_xml }
map_attribute "Size", to: :size, with: { to: :size_to_xml, from: :size_from_xml }
map_content with: { to: :description_to_xml, from: :description_from_xml }
map_element :metadata, to: :metadata, with: { to: :metadata_to_xml, from: :metadata_from_xml }
end
def name_to_xml(model, parent, doc)
el = doc.create_element("Name")
doc.add_text(el, "XML Masterpiece: #{model.name}")
doc.add_element(parent, el)
end
def name_from_xml(model, value)
model.name = value.sub(/^XML Masterpiece: /, "")
end
def size_to_xml(model, parent, doc)
doc.add_attribute(parent, "Size", model.size + 3)
end
def size_from_xml(model, value)
model.size = value.to_i - 3
end
def description_to_xml(model, parent, doc)
doc.add_text(parent, "XML Description: #{model.description}")
end
def description_from_xml(model, value)
model.description = value.join.strip.sub(/^XML Description: /, "")
end
def metadata_to_xml(model, parent, doc)
metadata_el = doc.create_element("metadata")
category_el = doc.create_element("category")
identifier_el = doc.create_element("identifier")
doc.add_text(category_el, model.metadata.category)
doc.add_text(identifier_el, model.metadata.identifier)
doc.add_element(metadata_el, category_el)
doc.add_element(metadata_el, identifier_el)
doc.add_element(parent, metadata_el)
end
def metadata_from_xml(model, value)
model.metadata ||= Metadata.new
model.metadata.category = value["elements"]["category"].text
model.metadata.identifier = value["elements"]["identifier"].text
end
end<CustomCeramic Size="15">
<Name>XML Masterpiece: Vase</Name>
XML Description: A beautiful ceramic vase
<metadata>
<category>Metadata</category>
<identifier>123</identifier>
</metadata>
</CustomCeramic>> CustomCeramic.from_xml(xml)
> #<CustomCeramic:0x0000000108d0e1f8
@element_order=["text", "Name", "text", "Size", "text"],
@name="Masterpiece: Vase",
@ordered=nil,
@size=12,
@description="A beautiful ceramic vase",
@metadata=#<Metadata:0x0000000105ad52e0 @category="Metadata", @identifier="123">>
> puts CustomCeramic.new(name: "Vase", size: 12, description: "A beautiful vase", metadata: Metadata.new(category: "Glaze", identifier: 15)).to_xml
# <CustomCeramic Size="15">
# <Name>XML Masterpiece: Vase</Name>
# <metadata>
# <category>Glaze</category>
# <identifier>15</identifier>
# </metadata>
# XML Description: A beautiful vase
# </CustomCeramic>def custom_method_from_xml(model, value)
instance = value.node # Lutaml::Model::Xml::AdapterElement
# OR
instance = value.node.adapter_node # Adapter::Element
xml = instance.to_xml
endWhen building a model from XML in custom methods, if the value parameter is a mapping_hash, then it allows access to the parsed XML structure through value.node which can be converted to an XML string using to_xml.
|
Note
|
For NokogiriAdapter, we can also call to_xml on value.node.adapter_node.
|
> value
> # {"text"=>["\n ", "\n ", "\n "], "elements"=>{"category"=>{"text"=>"Metadata"}}}
> value.to_xml
> # undefined_method `to_xml`
> value.node
# Nokogiri Adapter Node
#<Lutaml::Model::Xml::NokogiriElement:0x0000000107656ed8
# @attributes={},
# @children=
# [#<Lutaml::Model::Xml::NokogiriElement:0x0000000107656cd0 @attributes={}, @children=[], @default_namespace=nil, @name="text", @namespace_prefix=nil, @text="\n ">,
# #<Lutaml::Model::Xml::NokogiriElement:0x00000001076569b0
# @attributes={},
# @children=
# [#<Lutaml::Model::Xml::NokogiriElement:0x00000001076567f8 @attributes={}, @children=[], @default_namespace=nil, @name="text", @namespace_prefix=nil, @text="Metadata">],
# @default_namespace=nil,
# @name="category",
# @namespace_prefix=nil,
# @text="Metadata">,
# #<Lutaml::Model::Xml::NokogiriElement:0x0000000107656028 @attributes={}, @children=[], @default_namespace=nil, @name="text", @namespace_prefix=nil, @text="\n ">],
# @default_namespace=nil,
# @name="metadata",
# @namespace_prefix=nil,
# @text="\n Metadata\n ">
# Ox Adapter Node
#<Lutaml::Model::Xml::OxElement:0x0000000107584f78
# @attributes={},
# @children=
# [#<Lutaml::Model::Xml::OxElement:0x0000000107584e60
# @attributes={},
# @children=[#<Lutaml::Model::Xml::OxElement:0x0000000107584d48 @attributes={}, @children=[], @default_namespace=nil, @name="text", @namespace_prefix=nil, @text="Metadata">],
# @default_namespace=nil,
# @name="category",
# @namespace_prefix=nil,
# @text="Metadata">],
# @default_namespace=nil,
# @name="metadata",
# @namespace_prefix=nil,
# @text=nil>
# Oga Adapter Node
# <Lutaml::Model::Xml::Oga::Element:0x0000000107314158
# @attributes={},
# @children=
# [#<Lutaml::Model::Xml::Oga::Element:0x0000000107314090 @attributes={}, @children=[], @default_namespace=nil, @name="text", @namespace_prefix=nil, @text="\n ">,
# #<Lutaml::Model::Xml::Oga::Element:0x000000010730fe78
# @attributes={},
# @children=[#<Lutaml::Model::Xml::Oga::Element:0x000000010730fd88 @attributes={}, @children=[], @default_namespace=nil, @name="text", @namespace_prefix=nil, @text="Metadata">],
# @default_namespace=nil,
# @name="category",
# @namespace_prefix=nil,
# @text="Metadata">,
# #<Lutaml::Model::Xml::Oga::Element:0x000000010730f8d8 @attributes={}, @children=[], @default_namespace=nil, @name="text", @namespace_prefix=nil, @text="\n ">],
# @default_namespace=nil,
# @name="metadata",
# @namespace_prefix=nil,
# @text="\n Metadata\n ">
> value.node.to_xml
> #<metadata><category>Metadata</category></metadata>Key-value data model serialization with custom methods
hsh | json | yaml | toml do
map 'attribute_name', to: :name_of_attribute, with: {
to: :method_name_to_serialize,
from: :method_name_to_deserialize
}
endwith: key to define custom serialization methodsThe following class will parse the JSON snippet below:
class CustomCeramic < Lutaml::Model::Serializable
attribute :name, :string
attribute :size, :integer
json do
map 'name', to: :name, with: { to: :name_to_json, from: :name_from_json }
map 'size', to: :size
end
def name_to_json(model, doc)
doc["name"] = "Masterpiece: #{model.name}"
end
def name_from_json(model, value)
model.name = value.sub(/^Masterpiece: /, '')
end
end{
"name": "Masterpiece: Vase",
"size": 12
}> CustomCeramic.from_json(json)
> #<CustomCeramic:0x0000000104ac7240 @name="Vase", @size=12>
> CustomCeramic.new(name: "Vase", size: 12).to_json
> #{"name"=>"Masterpiece: Vase", "size"=>12}Only One Custom Method
Only one custom method can be added for the serialization or deserialization of an attribute.
Syntax:
xml do
map_element 'element_name', to: :name_of_element, with: {
to: :method_name_to_serialize # only 'to' is implemented
}
map_element 'element_name', to: :name_of_element, with: {
from: :method_name_to_deserialize # only 'from' is implemented
}
end
hsh | json | yaml | toml do
map 'attribute_name', to: :name_of_attribute, with: {
to: :method_name_to_serialize # only 'to' is implemented
}
map 'attribute_name', to: :name_of_attribute, with: {
from: :method_name_to_deserialize # only 'from' is implemented
}
endThis is only applicable if the to: :name_of_element (in xml mapping) or to: :name_of_attribute (in key_value mapping) option is set.
If it is not set, then both custom methods must be provided.
class CustomCeramic < Lutaml::Model::Serializable
attribute :name, :string
attribute :size, :integer
xml do
map_element "Name", to: :name, with: { to: :name_to_xml }
map_attribute "Size", to: :size
end
json do
map 'name', to: :name, with: { from: :name_from_json }
map 'size', to: :size
end
def name_to_xml(model, parent, doc)
el = doc.create_element("Name")
doc.add_text(el, "XML Masterpiece: #{model.name}")
doc.add_element(parent, el)
end
def name_from_json(model, value)
model.name = value.sub(/^Masterpiece: /, '')
end
end<CustomCeramic Size="15">
<Name>Vase</Name>
</CustomCeramic>> CustomCeramic.from_xml(xml)
> #<CustomCeramic:0x0000000108d0e1f8
@element_order=["text", "Name", "text", "Size", "text"],
@name="Vase",
@ordered=nil,
@size=15>
> puts CustomCeramic.new(name: "Vase", size: 15).to_xml
# <CustomCeramic Size="15">
# <Name>XML Masterpiece: Vase</Name>
# </CustomCeramic>{
"name": "Masterpiece: Vase",
"size": 12
}> CustomCeramic.from_json(json)
> #<CustomCeramic:0x0000000104ac7240 @name="Vase", @size=12>
> CustomCeramic.new(name: "Vase", size: 12).to_json
> # {"name":"Vase","size":12}Handling the missing values family
General
Different information models define different primitive value types, and the same goes for the notions of the "missing values" family:
- the empty value
-
the value is present but empty
- the non-existent value
-
the value is not present
- the undefined value
-
the value is not defined
There are also different ways to represent these missing values when the attribute accepts a single value or a collection of values.
| Technology | Missing value type | Realized as |
|---|---|---|
Lutaml::Model |
empty value |
Ruby empty string ( |
non-existent value |
Ruby |
|
undefined value |
|
|
XML element |
empty value |
XML blank element: |
non-existent value |
XML blank element with attribute |
|
undefined value |
the XML element is omitted |
|
XML attribute |
empty value |
XML blank attribute: |
non-existent value |
the XML attribute is omitted |
|
undefined value |
the XML attribute is omitted |
|
JSON |
empty value |
JSON empty string ( |
non-existent value |
JSON |
|
undefined value |
the JSON key is omitted |
|
TOML |
empty value |
TOML empty string |
non-existent value |
the TOML key is omitted since TOML does not support the concept of null. |
|
undefined value |
the TOML key is omitted |
|
Note
|
The Uninitialized class is a special Lutaml::Model construct, it is not
supported by normal Ruby objects.
|
The challenge for the developer is how to represent fully compatible semantics using interoperable data models across different technologies.
Lutaml::Model provides you with several mechanisms to retain the missing values semantics. An example mapping is shown in the following diagram.
┌─────────────────────────────────────────────────────────────────────────────┐
│ ╔════════════════════════╗ ╔════════════════════════╗ │
│ ║ Lutaml::Model Values ║ ║ YAML Values ║ │
│ ╠════════════════════════╣ ╠════════════════════════╣ │
│ ║ ║ mapping ║ ║ │
│ ║ "empty" ║◀──────────────────▶║ "empty" ║ │
│ ║ (empty string, []) ║ to empty ║ (empty string, []) ║ │
│ ║ ║ ║ ║ │
│ ╟────────────────────────╢ ╟────────────────────────╢ │
│ ║ ║ mapping ║ ║ │
│ ║ "non-existent" ║◀──────────────────▶║ "non-existent" ║ │
│ ║ (nil) ║ to non-existent ║ (null) ║ │
│ ║ ║ ║ ║ │
│ ╟────────────────────────╢ ╟────────────────────────╢ │
│ ║ ║ mapping ║ ║ │
│ ║ "undefined" ║◀──────────────────▶║ "undefined" ║ │
│ ║ (uninitialized) ║ to undefined ║ (key omitted) ║ │
│ ║ ║ ║ ║ │
│ ╚════════════════════════╝ ╚════════════════════════╝ │
└─────────────────────────────────────────────────────────────────────────────┘In the case where the interoperating technologies do not support the full spectrum of missing value types, it is necessary for the developer to understand any such behavior and relevant handling.
┌─────────────────────────────────────────────────────────────────────────────┐
│ ╔════════════════════════╗ ╔════════════════════════╗ │
│ ║ Lutaml::Model Values ║ ║ TOML Values ║ │
│ ╠════════════════════════╣ ╠════════════════════════╣ │
│ ║ ║ mapping ║ ║ │
│ ║ "empty" ║◀──────────────────▶║ "empty" ║ │
│ ║ (empty string, []) ║ to empty ║ (empty string, []) ║ │
│ ║ ║ ║ ║ │
│ ╟────────────────────────╢ ╟────────────────────────╢ │
│ ║ ║ mapping ║ ║ │
│ ║ "non-existent" ║◀──────────────────▶║ ║ │
│ ║ (nil) ║ to undefined ║ "undefined" ║ │
│ ║ ║ ║ (key omitted) ║ │
│ ╟────────────────────────╢ ║ ║ │
│ ║ ║ mapping ║ ║ │
│ ║ "undefined" ║─────(one-way)─────▶║ TOML does not ║ │
│ ║ (uninitialized) ║ to undefined ║ support NULL ║ │
│ ║ ║ ║ ║ │
│ ╚════════════════════════╝ ╚════════════════════════╝ │
└─────────────────────────────────────────────────────────────────────────────┘There are the following additional challenges that a developer must take into account of:
-
Single attribute value vs collection attribute value. Different technologies treat single/collection values differently.
-
External schemas and systems that interoperate with serializations from Lutaml::Model. Many schemas and systems adopt "different" conventions for representing missing value semantics (sometimes very awkward ones).
The solution for the first challenge is to understand the behavior of the different technologies used. The default mappings are described in Value representation in Lutaml::Model and Value representation in serialization formats.
Value representation in Lutaml::Model
The following table summarizes the behavior of the Lutaml::Model in regards of the "missing values" family.
| LutaML value type | Cardinality (1 or n) | Missing value type | Ruby value |
|---|---|---|---|
Collection attribute |
collection |
empty value |
|
non-existent value |
|
||
undefined value |
No assigned value |
||
|
single |
empty value |
|
non-existent value |
|
||
undefined value |
No assigned value |
||
|
single |
empty value |
N/A |
non-existent value |
|
||
undefined value |
No assigned value |
||
|
single |
empty value |
N/A |
non-existent value |
|
||
undefined value |
No assigned value |
||
|
single |
empty value |
N/A |
non-existent value |
|
||
undefined value |
No assigned value |
||
|
single |
empty value |
N/A |
non-existent value |
|
||
undefined value |
No assigned value |
||
|
single |
empty value |
N/A |
non-existent value |
|
||
undefined value |
No assigned value |
||
|
single |
empty value |
N/A |
non-existent value |
|
||
undefined value |
No assigned value |
||
|
single |
empty value |
N/A |
non-existent value |
|
||
undefined value |
No assigned value |
||
|
single |
empty value |
N/A |
non-existent value |
|
||
undefined value |
No assigned value |
||
|
single |
empty value |
|
non-existent value |
|
||
undefined value |
No assigned value |
Value representation in serialization formats
Every serialization format uses a different information model to represent these missing values.
Some serialization formats support all 3 types of missing values, while others only support a subset of them.
| Serialization format | Cardinality (1 or n) | Missing value type | Example |
|---|---|---|---|
XML |
collection |
empty collection |
the XML blank element: |
non-existent collection |
a blank element with attribute |
||
undefined collection |
the XML element is omitted |
||
single |
empty value |
the XML blank element: |
|
non-existent value |
a blank element with attribute |
||
undefined value |
the XML element is omitted |
||
JSON |
collection |
empty collection |
an empty array ( |
non-existent collection |
the value |
||
undefined collection |
the key is omitted |
||
single |
empty value |
an empty string ( |
|
non-existent value |
the value |
||
undefined value |
the key is omitted |
||
YAML |
collection |
empty collection |
an empty array ( |
non-existent collection |
the value |
||
undefined collection |
the key is omitted |
||
single |
empty value |
an empty string ( |
|
non-existent value |
the value |
||
undefined value |
the key is omitted |
||
TOML |
collection |
empty collection |
an empty array ( |
non-existent collection |
TOML does not support the concept of "null" |
||
undefined collection |
the key is omitted |
||
single |
empty value |
an empty string ( |
|
non-existent value |
TOML does not support the concept of "null" |
||
undefined value |
the key is omitted |
Missing value mapping
General
Lutaml::Model provides a comprehensive way to handle the missing values family across different serialization formats.
The value_map option as applied to serialization mapping rules allow users to
meticulously define how each and every missing value should be mapped from a
serialization format to a Lutaml::Model object.
The value_map option is used to define mappings for both from and to values:
-
frompairs -
A hash of key-value pairs that determines the mapping of a missing value at the serialization format ("from") to a LutaML Model missing value where this mapping applies. The key is the missing value type in the serialization format, and the value is the missing value type in the LutaML Model.
NoteIn other words, used when converting the serialized format into a Lutaml::Model Ruby object. -
topairs -
A hash of key-value pairs that determines the mapping of a LutaML Model ("to") missing to a missing value choice at the serialization format where this mapping applies. The key is the missing value type in the LutaML Model, and the value is the missing value type in the serialization format.
NoteIn other words, used when converting a Lutaml::Model Ruby object into the serialized format.
Syntax:
{map_command} 'format-key', to: :attribute_name, value_map: { (1)
from: {
{format-missing-value-n}: {model-missing-value-n}, (2)
{format-missing-value-m}: {model-missing-value-m},
{format-missing-value-o}: {model-missing-value-o}
},
to: {
{model-missing-value-n}: {format-missing-value-n}, (3)
{model-missing-value-m}: {format-missing-value-m},
{model-missing-value-o}: {format-missing-value-o}
}
}-
The
{map_command}is a mapping rule with the actual command depending on the serialization format. -
In the
frommapping, the keys are the missing value types in the serialization format. -
In the
tomapping, the keys are the missing value types in the LutaML Model.
The missing value type mapping differs per serialization format, as serialization formats may not fully support all missing value types.
The availability of from and to keys and values depend on the types of
missing values supported by that particular serialization format.
The available values for from and to for serialization formats
are presented below, where the allowed values are to be used in the direction
of the format. That means if the format supports :empty, it can be used
as a key in from: direction, and the value in the to: direction (see {format-missing-value-n}) in the syntax.
| Map command | Missing value types available (key in from: direction, value in the to: direction) |
|---|---|
XML element |
|
XML attribute |
|
Hash |
|
JSON |
|
YAML |
|
TOML |
|
nil cannot be used; therefore in a from:
value map, it is not possible to indicate nil: {model-missing-value}.
In an XML mapping block, it is possible to do the following.
xml do
map_element 'status', to: :status, value_map: {
from: { empty: :nil, omitted: :omitted, nil: :nil },
to: { empty: :nil, omitted: :omitted, nil: :nil }
}
endEach serialization format has specific behavior when handling values
such as empty, omitted, and nil.
Users can specify the mapping for both from and to values using the
value_map option in the attribute definition.
-
The keys that can be used in the
fromandtomappings areempty,omitted, andnil. -
The values in the mappings can also be
empty,omitted, andnil.
|
Note
|
Since nil is not supported in TOML, so mappings like nil:
{any_option} or {any_option}: :nil will not work in TOML.
|
|
Note
|
In a collection attribute, the values of value_map also depend on the
initialize_empty setting, where an omitted value in the serialization format can still lead to a nil
or an empty array [] at the attribute-level (instead of the mapping-level).
|
Default value maps for serialization formats
The table below describes the default value_map configurations for supported
serialization formats.
Default value map for XML element (single attribute)
attribute :attr, :string
xml do
map_element 'key', to: :attr, value_map: {
from: { empty: :nil, omitted: :omitted, nil: :nil },
to: { empty: :empty, omitted: :omitted, nil: :nil }
}
endDirection |
Map rule |
XML source |
Model target |
|---|---|---|---|
|
|
blank XML element ( |
|
|
absent XML element |
omitted from the model |
|
|
blank XML element with attribute |
|
|
Direction |
Map rule |
Model source |
XML target |
|
|
empty string ( |
blank XML element |
|
omitted in the model |
XML element not rendered |
|
|
|
blank XML element with attribute |
Default value map for XML element (collection attribute)
attribute :attr, :string, collection: true
xml do
map_element 'key', to: :attr, value_map: {
from: { empty: :empty, omitted: :omitted, nil: :nil },
to: { empty: :empty, omitted: :omitted, nil: :nil }
}
endDirection |
Map rule |
XML source |
Model target |
|---|---|---|---|
|
|
blank XML element ( |
empty array ( |
|
absent XML element |
omitted from the model |
|
|
blank XML element with attribute |
|
|
Direction |
Map rule |
Model source |
XML target |
|
|
empty array ( |
blank XML element |
|
omitted in the model |
XML element not rendered |
|
|
|
blank XML element with attribute |
Default value map for XML attribute (single attribute)
attribute :attr, :string
xml do
map_attribute 'attr_name', to: :attr, value_map: {
from: { empty: :nil, omitted: :omitted },
to: { empty: :empty, nil: :empty, omitted: :omitted }
}
endDirection |
Map rule |
XML source |
Model target |
|---|---|---|---|
|
|
blank XML attribute ( |
|
|
absent XML attribute |
omitted from the model |
|
Direction |
Map rule |
Model source |
XML target |
|
|
empty string ( |
blank XML attribute ( |
|
omitted in the model |
XML attribute not rendered |
|
|
|
blank XML attribute ( |
Default value map for XML attribute (collection attribute)
attribute :attr, :string, collection: true
xml do
map_attribute 'attr_name', to: :attr, value_map: {
from: { empty: :empty, omitted: :omitted },
to: { empty: :empty, nil: :omitted, omitted: :omitted }
}
endDirection |
Map rule |
XML source |
Model target |
|---|---|---|---|
|
|
blank XML attribute ( |
empty array ( |
|
absent XML attribute |
omitted from the model |
|
Direction |
Map rule |
Model source |
XML target |
|
|
empty array ( |
blank XML attribute ( |
|
omitted in the model |
XML attribute not rendered |
|
|
|
XML attribute not rendered |
Default value map for YAML (single attribute)
attribute :attr, :string
yaml do
map 'key', to: :attr, value_map: {
from: { empty: :empty, omitted: :omitted, nil: :nil },
to: { empty: :empty, omitted: :omitted, nil: :nil }
}
endDirection |
Map rule |
XML source |
Model target |
|---|---|---|---|
|
|
empty string in YAML ( |
empty string ( |
|
absent YAML key |
omitted from the model |
|
|
YAML |
|
|
Direction |
Map rule |
Model source |
XML target |
|
|
empty string ( |
empty string in YAML ( |
|
omitted in the model |
YAML key omitted |
|
|
|
YAML |
|
Note
|
In order to treat a YAML value like status: '' to nil, the mapping of
value_map: { from: { empty: :nil } } can be applied.
|
Default value map for YAML (collection attribute)
attribute :attr, :string, collection: true
yaml do
map 'key', to: :attr, value_map: {
from: { empty: :empty, omitted: :omitted, nil: :nil },
to: { empty: :empty, omitted: :omitted, nil: :nil }
}
endDirection |
Map rule |
YAML source |
Model target |
|---|---|---|---|
|
|
empty YAML array ( |
empty array ( |
|
absent YAML key |
omitted from the model |
|
|
YAML |
|
|
Direction |
Map rule |
Model source |
YAML target |
|
|
empty array ( |
empty YAML array ( |
|
omitted in the model |
YAML key omitted |
|
|
|
YAML |
|
Note
|
If the YAML key for the collection attribute is omitted, it will be treated
as nil or an empty array depending on the initialize_empty setting.
|
Default value map for JSON (single attribute)
attribute :attr, :string
json do
map 'key', to: :attr, value_map: {
from: { empty: :empty, omitted: :omitted, nil: :nil },
to: { empty: :empty, omitted: :omitted, nil: :nil }
}
endDirection |
Map rule |
JSON source |
Model target |
|---|---|---|---|
|
|
empty string in JSON ( |
empty string ( |
|
absent JSON key |
omitted from the model |
|
|
JSON |
|
|
Direction |
Map rule |
Model source |
JSON target |
|
|
empty string ( |
empty string in JSON ( |
|
omitted in the model |
JSON key omitted |
|
|
|
JSON |
Default value map for JSON (collection attribute)
attribute :attr, :string, collection: true
json do
map 'key', to: :attr, value_map: {
from: { empty: :empty, omitted: :omitted, nil: :nil },
to: { empty: :empty, omitted: :omitted, nil: :nil }
}
endDirection |
Map rule |
JSON source |
Model target |
|---|---|---|---|
|
|
empty JSON array ( |
empty array ( |
|
absent JSON key |
omitted from the model |
|
|
JSON |
|
|
Direction |
Map rule |
Model source |
JSON target |
|
|
empty array ( |
empty JSON array ( |
|
omitted in the model |
JSON key omitted |
|
|
|
JSON |
Default value map for TOML (single attribute)
TOML does not support the concept of nil and therefore the mapping of from:
direction with nil to will not work in TOML.
The nil mapping is only supported in the to: direction (model to TOML).
attribute :attr, :string
toml do
map 'key', to: :attr, value_map: {
from: { empty: :empty, omitted: :omitted },
to: { empty: :empty, omitted: :omitted, nil: :omitted }
}
endDirection |
Map rule |
TOML source |
Model target |
|---|---|---|---|
|
|
empty string in TOML ( |
empty string ( |
|
absent TOML key |
omitted from the model |
|
Direction |
Map rule |
Model source |
TOML target |
|
|
empty string ( |
empty string in TOML ( |
|
omitted in the model |
TOML key omitted |
|
|
|
TOML key omitted |
Default value map for TOML (collection attribute)
TOML does not support the concept of nil and therefore the mapping of from:
direction with nil to will not work in TOML.
The nil mapping is only supported in the to: direction (model to TOML).
attribute :attr, :string, collection: true
toml do
map 'key', to: :attr, value_map: {
from: { empty: :empty, omitted: :omitted },
to: { empty: :empty, omitted: :omitted, nil: :omitted }
}
endDirection |
Map rule |
TOML source |
Model target |
|---|---|---|---|
|
|
empty TOML array ( |
empty array ( |
|
absent TOML key |
omitted from the model |
|
Direction |
Map rule |
Model source |
TOML target |
|
|
empty array ( |
empty TOML array ( |
|
omitted in the model |
TOML key omitted |
|
|
|
TOML key omitted |
Replacing missing values type mapping with value_map
The value_map option can be defined to meticulously map for each serialization
format as follows.
value_map with from and to valuesclass ExampleClass < Lutaml::Model::Serializable
attribute :status, :string
xml do
map_element 'status', to: :status, value_map: {
from: { empty: :nil, omitted: :omitted, nil: :nil },
to: { empty: :nil, omitted: :omitted, nil: :nil }
}
end
hsh | json | yaml | toml | key_value do
map 'status', to: :status, value_map: {
from: { empty: :nil, omitted: :omitted, nil: :nil },
to: { empty: :nil, omitted: :omitted, nil: :nil }
}
end
end
yaml = <<~YAML
---
status: ''
YAML
ExampleClass.from_yaml(yaml)
# => #<ExampleClass:0x000000011954efb0 @status=nil>
yaml1 = <<~YAML
---
YAML
ExampleClass.from_yaml(yaml1)
# => #<ExampleClass:0x000000011954efb0 @status=uninitialized>
yaml2 = <<~YAML
---
status:
YAML
ExampleClass.from_yaml(yaml2)
# => #<ExampleClass:0x000000011954efb0 @status=nil>When defining an attribute with collection: true, the attribute will behave as follows:
attribute :status, :string, collection: trueHere’s an example of how you can use the value_map with a collection attribute.
value_map with a collection attributeclass ExampleClass < Lutaml::Model::Serializable
attribute :status, :string, collection: true
xml do
map_element 'status', to: :status, value_map: {
from: { empty: :nil, omitted: :omitted, nil: :nil },
to: { empty: :nil, omitted: :omitted, nil: :nil }
}
end
hsh | json | yaml | key_value do
map 'status', to: :status, value_map: {
from: { empty: :nil, omitted: :omitted, nil: :nil },
to: { empty: :nil, omitted: :omitted, nil: :nil }
}
end
toml do
map 'status', to: :status, value_map: {
from: { empty: :nil, omitted: :omitted },
to: { empty: :nil, omitted: :omitted, nil: :omitted }
}
end
end
yaml = <<~YAML
---
status: ['new', 'assigned']
YAML
y = ExampleClass.from_yaml(yaml)
# => #<ExampleClass:0x000000011954efb0 @status=["new", "assigned"]>Specific overrides of value map (render_* and treat_*)
General
There are times that one may want to simply override handling of selective missing value types rather than re-define the entire value map.
The :render_* and :treat_* options are simple switches that override the default
value map provided for the different serialization formats.
Syntax:
{map_command} 'format-key', to: :attribute_name, (1)
:render_{model-value}: :as_{format-value}, (2)
# ...
:treat_{format-value}: :as_{model-value}, (3)
# ...-
The
{map_command}is a mapping rule with the actual command depending on the serialization format. The attribute ofattribute_namemay be a single or a collection value. -
The
:render_*mapping overrides the default value map for missing value types in model-to-serialization. -
The
:treat_*mapping overrides the default value map for missing value types in serialization-to-model.
Specifically,
-
The
:render_{model-value}: :as_{format-value}options are used to override the default behavior of rendering missing value types into the serialization format.{model-value}-
specifies the missing value type in the LutaML Model.
{format-value}-
specifies the missing value type in the serialization format.
-
The
:treat_{format-value}: :as_{model-value}options are used to override the default behavior of importing missing value types into the model.{format-value}-
specifies the missing value type in the serialization format.
{model-value}-
specifies the missing value type in the LutaML Model.
In effect, the default value_map is overriden by the :render_* and :treat_*
directives.
Given the default mapping for an XML element, the :render_* and :treat_*
options can be used to selectively override behavior.
xml do
map_element 'key', to: :attr, value_map: {
from: { empty: :nil, omitted: :omitted, nil: :nil },
to: { empty: :empty, omitted: :omitted, nil: :nil }
}
endBy changing to this:
xml do
map_element 'key', to: :attr,
render_nil: :as_empty, (1)
treat_omitted: :as_nil (2)
end-
This overrides the
to:directionnil: :nilmapping. -
This overrides the
from:directionomitted: :omittedmapping
The resulting value map would be:
xml do
map_element 'status', to: :status, value_map: {
from: { empty: :nil, omitted: :omitted, nil: :empty }, (1)
to: { empty: :nil, omitted: :nil, nil: :nil } (2)
}
end-
See that
nil: :nilis nownil: :empty. -
See that
omitted: :omittedis nowomitted: :nil
render_nil
General
:render_nil is a specially handled case of the :render_* pattern due to
legacy. It is used to override default value map behavior for the nil model
value.
render_nil accepts these values:
-
:as_empty/:as_blank -
if the value is nil, render it as an empty string.
:nil-
if the value is nil, render it as an element with attribute
xsi:nil. :omit-
if the value is nil, omit the element or attribute.
true-
(legacy) setting
render_nil: truewill render the attribute as an empty element if the attribute isnil. This has the same effect asrender_nil: :as_empty.
Syntax:
xml do
map_element 'key_value_model_attribute_name', to: :name_of_attribute, render_nil: {option}
endhsh | json | yaml | toml do
map 'key_value_model_attribute_name', to: :name_of_attribute, render_nil: {option}
endRender nil as true
render_nil: true option to render an attribute value of nil as an empty elementclass Ceramic < Lutaml::Model::Serializable
attribute :type, :string
attribute :glaze, :string
xml do
map_element 'type', to: :type, render_nil: true
map_element 'glaze', to: :glaze
end
json do
map 'type', to: :type, render_nil: true
map 'glaze', to: :glaze
end
end> Ceramic.new.to_json
> # { 'type': null }
> Ceramic.new(type: "Porcelain", glaze: "Clear").to_json
> # { 'type': 'Porcelain', 'glaze': 'Clear' }> Ceramic.new.to_xml
> # <Ceramic><type></type></Ceramic>
> Ceramic.new(type: "Porcelain", glaze: "Clear").to_xml
> # <Ceramic><type>Porcelain</type><glaze>Clear</glaze></Ceramic>render_nil: true option to render an empty attribute collection of nil as an empty elementclass Ceramic < Lutaml::Model::Serializable
attribute :type, :string
attribute :glazes, :string, collection: true
xml do
map_element 'type', to: :type, render_nil: true
map_element 'glazes', to: :glazes, render_nil: true
end
json do
map 'type', to: :type, render_nil: true
map 'glazes', to: :glazes, render_nil: true
end
end> Ceramic.new.to_json
> # { 'type': null, 'glazes': [] }
> Ceramic.new(type: "Porcelain", glazes: ["Clear"]).to_json
> # { 'type': 'Porcelain', 'glazes': ['Clear'] }> Ceramic.new.to_xml
> # <Ceramic><type></type><glazes></glazes></Ceramic>
> Ceramic.new(type: "Porcelain", glazes: ["Clear"]).to_xml
> # <Ceramic><type>Porcelain</type><glazes>Clear</glazes></Ceramic>Render nil as omit
Using render_nil: :omit with a nil value will omit the key from XML and
key-value formats.
class SomeModel < Lutaml::Model::Serializable
attribute :coll, :string, collection: true
xml do
root "some-model"
map_element 'collection', to: :coll, render_nil: :omit
end
key_value do
map 'collection', to: :coll, render_nil: :omit
end
end
puts SomeModel.new.coll
# => nil
puts SomeModel.new.to_xml
# =>
# <some-model></some-model>
puts SomeModel.new.to_yaml
# =>
# ---Render nil as nil
Using render_nil: :as_nil with a nil value will create an empty element with
xsi:nil attribute in XML and create a key with explicit null value in
key-value formats.
|
Note
|
TOML does not support this option.
|
class SomeModel < Lutaml::Model::Serializable
attribute :coll, :string, collection: true
xml do
root "some-model"
map_element 'collection', to: :coll, render_nil: :as_nil
end
hsh | json | yaml do
map 'collection', to: :coll, render_nil: :as_nil
end
end
puts SomeModel.new.coll
# => nil
puts SomeModel.new.to_xml
# =>
# <some-model xsi:xmlns="..."><collection xsi:nil="true"/></some-model>
puts SomeModel.new.to_yaml
# =>
# ---
# coll: nullRender nil as blank
Using render_nil: :as_blank | :as_empty will create a blank element in XML and
create a key with an explicit empty array in key-value formats.
class SomeModel < Lutaml::Model::Serializable
attribute :coll, :string, collection: true
xml do
root "some-model"
map_element 'collection', to: :coll, render_nil: :as_blank
end
key_value do
map 'collection', to: :coll, render_nil: :as_empty
end
end
puts SomeModel.new.coll
# => nil
puts SomeModel.new.to_xml
# =>
# <some-model><collection/></some-model>
puts SomeModel.new.to_yaml
# =>
# ---
# coll: []render_empty
General
:render_empty is a specially handled case of the :render_* pattern due to
legacy. It is used to override default value map behavior for the nil model
value.
render_empty accepts these values:
-
:as_empty/:as_blank -
if the value is nil, render it as an empty string.
:nil-
if the value is nil, render it as an element with attribute
xsi:nil. :omit-
if the value is nil, omit the element or attribute.
Syntax:
xml do
map_element 'key_value_model_attribute_name', to: :name_of_attribute, render_empty: {option}
endhsh | json | yaml | toml do
map 'key_value_model_attribute_name', to: :name_of_attribute, render_empty: {option}
endRender empty as omit
Using render_empty: :omit with an empty value or empty collection will omit
the key from XML and key-value formats.
class SomeModel < Lutaml::Model::Serializable
attribute :coll, :string, collection: true
xml do
root "some-model"
map_element 'collection', to: :coll, render_empty: :omit
end
key_value do
map 'collection', to: :coll, render_empty: :omit
end
end
puts SomeModel.new(coll: []).coll
# => []
puts SomeModel.new.to_xml
# =>
# <some-model></some-model>
puts SomeModel.new.to_yaml
# =>
# ---Render empty as nil
Using render_empty: :as_nil will create an empty element with the xsi:nil
attribute in XML, and create a key with explicit null value in key-value formats.
|
Note
|
TOML does not support this option.
|
class SomeModel < Lutaml::Model::Serializable
attribute :coll, :string, collection: true
xml do
root "some-model"
map_element 'collection', to: :coll, render_empty: :as_nil
end
hsh | json | yaml do
map 'collection', to: :coll, render_empty: :as_nil
end
end
puts SomeModel.new(coll: []).coll
# => []
puts SomeModel.new.to_xml
# =>
# <some-model xsi:xmlns="..."><collection xsi:nil="true"/></some-model>
puts SomeModel.new.to_yaml
# =>
# ---
# coll: nullRender empty as blank/empty
Using render_empty: :as_blank or render_empty: :as_empty will create a blank
element in XML and create a key with an explicit empty array in key-value
formats.
class SomeModel < Lutaml::Model::Serializable
attribute :coll, :string, collection: true
xml do
root "some-model"
map_element 'collection', to: :coll, render_empty: :as_blank
end
key_value do
map 'collection', to: :coll, render_empty: :as_empty
end
end
puts SomeModel.new(coll: []).coll
# => []
puts SomeModel.new.to_xml
# =>
# <some-model><collection/></some-model>
puts SomeModel.new.to_yaml
# =>
# ---
# coll: []Parent and root context accessors (parent, root)
Models created by transforms can access their immediate parent and the document root via two special accessors added:
-
__parent:: The direct parent model instance -
__root:: The top-level/root model instance
These are assigned during deserialization so nested child models can reference context.
class Tag < Lutaml::Model::Serializable
attribute :text, :string
xml do
root "Tag"
map_content to: :text
end
key_value do
map "text", to: :text
end
end
class Tags < Lutaml::Model::Serializable
attribute :tag, Tag, collection: true
xml do
root "Tags"
map_element "Tag", to: :tag
end
key_value do
map "Tag", to: :tag
end
end
class Post < Lutaml::Model::Serializable
attribute :tags, Tags, collection: true
xml do
root "Post"
map_element "Tags", to: :tags
end
key_value do
map "Tags", to: :tags
end
end
post = Post.from_xml(<<~XML)
<Post>
<Tags>
<Tag>ruby</Tag>
<Tag>coding</Tag>
</Tags>
<Tags>
<Tag>JSON</Tag>
<Tag>coding</Tag>
</Tags>
</Post>
XML
first_tag = post.tags.first.tag.first # #<Tag:0x00000002e661a650 @text="ruby">
first_tag.__parent # #<Tags:0x00000002e5e90010 @tag=[#<Tag:0x00000002e5e5bb80 @text="ruby">, #<Tag:0x00000002e5e58ca0 @text="coding">]>
first_tag.__root # should return the `post` object
# Key-value formats also set these accessors
yaml = {
"Tags" => [
{
"Tag" => [
{ "text" => "ruby" },
{ "text" => "coding"}
]
},
{
"Tag" => [
{ "text" => "JSON" },
{ "text" => "coding" }
]
}
]
}.to_yaml
post2 = Post.from_yaml(yaml)
# Same as explained in XML `post` example
post2.tags.first.__parent
post2.tags.first.__root|
Note
|
If you manually instantiate models, you can set these accessors yourself if needed: |
child.__parent = parent
child.__root = parent.__root || parentSchema generation and import
Schema generation
Lutaml::Model provides functionality to generate schema definitions from LutaML models. This allows you to create schemas that can be used for validation or documentation purposes.
Currently, the following schema formats are supported:
-
JSON Schema (JSON Schema)
-
YAML Schema (YAML)
JSON Schema generation
The Lutaml::Model::Schema.to_json method generates a JSON Schema from a LutaML model class. The generated schema includes:
-
Properties based on model attributes
-
Validation constraints (patterns, enumerations, etc.)
-
Support for polymorphic types
-
Support for inheritance
-
Support for choice attributes
-
Collection constraints
Example:
class Glaze < Lutaml::Model::Serializable
attribute :color, :string
attribute :finish, :string
end
class Vase < Lutaml::Model::Serializable
attribute :height, :float
attribute :diameter, :float
attribute :glaze, Glaze
attribute :materials, :string, collection: true
end
# Generate JSON schema
schema = Lutaml::Model::Schema.to_json(
Vase,
id: "https://example.com/vase.schema.json",
description: "A vase schema",
pretty: true
)
# Write to file
File.write("vase.schema.json", schema)The generated schema will include definitions for all nested models and their attributes.
YAML Schema generation
The Lutaml::Model::Schema.to_yaml method generates a YAML Schema from a LutaML model class. The generated schema includes the same features as the JSON Schema generation.
Example:
class Glaze < Lutaml::Model::Serializable
attribute :color, :string
attribute :finish, :string
end
class Vase < Lutaml::Model::Serializable
attribute :height, :float
attribute :diameter, :float
attribute :glaze, Glaze
attribute :materials, :string, collection: true
end
# Generate YAML schema
schema = Lutaml::Model::Schema.to_yaml(
Vase,
id: "http://example.com/schemas/vase",
description: "A vase schema",
pretty: true
)
# Write to file
File.write("vase.schema.yaml", schema)Importing data models
Lutaml::Model provides a way to import data models defined from various formats into the LutaML data modeling system.
Data model languages supported are:
The following figure illustrates the process of importing an XML Schema model to create LutaML core models. Once the LutaML core models are created, they can be used to parse and generate XML documents according to the imported XML Schema model.
Today, the LutaML core models are written into Ruby files, which can be used to parse and generate XML documents according to the imported XML Schema. This is to be changed so that the LutaML core models are directly loaded and interpreted.
╔════════════════════════════╗ ╔═══════════════════════╗
║ Serialization Models ║ ║ Core Model ║
╚════════════════════════════╝ ╚═══════════════════════╝
╭┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╮ ╭┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╮
┆ XML Schema (XSD/RNG/RNC) ┆ ┆ Model ┆
┆ │ ┆ ┌────────────────┐ ┆ │ ┆
┆ ┌──────┴──────┐ ┆ │ │ ┆ ┌────────┴──┐ ┆
┆ │ │ ┆ │ Model │ ┆ │ │ ┆
┆ Models Value Types ┆──►│ Importing │──►┆ Models Value Types ┆
┆ │ │ ┆ │ │ ┆ │ │ ┆
┆ │ │ ┆ └────────────────┘ ┆ │ │ ┆
┆ ┌────┴────┐ ┌─┴─┐ ┆ │ ┆ │ ┌──────┴──┐ ┆
┆ │ │ │ │ ┆ │ ┆ │ │ │ ┆
┆ Element Value xs:string ┆ │ ┆ │ String Integer ┆
┆ Attribute Type xs:date ┆ │ ┆ │ Date Float ┆
┆ Union Complex xs:boolean ┆ │ ┆ │ Time Boolean ┆
┆ Sequence Choice xs:anyURI ┆ │ ┆ │ ┆
╰┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╯ │ ┆ └──────┐ ┆
│ ┆ │ ┆
│ ┆ Contains ┆
│ ┆ more Models ┆
│ ┆ (recursive) ┆
│ ┆ ┆
│ ╰┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╯
│ ┌────────────────┐
│ │ │
│ │ Model │
└──────────► │ Transformation │
│ & │
│ Mapping Rules │
│ │
└────────────────┘XML Schema (XSD)
W3C XSD is a schema language designed to define the structure of XML documents, alongside other XML schema languages like DTD, RELAX NG, and Schematron.
Lutaml::Model supports the import of XSD schema files to define information models that can be used to parse and generate XML documents.
Specifically, the Lutaml::Model::Schema#from_xml method loads XML Schema files
(XSD, .xsd) and generates Ruby files (.rb) that inherit from
Lutaml::Model::Serializable that are saved to disk.
Syntax:
Lutaml::Model::Schema.from_xml(
xsd_schema, (1)
options: options (2)
)-
The
xsd_schemais the XML Schema string to be converted to model files. -
The
optionshash is an optional argument.options-
Optional hash containing potentially the following key-values.
output_dir-
The directory where the model files will be saved. If not provided, a default directory named
lutaml_models_<timestamp>is created."path/to/directory" create_files-
A
booleanargument (falseby default) to create files directly in the specified directory as defined by theoutput_diroption.create_files: (true | false) load_classes-
A
booleanargument (falseby default) to load generated classes before returning them.load_classes: (true | false) namespace-
The namespace of the schema. This will be added in the
Lutaml::Model::Serializablefile’sxml doblock. prefix-
The prefix of the namespace provided in the
namespaceoption.example-prefix location-
The URL or path of the directory containing all the files of the schema. For more information, refer to the XML Schema specification.
"http://example.com/example.xsd""path/to/schema/directory"
|
Note
|
If both create_files and load_classes are provided, the create_files
argument will take priority and generate files without loading them!
|
The generated LutaML models consists of two different kind of Ruby classes depending on the XSD schema:
- XSD "SimpleTypes"
-
converted into classes that inherit from
Lutaml::Model::Type::Value, which define the data types with restrictions and other validations of these values. - XSD "ComplexTypes"
-
converted into classes that inherit from
Lutaml::Model::Serializablethat model according to the defined structure.
Lutaml::Model uses the lutaml-xsd gem to
automatically resolve the include and import elements, enabling
Lutaml-Model to generate the corresponding model files.
This auto-resolving feature allows seamless integration of these files into your models without the need for manual resolution of includes and imports.
Lutaml::Model::Schema#from_xml to convert an XML Schema to model filesxsd_schema = <<~XSD
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
/* your schema here */
</xs:schema>
XSD
options = {
# These are all optional:
output_dir: 'path/to/directory',
namespace: 'http://example.com/namespace',
prefix: "example-prefix",
location: "http://example.com/example.xsd",
# or
# location: "path/to/schema/directory"
create_files: true, # Default: false
# OR
load_classes: true, # Default: false
}
# generates the files in the output_dir | default_dir
Lutaml::Model::Schema.from_xml(xsd_schema, options: options)You could also directly load the generated Ruby files into your application by requiring them.
Lutaml::Model::Schema.from_xml(xsd_schema, options: {output_dir: 'path/to/directory'})
require_relative 'path/to/directory/*.rb'Validation
General
Lutaml::Model provides a way to validate data models using the validate and
validate! methods.
-
The
validatemethod sets anerrorsarray in the model instance that contains all the validation errors. This method is used for checking the validity of the model silently. -
The
validate!method raises aLutaml::Model::ValidationErrorthat contains all the validation errors. This method is used for forceful validation of the model through raising an error.
Lutaml::Model supports the following validation methods:
-
collection:: Validates collection size range. -
values:: Validates the value of an attribute from a set of fixed values. -
choice:: Validates that attribute specified within defined range
The following class will validate the name is present and degree_settings attributes to ensure that
it has at least one element and that the description attribute is one of the
values in the set [one, two, three].
class Klin < Lutaml::Model::Serializable
attribute :name, :string, required: true
attribute :degree_settings, :integer, collection: (1..)
attribute :description, :string, values: %w[one two three]
attribute :id, :integer
attribute :age, :integer
choice(min: 1, max: 1) do
choice(min: 1, max: 2) do
attribute :prefix, :string
attribute :forename, :string
end
attribute :nick_name, :string
end
xml do
map_element 'name', to: :name
map_attribute 'degree_settings', to: :degree_settings
end
end
klin = Klin.new(name: "Klin", degree_settings: [100, 200, 300], description: "one", prefix: "Ben")
klin.validate
# => []
klin = Klin.new(name: "Klin", degree_settings: [], description: "four", prefix: "Ben", nick_name: "Smith")
klin.validate
# => [
# #<Lutaml::Model::CollectionSizeError: degree_settings must have at least 1 element>,
# #<Lutaml::Model::ValueError: description must be one of [one, two, three]>,
# #<Lutaml::Model::ChoiceUpperBoundError: Attribute count exceeds the upper bound>
# ]
e = klin.validate!
# => Lutaml::Model::ValidationError: [
# degree_settings must have at least 1 element,
# description must be one of [one, two, three],
# Attribute count exceeds the upper bound
# ]
e.errors
# => [
# #<Lutaml::Model::CollectionSizeError: degree_settings must have at least 1 element>,
# #<Lutaml::Model::ValueError: description must be one of [one, two, three]>,
# #<Lutaml::Model::ChoiceUpperBoundError: Attribute count exceeds the upper bound>
# #<Lutaml::Model::ChoiceLowerBoundError: Attribute count is less than lower bound>
# ]
klin = Klin.new(degree_settings: [100, 200, 300], description: "one", prefix: "Ben")
klin.validate
# => [
# #<Lutaml::Model::RequiredAttributeMissingError: Missing required attribute: name>
# ]Custom validation
To add custom validation, override the validate method in the model class.
Additional errors should be added to the errors array.
The following class validates the degree_settings attribute when the type is
glass to ensure that the value is less than 1300.
class Klin < Lutaml::Model::Serializable
attribute :name, :string
attribute :type, :string, values: %w[glass ceramic]
attribute :degree_settings, :integer, collection: (1..)
def validate
errors = super
if type == "glass" && degree_settings.any? { |d| d > 1300 }
errors << Lutaml::Model::Error.new("Degree settings for glass must be less than 1300")
end
end
end
klin = Klin.new(name: "Klin", type: "glass", degree_settings: [100, 200, 1400])
klin.validate
# => [#<Lutaml::Model::Error: Degree settings for glass must be less than 1300>]Liquid template access
General
|
Warning
|
The Liquid template feature is optional. To enable it, please
explicitly require the liquid gem.
|
The Liquid template language is an open-source template language developed by Shopify and written in Ruby.
Lutaml::Model::Serializable objects can be safely accessed within Liquid
templates through a to_liquid method that converts the objects into
Liquid::Drop instances.
-
All attributes are accessible in the Liquid template by their names.
-
Nested attributes are also converted into
Liquid::Dropobjects so inner attributes can be accessed using the Liquid dot notation.
|
Note
|
Every Lutaml::Model::Serializable class extends the Liquefiable module
which generates a corresponding Liquid::Drop class.
|
|
Note
|
Methods defined in the Lutaml::Model::Serializable class are not
accessible in the Liquid template.
|
to_liquid to convert model instances into corresponding Liquid drop instancesclass Ceramic < Lutaml::Model::Serializable
attribute :name, :string
attribute :temperature, :integer
end
ceramic = Ceramic.new({ name: "Porcelain Vase", temperature: 1200 })
ceramic_drop = ceramic.to_liquid
# Ceramic::CeramicDrop
puts ceramic_drop.name
# "Porcelain Vase"
puts ceramic_drop.temperature
# 1200class Ceramic < Lutaml::Model::Serializable
attribute :name, :string
attribute :temperature, :integer
end
class CeramicCollection < Lutaml::Model::Serializable
attribute :ceramics, Ceramic, collection: true
endsample.yml:
---
ceramics:
- name: Porcelain Vase
temperature: 1200
- name: Earthenware Pot
temperature: 950
- name: Stoneware Jug
temperature: 1200template.liquid:
{% for ceramic in ceramic_collection.ceramics %}
* Name: "{{ ceramic.name }}"
** Temperature: {{ ceramic.temperature }}
{%- endfor %}# Load the Lutaml::Model collection
ceramic_collection = CeramicCollection.from_yaml(File.read("sample.yml"))
# Load the Liquid template
template = Liquid::Template.parse(File.read("template.liquid"))
# Pass the Lutaml::Model collection to the Liquid template and render
output = template.render("ceramic_collection" => ceramic_collection)
puts output
# >
# * Name: "Porcelain Vase"
# ** Temperature: 1200
# * Name: "Earthenware Pot"
# ** Temperature: 950
# * Name: "Stoneware Jug"
# ** Temperature: 1200class Glaze < Lutaml::Model::Serializable
attribute :color, :string
attribute :opacity, :string
end
class CeramicWork < Lutaml::Model::Serializable
attribute :name, :string
attribute :glaze, Glaze
end
class CeramicCollection < Lutaml::Model::Serializable
attribute :ceramics, Ceramic, collection: true
end
ceramic_work = CeramicWork.new({
name: "Celadon Bowl",
glaze: Glaze.new({
color: "Jade Green",
opacity: "Translucent"
})
})
ceramic_work_drop = ceramic_work.to_liquid
# CeramicWork::CeramicWorkDrop
puts ceramic_work_drop.name
# "Celadon Bowl"
puts ceramic_work_drop.glaze.color
# "Jade Green"
puts ceramic_work_drop.glaze.opacity
# "Translucent"ceramics.yml:
---
ceramics:
- name: Celadon Bowl
glaze:
color: Jade Green
opacity: Translucent
- name: Earthenware Pot
glaze:
color: Rust Red
opacity: Opaque
- name: Stoneware Jug
glaze:
color: Cobalt Blue
opacity: Transparenttemplates/_ceramics.liquid:
{% for ceramic in ceramic_collection.ceramics %}
{% render 'ceramic' ceramic: ceramic %}
{%- endfor %}|
Note
|
render is a Liquid tag that renders a partial template, by default
Liquid uses the pattern _%s.liquid to find the partial template. Here
ceramic refers to the file at templates/_ceramic.liquid.
|
templates/_ceramic.liquid:
* Name: "{{ ceramic.name }}"
** Temperature: {{ ceramic.temperature }}
{%- if ceramic.glaze %}
** Glaze (color): {{ ceramic.glaze.color }}
** Glaze (opacity): {{ ceramic.glaze.opacity }}
{%- endif %}require 'liquid'
# Create a Liquid template object that supports dynamic loading
template = Liquid::Template.new
# Link the Liquid template object to a "local file system" (directory)
file_system = Liquid::LocalFileSystem.new('templates/')
template.registers[:file_system] = file_system
# Load the partial template, this is necessary.
# This will also allow Liquid to load any inner partials from the file system
# dynamically (see `file_system.pattern` to see what it loads)
template.parse(file_system.read_template_file('ceramics'))
# Read the lutaml-model collection
ceramic_collection = CeramicCollection.from_yaml(File.read("ceramics.yml"))
# Render the template with the collection
output = template.render("ceramic_collection" => ceramic_collection)
puts output
# >
# * Name: "Celadon Bowl"
# ** Temperature: 1200
# ** Glaze (color): Jade Green
# ** Glaze (finish): Translucent
# * Name: "Earthenware Pot"
# ** Temperature: 950
# ** Glaze (color): Rust Red
# ** Glaze (finish): Opaque
# * Name: "Stoneware Jug"
# ** Temperature: 1200
# ** Glaze (color): Cobalt Blue
# ** Glaze (finish): TransparentAutomatic attribute access
All model attributes are automatically available in liquid drops without any additional configuration.
class User < Lutaml::Model::Serializable
attribute :name, :string
attribute :email, :string
attribute :age, :integer
end
user = User.new(name: "John", email: "john@example.com", age: 30)
drop = user.to_liquid
# All attributes work directly in templates
template = Liquid::Template.parse("{{name}} ({{email}}) is {{age}} years old")
result = template.render(drop)
# => "John (john@example.com) is 30 years old"Custom attribute mapping
You can define custom methods for your Liquid Drop classes and map them to specific keys in templates.
All model attributes are automatically available in liquid drops by default using the same name.
Use the liquid block to define custom mappings:
class Product < Lutaml::Model::Serializable
attribute :name, :string
attribute :price, :decimal
attribute :description, :string
liquid do
map "display_name", to: :formatted_name
map "price_with_currency", to: :formatted_price
end
def formatted_name
name.upcase
end
def formatted_price
"$#{price}"
end
end
product = Product.new(
name: "Laptop",
price: 999.99,
description: "High-performance laptop for professional use"
)
drop = product.to_liquid
# All attributes are automatically available
template = Liquid::Template.parse("{{name}} - {{price}} - {{description}}")
result = template.render(drop)
# => "Laptop - 999.99 - High-performance laptop for professional use"
# Use mapped methods in templates
template = Liquid::Template.parse("{{display_name}} costs {{price_with_currency}}")
result = template.render(drop)
# => "LAPTOP costs $999.99"Liquid Drop class inheritance
For advanced customization, you can create custom Liquid::Drop classes that inherit from the default base Liquid::Drop class auto-generated by Lutaml::Model.
|
Note
|
When creating custom drop methods, use |
class Schema < Lutaml::Model::Serializable
attribute :path, :string
attribute :source, :string
# Specify the name of your custom drop class
liquid_class "CustomSchemaDrop"
# You can still use liquid mappings
liquid do
map "template_path", to: :build_template_path
end
def build_template_path
File.join("templates", path)
end
end
# Get the base drop class for inheritance
BaseDropClass = Schema.to_liquid_class
# Create your custom drop class that inherits from the base
class CustomSchemaDrop < BaseDropClass
# Add new methods not in the original drop
def formatted_source
"Source: #{@object.source.upcase}"
end
# Override existing methods
def path
"custom/#{@object.path}"
end
endSophisticated inheritance hierarchies are supported:
# Base model
class Document < Lutaml::Model::Serializable
attribute :title, :string
attribute :content, :string
liquid_class "DocumentDrop"
end
# Get the base drop class
BaseDocumentDrop = Document.to_liquid_class
# Create a specialized drop class
class DocumentDrop < BaseDocumentDrop
def word_count
@object.content.split.size
end
def summary
@object.content
end
def metadata
{
title: @object.title,
word_count: word_count,
char_count: @object.content.length
}
end
end
# Subclass for specific document types
class TechnicalDocument < Document
attribute :version, :string
attribute :author, :string
liquid_class "TechnicalDocumentDrop"
end
# Get the base drop class for technical documents
BaseTechnicalDrop = TechnicalDocument.to_liquid_class
# Create specialized drop for technical documents
class TechnicalDocumentDrop < BaseTechnicalDrop
def version_info
"Version #{@object.version} by #{@object.author}"
end
def technical_summary
"#{summary(50)} [#{version_info}]"
end
# Override parent method
def metadata
super.merge(
version: @object.version,
author: @object.author,
type: 'technical'
)
end
endUsing custom Liquid Drop classes
It is straightforward to use your custom Liquid Drop classes as a full replacement of the default drop class.
schema = Schema.new(path: "config.xml", source: "database settings")
drop = schema.to_liquid
# The drop is now an instance of CustomSchemaDrop
puts drop.class # => CustomSchemaDrop
puts drop.is_a?(CustomSchemaDrop) # => true
# All attributes are automatically accessible
puts drop.path # => "custom/config.xml" (overridden)
puts drop.source # => "database settings" (original attribute)
# New methods work
puts drop.formatted_source # => "Source: DATABASE SETTINGS"
# Liquid mappings still work
puts drop.template_path # => "templates/config.xml"
# Use in templates
template = Liquid::Template.parse("""
Path: {{path}}
Source: {{formatted_source}}
Template: {{template_path}}
""")
result = template.render(drop)Error handling
When using custom liquid classes, you may encounter the following error:
LiquidClassNotFoundError
This error is raised when a custom liquid class specified with liquid_class is not defined or loaded in memory when to_liquid is called.
class Schema < Lutaml::Model::Serializable
attribute :path, :string
# Specify a custom drop class name
liquid_class "CustomSchemaDrop"
end
# This will raise LiquidClassNotFoundError if CustomSchemaDrop is not defined
schema = Schema.new(path: "config.xml")
schema.to_liquid
# => Lutaml::Model::LiquidClassNotFoundError: Liquid class 'CustomSchemaDrop' is not defined in memory. Please ensure the class is loaded before using it.To fix this error, ensure that your custom drop class is defined before calling to_liquid:
# Define the base drop class first
BaseDropClass = Schema.to_liquid_class
# Define your custom drop class
class CustomSchemaDrop < BaseDropClass
def formatted_path
"Path: #{@object.path}"
end
end
# Now to_liquid will work correctly
schema = Schema.new(path: "config.xml")
drop = schema.to_liquid # Works!
puts drop.class # => CustomSchemaDropSerialization adapters
General
The LutaML component that serializes a model into a serialization format is called an adapter. A serialization format may be supported by multiple adapters.
An adapter typically:
-
supports a specific serialization format
-
provides a set of methods to serialize and deserialize models and collections of models
LutaML, out of the box, supports the following serialization formats:
-
XML (W3C XML Schema (Second Edition), XML 1.0)
-
YAML (YAML version 1.2)
-
JSON (ECMA-404 The JSON Data Interchange Standard, unofficial link: JSON)
-
TOML (TOML version 1.0)
The adapter interface is also used to support certain transformation of models
into an "end format", which is not a serialization format. For example, the
Lutaml::Model::Hash is used to convert a model into a hash format
that is not a serialization format.
Users can extend LutaML by creating custom adapters for other serialization formats or for other data formats. The Custom Adapters Guide describes this process in detail.
For certain serialization formats, LutaML provides multiple adapters to support different serialization libraries. Please refer to their specific sections for more information.
Configuration
General
It is necessary to configure the adapter to be used for serialization and deserialization for a set of formats that the LutaML models will be transformed into.
There are two cases where you need to define such configuration:
-
End-user usage of the LutaML models. This is the case where you are using LutaML models in your application and want to serialize them into a specific format. If you are a gem developer that relies on lutaml-model, this case does not apply to you, because the end-user of your gem should determine the adapter configuration.
-
Testing purposes, e.g. RSpec. In order to run tests that involve verifying correctness of serialization, it is necessary to define adapter configuration.
There are two ways to specify a configuration:
-
by providing a predefined symbol (preferred)
-
by providing the actual adapter classes
There is a default configuration for adapters for commonly used formats:
-
YAML:
yaml_adapter_typeis set to:standard_yaml -
JSON:
json_adapter_typeis set to:standard_json -
Hash:
hash_adapter_typeis set to:standard_hash -
XML: not defined
-
TOML: not defined
Configure adapters through symbol choices
The end-user or a gem developer can copy and paste the following configuration into an early loading file in their application or gem.
This configuration is preferred over the class choices because it is more
concise and does not require any require code specific to the internals
of the LutaML runtime implementation.
Syntax:
require 'lutaml/model'
Lutaml::Model::Config.configure do |config|
config.xml_adapter_type = :nokogiri # can be one of [:nokogiri, :ox, :oga]
config.hash_adapter_type = :standard_hash
config.yaml_adapter_type = :standard_yaml
config.json_adapter_type = :standard_json # can be one of [:standard_json, :multi_json, :oj]
config.toml_adapter_type = :toml_rb # can be one of [:toml_rb, :tomlib]
endConfigure adapters through class choices
The end-uesr or a gem developer can copy and paste the following configuration into an early loading file in their application or gem.
Only the serialization formats used will require a configuration.
Syntax:
require 'lutaml/model'
require 'lutaml/model/xml/nokogiri_adapter'
require 'lutaml/model/hash/standard_adapter'
require 'lutaml/model/json/standard_adapter'
require 'lutaml/model/yaml/standard_adapter'
require 'lutaml/model/toml/toml_rb_adapter'
Lutaml::Model::Config.configure do |config|
config.xml_adapter = Lutaml::Model::Xml::NokogiriAdapter
config.hash_adapter = Lutaml::Model::Hash::StandardAdapter
config.yaml_adapter = Lutaml::Model::Yaml::StandardAdapter
config.json_adapter = Lutaml::Model::Json::StandardAdapter
config.toml_adapter = Lutaml::Model::Toml::TomlRbAdapter
endXML
Lutaml::Model supports the following XML adapters:
- Nokogiri
-
(default) Popular
libxmlbased XML parser for Ruby. Requires native extensions (i.e. compiled C code). Requires thenokogirigem. - Oga
-
(optional) Pure Ruby XML parser. Does not require native extensions and is suitable for Opal (Ruby on JavaScript). Requires the
ogagem. - Ox
-
(optional) Fast XML parser and object serializer for Ruby, implemented partially in C. Requires native extensions (i.e. compiled C code). Requires the
oxgem.
require 'lutaml/model'
Lutaml::Model::Config.configure do |config|
config.xml_adapter = :nokogiri
# or
config.xml_adapter = :oga
# or
config.xml_adapter = :ox
endYAML
Lutaml::Model supports only one YAML adapter.
- YAML
-
(default) The Psych YAML parser and emitter for Ruby. Included in the Ruby standard library.
require 'lutaml/model'
Lutaml::Model::Config.configure do |config|
config.yaml_adapter = :standard_yaml
endJSON
Lutaml::Model supports the following JSON adapters:
- JSON
-
(default) The standard JSON library for Ruby. Included in the Ruby standard library.
- MultiJson
-
(optional) A gem that provides a common interface to multiple JSON libraries. Requires the
multi_jsongem. - Oj
-
(optional) A fast JSON parser and Object marshaller as a Ruby gem. Requires the
ojgem.
require 'lutaml/model'
Lutaml::Model::Config.configure do |config|
config.json_adapter = :standard_json
# or
config.json_adapter = :multi_json
# or
config.json_adapter = :oj
endTOML
Lutaml::Model supports the following TOML adapters:
- Toml-rb
-
(default) A TOML parser and serializer for Ruby that is compatible with the TOML v1.0.0 specification. Requires the
toml-rbgem. - Tomlib
-
(optional) Toml-rb fork that is compatible with the TOML v1.0.0 specification, but with additional features. Requires the
tomlibgem.
require 'lutaml/model'
Lutaml::Model::Config.configure do |config|
config.toml_adapter = :toml_rb
# or
config.toml_adapter = :tomlib
endError handling
When parsing invalid serialization format data, an InvalidFormatError is
raised to indicate that the input format is malformed and cannot be parsed.
Certain adapters may have specific behaviors when handling invalid input:
-
⚠️ Caution TOML: the Tomlib adapter can cause segmentation faults when parsing invalid TOML data on Windows with Ruby versions before 3.3. A segmentation fault will crash the Ruby process entirely and cannot be caught with Ruby exception handling.
Custom serialization adapters
Lutaml::Model provides a flexible system for creating custom adapters to handle different data formats.
Please refer to Custom adapters for details and examples.
Schema generation
Lutaml::Model provides functionality to generate schema definitions from LutaML models. This allows you to create schemas that can be used for validation or documentation purposes.
Currently, the following schema formats are supported:
-
JSON Schema (JSON Schema)
-
YAML Schema (YAML)
-
RELAX NG (RELAX NG)
The schema generation supports advanced features such as:
-
Validation constraints (patterns, enumerations, ranges)
-
Choice attributes with min/max constraints
-
Polymorphic types with oneOf validation
-
Collection constraints with minItems/maxItems
Please refer to Schema Generation for details and examples.
Schema import
Lutaml::Model provides functionality to import schema definitions into LutaML models. This allows you to create models from existing schema definitions.
Currently, the following schema formats are supported:
-
JSON/YAML Schema
Please refer to Schema Import for details and examples.
Custom Registers
A LutaML::Model Register allows for dynamic modification and reconfiguration of model hierarchies without altering the original model definitions. For more information, refer to the Custom Registers Guide.
|
Note
|
Before using the Lutaml::Model::Register instance, make sure to register it in Lutaml::Model::GlobalRegister.
|
|
Note
|
By default, a default_register with the id :default is created and registered in the GlobalRegister. This default register is also set in Lutaml::Model::Config.default_register as the default value.
|
The default register can be set at the configuration level using the following syntax:
Lutaml::Model::Config.default_register = :default # the register id goes here.Comparison with Shale
Lutaml::Model is a serialization library that is similar to Shale, but with some differences in implementation.
| Feature | Lutaml::Model | Shale | Notes |
|---|---|---|---|
Data model definition |
3 types:
|
2 types:
|
|
Value types |
|
|
Lutaml::Model supports additional value types |
Configuration |
|
|
Lutaml::Model uses a configuration block to set the serialization adapters. |
Custom serialization methods |
|
|
Lutaml::Model uses the |
Serialization formats |
XML, YAML, JSON, TOML |
XML, YAML, JSON, TOML, CSV |
Lutaml::Model does not support CSV. |
Validation |
Supports collection range, fixed values, and custom validation |
Requires implementation |
|
Adapter support |
XML (Nokogiri, Ox, Oga), YAML, JSON (JSON, MultiJson), TOML (Toml-rb, Tomlib) |
XML (Nokogiri, Ox), YAML, JSON (JSON, MultiJson), TOML (Toml-rb, Tomlib), CSV |
Lutaml::Model does not support CSV. |
XML features |
|||
XML default namespace |
Yes. Supports |
No. Only supports |
|
XML mixed content support |
Yes. Supports the following kind of XML through mixed content support. <description>My name is
<bold>John Doe</bold>,
and I'm <i>28</i>
years old</description> |
No. Shale’s |
|
XML namespace inheritance |
Yes. Supports the |
No. |
|
Support for |
Yes. Automatically supports the |
Requires manual specification on every XML element that uses it. |
|
Compiling XML Schema to Lutaml::Model::Serializable classes |
Yes. Using
|
Yes, Provides only an array of the classes and doesn’t support |
|
Attribute features |
|||
Attribute delegation |
|
|
|
Enumerations |
Yes. Supports enumerations as value types through the
|
No. |
Lutaml::Model supports enumerations as value types. |
Attribute extraction |
Yes. Supports attribute extraction from key-value data models. |
No. |
Lutaml::Model supports attribute extraction from key-value data models. |
Register |
Yes. Supports three types of registers(read more details) with different types of functionalities. |
Supports |
|
Migration steps from Shale
The following sections provide a guide for migrating from Shale to Lutaml::Model.
Step 1: Replace inheritance class
Lutaml::Model uses Lutaml::Model::Serializable as the base inheritance class.
class Example < Lutaml::Model::Serializable
# ...
end|
Note
|
class Example
include Lutaml::Model::Serialize
# ...
end |
Shale uses Shale::Mapper as the base inheritance class.
class Example < Shale::Mapper
# ...
endActions:
-
Replace mentions of
Shale::MapperwithLutaml::Model::Serializable. -
Potentially replace inheritance with inclusion for suitable cases.
Step 2: Replace value type definitions
Value types in Lutaml::Model are under the Lutaml::Model::Type module,
or use the LutaML type symbols.
class Example < Lutaml::Model::Serializable
attribute :length, :integer
attribute :description, :string
end|
Note
|
class Example < Lutaml::Model::Serializable
attribute :length, Lutaml::Model::Type::Integer
attribute :description, "String"
end |
Value types in Shale are under the Shale::Type module.
class Example < Shale::Mapper
attribute :length, Shale::Type::Integer
attribute :description, Shale::Type::String
endAction:
-
Replace mentions of
Shale::TypewithLutaml::Model::Type. -
Potentially replace value type definitions with strings or symbols.
Step 3: Configure serialization adapters
Lutaml::Model uses a configuration block to set the serialization adapters.
require 'lutaml/model/xml/nokogiri_adapter'
Lutaml::Model::Config.configure do |config|
config.xml_adapter = Lutaml::Model::Xml::NokogiriAdapter
endThe equivalent for Shale is this:
require 'shale/adapter/nokogiri'
Shale.xml_adapter = Shale::Adapter::Nokogiri|
Note
|
Lutaml::Model::XmlAdapter is deprecated and will be removed in the next major upgrade. Use Lutaml::Model::Xml instead.
|
Here are places that this code may reside at:
-
If your code is a standalone Ruby script, this code will be present in your code.
-
If your code is organized in a Ruby gem, this code will be specified somewhere referenced by
lib/your_gem_name.rb. -
If your code contains tests or specs, they will be in the test setup file, e.g. RSpec
spec/spec_helper.rb.
Actions:
-
Replace the Shale configuration block with the
Lutaml::Model::Configconfiguration block. -
Replace the Shale adapter with the
Lutaml::Modeladapter.
Step 4: Rewrite custom serialization methods
There is an implementation difference between Lutaml::Model and Shale for custom serialization methods.
Custom serialization methods in Lutaml::Model map to individual attributes.
For custom serialization methods, Lutaml::Model uses the :with keyword
instead of the :using keyword used by Shale.
class Example < Lutaml::Model::Serializable
attribute :name, :string
attribute :size, :integer
attribute :color, :string
attribute :description, :string
json do
map "name", to: :name, with: { to: :name_to_json, from: :name_from_json }
map "size", to: :size
map "color", to: :color,
with: { to: :color_to_json, from: :color_from_json }
map "description", to: :description,
with: { to: :description_to_json, from: :description_from_json }
end
xml do
root "CustomSerialization"
map_element "Name", to: :name,
with: { to: :name_to_xml, from: :name_from_xml }
map_attribute "Size", to: :size
map_element "Color", to: :color,
with: { to: :color_to_xml, from: :color_from_xml }
map_content to: :description,
with: { to: :description_to_xml,
from: :description_from_xml }
end
def name_to_json(model, doc)
doc["name"] = "JSON Masterpiece: #{model.name}"
end
def name_from_json(model, value)
model.name = value.sub(/^JSON Masterpiece: /, "")
end
def color_to_json(model, doc)
doc["color"] = model.color.upcase
end
def color_from_json(model, value)
model.color = value.downcase
end
def description_to_json(model, doc)
doc["description"] = "JSON Description: #{model.description}"
end
def description_from_json(model, value)
model.description = value.sub(/^JSON Description: /, "")
end
def name_to_xml(model, parent, doc)
el = doc.create_element("Name")
doc.add_text(el, "XML Masterpiece: #{model.name}")
doc.add_element(parent, el)
end
def name_from_xml(model, value)
model.name = value.sub(/^XML Masterpiece: /, "")
end
def color_to_xml(model, parent, doc)
color_element = doc.create_element("Color")
doc.add_text(color_element, model.color.upcase)
doc.add_element(parent, color_element)
end
def color_from_xml(model, value)
model.color = value.downcase
end
def description_to_xml(model, parent, doc)
doc.add_text(parent, "XML Description: #{model.description}")
end
def description_from_xml(model, value)
model.description = value.join.strip.sub(/^XML Description: /, "")
end
endCustom serialization methods in Shale do not map to specific attributes, but allow the user to specify where the data goes.
class Example < Shale::Mapper
attribute :name, Shale::Type::String
attribute :size, Shale::Type::Integer
attribute :color, Shale::Type::String
attribute :description, Shale::Type::String
json do
map "name", using: { from: :name_from_json, to: :name_to_json }
map "size", to: :size
map "color", using: { from: :color_from_json, to: :color_to_json }
map "description", to: :description, using: { from: :description_from_json, to: :description_to_json }
end
xml do
root "CustomSerialization"
map_element "Name", using: { from: :name_from_xml, to: :name_to_xml }
map_attribute "Size", to: :size
map_element "Color", using: { from: :color_from_xml, to: :color_to_xml }
map_content to: :description, using: { from: :description_from_xml, to: :description_to_xml }
end
def name_to_json(model, doc)
doc['name'] = "JSON Masterpiece: #{model.name}"
end
def name_from_json(model, value)
model.name = value.sub(/^JSON Masterpiece: /, "")
end
def color_to_json(model, doc)
doc['color'] = model.color.upcase
end
def color_from_json(model, doc)
model.color = doc['color'].downcase
end
def description_to_json(model, doc)
doc['description'] = "JSON Description: #{model.description}"
end
def description_from_json(model, doc)
model.description = doc['description'].sub(/^JSON Description: /, "")
end
def name_from_xml(model, node)
model.name = node.text.sub(/^XML Masterpiece: /, "")
end
def name_to_xml(model, parent, doc)
name_element = doc.create_element('Name')
doc.add_text(name_element, model.street.to_s)
doc.add_element(parent, name_element)
end
end|
Note
|
There are cases where the Shale implementation of custom methods work differently from the Lutaml::Model implementation. In these cases, you will need to adjust the custom methods accordingly. |
Actions:
-
Replace the
usingkeyword with thewithkeyword. -
Adjust the custom methods.
About LutaML
The name "LutaML" is pronounced as "Looh-tah-mel".
The name "LutaML" comes from the Latin word for clay, "Lutum", and "ML" for "Markup Language". Just as clay can be molded and modeled into beautiful and practical end products, the Lutaml::Model gem is used for data modeling, allowing you to shape and structure your data into useful forms.
License and Copyright
This project is licensed under the BSD 2-clause License. See the LICENSE.md file for details.
Copyright Ribose.