DynamicSchema

The DynamicSchema gem provides an elegant and expressive way to define domain-specific language (DSL) schemas, making it effortless to build and validate complex Ruby Hash constructs.

This is particularly useful when dealing with intricate configuration or interfacing with external APIs, where data structures need to adhere to specific formats and validations. By allowing default values, type constraints, nested schemas, and transformations, DynamicSchema ensures that your data structures are both robust and flexible.

You can trivially define a custom schema:

openai_request_schema = DynamicSchema.define do 
  model           String, default: 'gpt-4o'
  max_tokens      Integer, default: 1024
  temperature     Float, in: 0..1

  message         arguments: [ :role ], as: :messages, array: true do 
    role          Symbol, in: [ :system, :user, :assistant ]
    content       array: true do 
      type        Symbol, default: :text 
      text        String
    end
  end
end

And then repeatedly use that schema to elegantly build a schema-conformant Hash:

request = openai_request_schema.build {
  message :system do 
    content text: "You are a helpful assistant that talks like a pirate."
  end
  message :user do 
    content text: ARGV[0] || "say hello!"
  end
}

You can find a full OpenAI request example in the /examples folder of this repository.

• Installation
• Usage
• Values
• Objects
• Types
• Options
  • default Option
  • required Option
  • array Option
  • as Option
  • in Option (Values Only)
  • arguments Option
• Class Schema
  • Definable
  • Buildable
• Struct
• Validation Methods
  • Validation Rules
  • validate!
  • validate
  • valid?
• Error Types
• Contributing
• License

Add this line to your application's Gemfile:

gem 'dynamicschema'

And then execute:

bundle install

Or install it yourself as:

gem install dynamicschema

To start using the dynamic_schema gem, simply require it in your Ruby file:

require 'dynamic_schema'

DynamicSchema lets you define a DSL made of values, objects, and options, then reuse that DSL to build and validate Ruby Hashes.

You start by constructing a DynamicSchema::Builder. You can do this by calling:

• DynamicSchema.define { … }
• DynamicSchema::Builder.new.define { … }

In both cases, you pass a block that declares the schema values and objects with their options.

schema = DynamicSchema.define do
  # values with an optional default
  api_key        String
  model          String,  default: 'gpt-4o'

  # object with its own values
  chat_options do
    max_tokens   Integer, default: 1024
    temperature  Float,   in: 0..1
  end
end

You can then:

# build without validation
built = schema.build do
  api_key 'secret'
  chat_options do
    temperature 0.7
  end
end

# build with validation (raises on first error)
built_validated = schema.build! do
  api_key 'secret'
  chat_options do
    temperature 0.7
  end
end

# validate an existing Hash (no building)
errors = schema.validate( { api_key: 'secret', chat_options: { temperature: 0.7 } } )
valid  = schema.valid?( { api_key: 'secret', chat_options: { temperature: 0.7 } } )

You can extend an existing schema using the inherit: option. Pass a Proc that describes the parent schema—typically from a class that includes DynamicSchema::Definable via its schema method.

class BaseSettings
  include DynamicSchema::Definable
  schema do
    api_key String, required: true
  end
end

# extend the base schema with additional fields
builder = DynamicSchema.define( inherit: BaseSettings.schema ) do
  region Symbol, in: %i[us eu apac]
end

settings = builder.build! do
  api_key 'secret'
  region  :us
end

You can call build, build!, validate, validate!, and valid? on the builder as needed.

In addition to building plain Ruby Hash values, DynamicSchema can generate lightweight Ruby classes from a schema. A DynamicSchema::Struct exposes readers and writers for the fields you define, and transparently wraps nested objects so that you can access them with dot-style accessors rather than deep hash indexing.

You create a struct class by passing the same schema shape you would give to a Builder. The schema can be provided as:

• a Proc that defines the schema
• a DynamicSchema::Builder
• a compiled Hash (advanced)

require 'dynamic_schema'

# simple struct with typed fields
Person = DynamicSchema::Struct.define do
  full_name String
  age       Integer
end

person = Person.build( full_name: 'Sam Lee', age: '42' )
person.age          # => 42 (coerced using the same converters as Builder)
person.full_name = 'Samira Lee'
person.to_h         # => { full_name: 'Samira Lee', age: 42 }

# nested object with its own accessors
Company = DynamicSchema::Struct.define do
  employee do
    full_name String
    years_of_service Integer
  end
end

acme = Company.build( employee: { full_name: 'Alex', years_of_service: 5 } )
acme.employee.full_name        # => 'Alex'
acme.employee.years_of_service # => 5

# array of nested objects
Order = DynamicSchema::Struct.define do
  items array: true do
    name  String
    price Integer
  end
end

order = Order.build( items: [ { name: 'Desk', price: 100 }, { name: 'Chair', price: 50 } ] )
order.items.map { | i | i.name }   # => [ 'Desk', 'Chair' ]

# referencing another struct class
OrderItem = DynamicSchema::Struct.define do
  name     String
  quantity Integer
end

OrderCollection = DynamicSchema::Struct.define do
  order_number String
  line_items   OrderItem, array: true
end

collection = OrderCollection.new( {
  order_number: 'A-100',
  line_items: [ { name: 'Desk', quantity: 1 }, { name: 'Chair', quantity: 2 } ]
} )
collection.line_items[ 0 ].name     # => 'Desk'
collection.line_items[ 1 ].quantity # => 2

• defining
  • DynamicSchema::Struct.define takes a block that looks exactly like a Builder schema.
  • Use array: true to expose arrays of nested structs.
  • You may reference another struct class as a value type; arrays of that type expose nested accessors for each element.
• building
  • StructClass.build( attributes ) constructs an instance and (optionally) coerces typed scalar fields using the same converters as the Builder.
  • StructClass.build! additionally validates the instance just like builder.build!.
• accessing
  • Use standard Ruby readers/writers: instance.attribute, instance.attribute = value.
  • #to_h returns a deep Hash of the current values (nested structs become hashes).

You can also create a struct class from a builder or a compiled hash if you already have a schema elsewhere:

builder = DynamicSchema.define do
  name String
end

NameStruct = DynamicSchema::Struct.new( builder )
NameStruct.build( name: 'Taylor' ).name  # => 'Taylor'

• validation
  • struct instances include the same validation helpers as hashes built via a builder.
  • StructClass.build! validates immediately and raises on the first error.
  • instances respond to #validate!, #validate, and #valid? using the compiled schema.

A value is a basic building block of your schema. Values represent individual settings, options or API parameters that you can define with specific types, defaults, and other options.

When defining a value, you provide the name as though you were calling a Ruby method, with arguments that include an optional type (which can be a Class, Module or an Array of these) as well as a Hash of options, all of which are optional:

name {type} default: {true|false}, required: {true|false}, array: {true|false}, as: {name}, in: {Array|Range}

require 'dynamic_schema'

# define a schema structure with values
schema = DynamicSchema.define do
  api_key
  version, String, default: '1.0'
end

# build the schema and set values
result = schema.build! do
  api_key 'your-api-key'
end

# access the schema values
puts result[:api_key]     # => "your-api-key"
puts result[:version]     # => "1.0"

• defining
  • api_key defines a value named api_key. Any type can be used to assign the value.
  • version, String, default: '1.0' defines a value with a default.
• building
  • schema.build! accepts both a Hash and a block where you can set the values.
  • Inside the block, api_key 'your-api-key' sets the value of api_key.
• accessing
  • result[:api_key] retrieves the value of api_key.
  • If a value has a default and you don't set it, the default value will be included in resulting hash.

A schema may be organized hierarchically, by creating collections of related values and even other collections. These collections are called objects.

An object is defined in a similar manner to a value. Simply provide the name as though calling a Ruby method, with a Hash of options and a block which encloses the child values and objects:

name arguments: [ {argument} ], default: {true|false}, required: {true|false}, array: {true|false}, as: {name} do 
  # child values and objects can be defined here
end 

Notice an object does not accept a type as it is always of type Object.

require 'dynamic_schema'

schema = DynamicSchema.define do
  api_key, String
  chat_options do
    model String, default: 'claude-3'
    max_tokens Integer, default: 1024
    temperature Float, default: 0.5, in: 0..1
    stream [ TrueClass, FalseClass ]
  end
end

result = schema.build! do
  api_key 'your-api-key'
  chat_options do
    temperature 0.8
    stream true
  end
end

# Accessing values
puts result[:api_key]                     # => "your-api-key"
puts result[:chat_options][:model]        # => "claude-3"
puts result[:chat_options][:temperature]  # => 0.8
puts result[:chat_options][:stream]       # => true

• defining
  • chat_options do ... end defines an object named chat_options.
  • Inside the object you can define values that belong to that object.
• building
  • In the build block, you can set values for values within objects by nesting blocks.
  • chat_options do ... end allows you to set values inside the chat_options object.
• accessing
  • You access values by chaining the keys: result[:chat_options][:model].

An object is always of type Object. A value can have no type or it can be of one or more types. You specify the value type by providing an instance of a Class when defining the value. If you want to specify multiple types simply provide an array of types.

require 'dynamic_schema'

schema = DynamicSchema.define do
  typeless_value
  symbol_value      Symbol 
  boolean_value     [ TrueClass, FalseClass ]
end

result = schema.build! do
  typeless_value    Struct.new(:name).new(name: 'Kristoph')
  symbol_value      "something"
  boolean_value     true 
end 

puts result[:typeless_value].name             # => "Kristoph"
puts result[:symbol_value]                    # => :something
puts result[:boolean_value]                   # => true 

• defining
  • typeless_value defines a value that has no type and will accept an assignment of any type
  • symbol_value defines a value that accepts symbols or types that can be coerced into symbols, such as strings (see Type Coercion)
  • boolean_value defines a value that can be either true or false

Both values and objects can be customized through options. The options for both values and objects include default, required, as and array. In addition values support the in criteria option while objects support the arguments option.

The :default option allows you to specify a default value that will be used if no value is provided during build.

schema = DynamicSchema.define do
  api_version String, default: 'v1'
  timeout Integer, default: 30
end

result = schema.build!
puts result[:api_version]  # => "v1"
puts result[:timeout]      # => 30

The :required option ensures that a value must be provided when building the schema. If a required value is missing when using build!, validate, or validate!, a DynamicSchema::RequiredOptionError will be raised.

schema = DynamicSchema.define do
  api_key String, required: true
  timeout Integer, default: 30
end

# This will raise DynamicSchema::RequiredOptionError
result = schema.build!

# This is valid
result = schema.build! do
  api_key 'my-secret-key'
end

The :array option wraps the value or object in an array in the resulting Hash, even if only one value is provided. This is particularly useful when dealing with APIs that expect array inputs.

schema = DynamicSchema.define do
  tags String, array: true
  message array: true do
    text String
    type String, default: 'plain'
  end
end

result = schema.build! do
  tags 'important'
  message do
    text 'Hello world'
  end
end

puts result[:tags]      # => ["important"]
puts result[:message]   # => [{ text: "Hello world", type: "plain" }]

The :as option allows you to use a different name in the DSL than what appears in the final Hash. This is particularly useful when interfacing with APIs that have specific key requirements.

schema = DynamicSchema.define do
  content_type String, as: "Content-Type", default: "application/json"
  api_key String, as: "Authorization"
end

result = schema.build! do
  api_key 'Bearer abc123'
end

puts result["Content-Type"]    # => "application/json"
puts result["Authorization"]   # => "Bearer abc123"

The :in option provides validation for values, ensuring they fall within a specified Range or are included in an Array of allowed values. This option is only available for values.

schema = DynamicSchema.define do
  temperature Float, in: 0..1
  status String, in: ['pending', 'processing', 'completed']
end

# Valid
result = schema.build! do
  temperature 0.7
  status 'pending'
end

# Will raise validation error - temperature out of range
result = schema.build! do
  temperature 1.5
  status 'pending'
end

# Will raise validation error - invalid status
result = schema.build! do
  temperature 0.7
  status 'invalid'
end

The :arguments option allows objects to accept arguments when building. Any arguments provided must appear when the object is built ( and so are implicitly 'required' ).

If the an argument is provided, the same argument appears in the attributes hash, or in the object block, the assignemnt in the block will take priority, followed by the attributes assigned and finally the argument.

schema = DynamicSchema.define do
  message arguments: [ :role ], as: :messages, array: true do
    role Symbol, required: true, in: [ :system, :user, :assistant ]
    content String
  end
end

result = schema.build! do
  message :system do
    content "You are a helpful assistant."
  end
  message :user do
    content "Hello!"
  end
end

DynamicSchema provides a number of modules you can include into your own classes to simplify their definition and construction.

The Definable module, when included in a class, will add the schema and the builder class methods.

By calling schema with a block you can define a schema for that specific class. You may also retrieve the defined schema by calling 'schema' ( with or without a block ). The 'schema' method may be called repeatedly to build up a schema with each call adding to the existing schema ( replacing values and objects of the same name if they appear in subsequent calls ).

The schema method will integrate with a class hierarchy. By including Definable in a base class you can call schema to define a schema for that base class and then in subsequent derived classes to augment it for those classes.

The builder method will return a memoized builder of the schema defined by calls to the schema method which can be used to build and validate schema conformant hashes.

class Setting 
  include DynamicSchema::Definable 
  schema do 
    name String 
  end 
end 

class DatabaSetting < Setting  
  schema do 
    database do 
      host  String
      port  String 
      name  String 
    end 
  end 

  def initialize( attributes = {} )
    # validate the attributes 
    self.class.builder.validate!( attributes )
    # retain them for future access 
    @attributes = attributes&.dup 
  end

end 

The Buildable module can be included in a class, in addition to Definable, to facilitate building that class using a schema assisted builder pattern. The Buildable module adds build! and build methods to the class which can be used to build that class, with and without validation respectively.

These methods accept both a Hash with attributes that follow the schema, as well as a block that can be used to build the class instance. The attributes and block can be used simultaneously.

Important Note that Buildable requires a class method builder ( which Definable provides ) and an initializer that accepts a Hash of attributes.

class Setting 
  include DynamicSchema::Definable 
  include DynamicSchema::Buildable
  schema do 
    name String 
  end 
end 

class DatabaseSetting < Setting  
  schema do 
    database do 
      adapter   Symbol
      host      String
      port      String 
      name      String 
    end 
  end 

  def initialize( attributes = {} )
    # validate the attributes 
    self.class.builder.validate!( attributes )
    # retain them for the future  
    @attributes = attributes&.dup 
  end
end 

database_settings = DatabaseSetting.build! name: 'settings.database' do 
  database adapter: :pg do 
    host     "localhost"
    port     "127.0.0.1"
    name     "mydb"
  end 
end

DynamicSchema provides three different methods for validating Hash structures against your defined schema: validate!, validate, and valid?.

These methods allow you to verify that your data conforms to your schema requirements, including type constraints, required fields, and value ranges.

When validating, DynamicSchema checks:

1. Required Fields: Any value or object marked as required: true is present.
2. Type Constraints: Any values match their specified types or can be coerced to the specified type.
3. Value Ranges: Any values fall within their specified :in constraints.
4. Objects: Any objects are recursively validated.
5. Arrays: Any validation rules are applied to each element when array: true

The validate! method performs strict validation and raises an exception when it encounters the first validation error.

schema = DynamicSchema.define do
  api_key String, required: true
  temperature Float, in: 0..1
end

# this will raise DynamicSchema::RequiredOptionError
schema.validate!( { temperature: 0.5 } )

# this will raise DynamicSchema::IncompatibleTypeError
schema.validate!( {
  api_key: ["not-a-string"],            
  temperature: 0.5
} )

# this will raise DynamicSchema::InOptionError
schema.validate!( {
  api_key: "abc123",
  temperature: 1.5     
} )

# this is valid and will not raise any errors
schema.validate!( {
  api_key: 123,  
  temperature: 0.5
} )

The validate method performs validation but instead of raising exceptions, it collects and returns an array of all validation errors encountered.

schema = DynamicSchema.define do
  api_key String, required: true
  model String, in: ['gpt-3.5-turbo', 'gpt-4']
  temperature Float, in: 0..1
end

errors = schema.validate({
  model: 'invalid-model',
  temperature: 1.5,
  api_key: ["invalid-type"]  # Array cannot be coerced to String
})

# errors will contain:
# - IncompatibleTypeError for api_key being an Array
# - InOptionError for invalid model
# - InOptionError for temperature out of range

The valid? method provides a simple boolean check of whether a Hash conforms to the schema.

schema = DynamicSchema.define do
  name String, required: true
  age Integer, in: 0..120
  id String  # Will accept both strings and numbers due to coercion
end

# Returns false
schema.valid?({
  name: ["Not a string"],  # Array cannot be coerced to String
  age: 150                 # Outside allowed range
})

# Returns true
schema.valid?({
  name: "John",
  age: 30,
  id: 12345               # Numeric value can be coerced to String
})

DynamicSchema provides specific error types for different validation failures:

• DynamicSchema::RequiredOptionError: Raised when a required field is missing
• DynamicSchema::IncompatibleTypeError: Raised when a value's type doesn't match the schema and cannot be coerced
• DynamicSchema::InOptionError: Raised when a value falls outside its specified range/set
• ArgumentError: Raised when the provided values structure isn't a Hash

Each error includes helpful context about the validation failure, including the path to the failing field and the specific constraint that wasn't met.

Bug reports and pull requests are welcome on GitHub at https://github.com/EndlessInternational/dynamic_schema.

The gem is available as open source under the terms of the MIT License.