Project

lutaml-xsd

0.01
The project is in a healthy, maintained state
Parser and builder for XML Schema Definition (XSD) files.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
 Dependencies

Runtime

 Project Readme

Expressir: EXPRESS in Ruby

Gem Version Build Status

Purpose

Expressir (“EXPRESS in Ruby”) is a Ruby parser for EXPRESS and a set of Ruby tools for accessing ISO EXPRESS data models.

Architecture

Expressir consists of 3 parts:

  1. Parsers. A parser allows Expressir to read EXPRESS files, including:

    • EXPRESS data modelling language (ISO 10303-11:2007)

    • EXPRESS data modelling language in XML (STEPmod)

    • EXPRESS XML (ISO 10303-28:2007) “Industrial automation systems and integration — Product data representation and exchange — Part 28: Implementation methods: XML representations of EXPRESS schemas and data, using XML schemas”

  2. Data model. The data model (lib/expressir/express) is the Ruby data model that fully represents an EXPRESS data model.

  3. Converters. A converter transforms the EXPRESS Ruby data model into an interoperable export format, including:

    • EXPRESS data modelling language (ISO 10303-11:2007)

Installation

Add this line to your application’s Gemfile:

gem "expressir"

And then execute:

$ bundle install

Or install it yourself as:

$ gem install expressir

Testing

Run the test suite:

bundle exec rake

Usage: CLI

This gem ships with a CLI tool. To check what’s available you can simply run expressir, by default it will display some usage instructions.

$ expressir

Commands:
  expressir benchmark FILE_OR_YAML             # Benchmark schema loading performance for a file or list of files from YAML
  expressir benchmark-cache FILE_OR_YAML       # Benchmark schema loading with caching
  expressir clean PATH                         # Strip remarks and prettify EXPRESS schema at PATH
  expressir format PATH                        # pretty print EXPRESS schema located at PATH
  expressir help [COMMAND]                     # Describe available commands or one specific command
  expressir validate *PATH                     # validate EXPRESS schema located at PATH
  expressir coverage *PATH                     # List EXPRESS entities and check documentation coverage
  expressir version                            # Expressir Version

Format schema

The format command pretty prints an EXPRESS schema, making it more readable while preserving its structure.

# Pretty print a schema to stdout
expressir format schemas/resources/action_schema/action_schema.exp

This command: 1. Parses the EXPRESS schema 2. Formats it with consistent indentation and spacing 3. Outputs the formatted schema to stdout

Clean schema

The clean command strips remarks and prettifies EXPRESS schemas. This is useful for removing all documentation comments while maintaining the schema’s functional definition. You can optionally save the result to a file.

# Output to stdout
expressir clean schemas/resources/action_schema/action_schema.exp

# Save to file
expressir clean schemas/resources/action_schema/action_schema.exp --output clean_schema.exp
Option Description

--output PATH

Path to save the cleaned schema (optional, defaults to stdout)

Validate schema

The validate command performs validation checks on EXPRESS schema files.

It verifies:

  1. That the schema can be parsed correctly

  2. That the schema includes a version string

# Validate a single schema
expressir validate schemas/resources/action_schema/action_schema.exp

# Validate multiple schemas
expressir validate schemas/resources/action_schema/action_schema.exp schemas/resources/approval_schema/approval_schema.exp

The command reports any schemas that:

  • Failed to parse

  • Are missing a version string

If all validations pass, it will display "Validation passed for all EXPRESS schemas."

Version

The version command displays the current version of the Expressir gem.

expressir version

Benchmarking

Expressir includes powerful benchmarking capabilities for measuring schema loading performance. You can benchmark individual files or multiple files listed in a YAML configuration.

Benchmark a single file

# Basic benchmarking
expressir benchmark schemas/resources/action_schema/action_schema.exp

# With detailed output
expressir benchmark schemas/resources/action_schema/action_schema.exp --verbose

# Using benchmark-ips for more detailed statistics
expressir benchmark schemas/resources/action_schema/action_schema.exp --ips

# With specific output format
expressir benchmark schemas/resources/action_schema/action_schema.exp --format json

Benchmark multiple files from YAML

Create a YAML file with a list of schema paths:

schemas.yml
schemas:
  - schemas/resources/action_schema/action_schema.exp
  - schemas/resources/approval_schema/approval_schema.exp
  - schemas/resources/date_time_schema/date_time_schema.exp

Then benchmark all schemas at once:

expressir benchmark schemas.yml --verbose

Benchmark with caching

You can also benchmark schema loading with caching to measure parsing time, cache writing time, and cache reading time:

# Benchmark a single file with caching
expressir benchmark-cache schemas/resources/action_schema/action_schema.exp

# With custom cache location
expressir benchmark-cache schemas/resources/action_schema/action_schema.exp --cache_path /tmp/schema_cache.bin

# Benchmark multiple files from YAML with caching
expressir benchmark-cache schemas.yml --verbose

Benchmark options

The benchmark commands support several options:

Option Description

--ips

Use benchmark-ips for detailed statistics

--verbose

Show detailed output

--save

Save benchmark results to file

--format FORMAT

Output format: json, csv, or default

--cache_path PATH

(benchmark-cache only) Path to store the cache file

When using the --format json option, results will be output in JSON format, making it easy to parse for further analysis or visualization.

Documentation coverage

Expressir can analyze EXPRESS schemas to check for documentation coverage. This helps identify which entities are properly documented with remarks and which ones require documentation.

Analyzing documentation coverage

Use the coverage command to check documentation coverage of EXPRESS schemas:

# Analyze a single EXPRESS file
expressir coverage schemas/resources/action_schema/action_schema.exp

# Analyze multiple EXPRESS files
expressir coverage schemas/resources/action_schema/action_schema.exp schemas/resources/approval_schema/approval_schema.exp

# Analyze all EXPRESS files in a directory (recursively)
expressir coverage schemas/resources/

# Analyze files specified in a YAML file
expressir coverage schemas.yml

The output shows which entities are missing documentation, calculates coverage percentages, and provides an overall documentation coverage summary.

Coverage options

The coverage command supports different output formats and exclusion options:

Option Description

--format text

(Default) Display a human-readable table with coverage information

--format json

Output in JSON format for programmatic processing

--format yaml

Output in YAML format for programmatic processing

--exclude TYPES

Comma-separated list of EXPRESS entity types to exclude from coverage analysis

--ignore-files PATH

Path to YAML file containing array of files to ignore from overall coverage calculation

Excluding entity types from coverage

You can exclude specific EXPRESS entity types from coverage analysis using the --exclude option.

This is useful when certain entity types don’t require documentation coverage, such as TYPE entities whose descriptions are generated by template strings.

# Exclude TYPE entities from coverage analysis
expressir coverage --exclude=TYPE schemas/resources/action_schema/action_schema.exp

# Exclude only SELECT type definitions (useful since TYPE descriptions are often template-generated)
expressir coverage --exclude=TYPE:SELECT schemas/resources/action_schema/action_schema.exp

# Exclude multiple entity types
expressir coverage --exclude=TYPE,CONSTANT,FUNCTION schemas/resources/action_schema/action_schema.exp

# Exclude parameters and variables (often don't require individual documentation)
expressir coverage --exclude=PARAMETER,VARIABLE schemas/resources/action_schema/action_schema.exp

# Combine with output format options
expressir coverage --exclude=TYPE:SELECT --format=json schemas/resources/action_schema/action_schema.exp

Elements checked in documentation coverage

Expressir checks documentation coverage for all EXPRESS elements (ModelElement subclasses), including:

Schema-level entities:

TYPE

Type definitions (supports subtype exclusion, see below)

ENTITY

Entity definitions

CONSTANT

Constant definitions

FUNCTION

Function definitions

RULE

Rule definitions

PROCEDURE

Procedure definitions

SUBTYPE_CONSTRAINT

Subtype constraint definitions

INTERFACE

Interface definitions

Nested entities within other constructs:

PARAMETER

Function and procedure parameters

VARIABLE

Variables within functions, rules, and procedures

ATTRIBUTE

Entity attributes

DERIVED_ATTRIBUTE

Derived attributes in entities

INVERSE_ATTRIBUTE

Inverse attributes in entities

UNIQUE_RULE

Unique rules within entities

WHERE_RULE

Where rules within entities and types

ENUMERATION_ITEM

Items within enumeration types

INTERFACE_ITEM

Items within interfaces

INTERFACED_ITEM

Interfaced items

SCHEMA_VERSION

Schema version information

SCHEMA_VERSION_ITEM

Schema version items

TYPE subtype exclusion

For TYPE elements, you can exclude specific subtypes using the TYPE:SUBTYPE syntax:

# Exclude only SELECT types
expressir coverage --exclude=TYPE:SELECT schemas/resources/action_schema/action_schema.exp

# Exclude multiple TYPE subtypes
expressir coverage --exclude=TYPE:SELECT,TYPE:ENUMERATION schemas/resources/action_schema/action_schema.exp

FUNCTION subtype exclusion

For FUNCTION elements, you can exclude inner functions (functions nested within other functions, rules, or procedures) using the FUNCTION:INNER syntax:

# Exclude inner functions from coverage analysis
expressir coverage --exclude=FUNCTION:INNER schemas/resources/action_schema/action_schema.exp

# Combine with other exclusions
expressir coverage --exclude=TYPE:SELECT,FUNCTION:INNER schemas/resources/action_schema/action_schema.exp

This is useful when you want to focus documentation coverage on top-level functions while excluding nested helper functions that may not require individual documentation. The exclusion works recursively, excluding functions at any nesting level within other constructs.

Valid FUNCTION subtypes that can be excluded:

INNER

Inner functions nested within other functions, rules, or procedures (at any depth)

FUNCTION outer_function : BOOLEAN;
  -- This inner function would be excluded with FUNCTION:INNER
  FUNCTION inner_helper_function : BOOLEAN;
    -- Even deeply nested functions are excluded
    FUNCTION deeply_nested_function : BOOLEAN;
      RETURN (TRUE);
    END_FUNCTION;
    RETURN (TRUE);
  END_FUNCTION;

  RETURN (TRUE);
END_FUNCTION;

RULE example_rule FOR (some_entity);
  -- Inner functions in rules are also excluded
  FUNCTION inner_function_in_rule : BOOLEAN;
    RETURN (TRUE);
  END_FUNCTION;
WHERE
  WR1: inner_function_in_rule();
END_RULE;

PROCEDURE example_procedure;
  -- Inner functions in procedures are also excluded
  FUNCTION inner_function_in_procedure : BOOLEAN;
    RETURN (TRUE);
  END_FUNCTION;
END_PROCEDURE;

The FUNCTION:INNER exclusion helps maintain focus on documenting the primary API functions while ignoring implementation details of nested helper functions.

Ignoring files from coverage calculation

You can exclude entire files from the overall coverage calculation using the --ignore-files option. This is useful when you have files that should not contribute to the overall documentation coverage statistics, such as test schemas, example files, or legacy schemas.

# Use ignore files to exclude specific files from coverage calculation
expressir coverage --ignore-files ignore_list.yaml schemas/resources/

# Combine with other options
expressir coverage --ignore-files ignore_list.yaml --exclude=TYPE:SELECT --format=json schemas/resources/
Ignore files YAML format

The ignore files YAML should contain an array of file patterns. Each pattern can be either an exact file path or use glob patterns for matching multiple files.

ignore_list.yaml
# Array of file patterns to ignore
- examples/test_schema.exp                    # Exact file path
- examples/*_test_*.exp                       # Glob pattern for test files
- legacy/old_*.exp                           # Glob pattern for legacy files
- temp/temporary_schema.exp                   # Another exact path
Pattern matching behavior

File patterns in the ignore files YAML support:

  • Exact paths: Match specific files exactly

  • Glob patterns: Use * for wildcard matching

  • Relative paths: Patterns are resolved relative to the YAML file’s directory

  • Absolute paths: Full system paths are also supported

# Examples of different pattern types
- schemas/action_schema/action_schema.exp     # Exact relative path
- /full/path/to/schema.exp                   # Absolute path
- schemas/**/test_*.exp                      # Recursive glob pattern
- temp/*.exp                                 # All .exp files in temp directory
Behavior of ignored files

When files are ignored using the --ignore-files option:

  1. Excluded from overall statistics: Ignored files do not contribute to the overall coverage percentage calculation

  2. Still processed and reported: Ignored files are still analyzed and appear in the output, but marked with an ignored: true flag

  3. Separate reporting section: In JSON/YAML output formats, ignored files appear in both the main files section (with the ignored flag) and in a separate ignored_files section

  4. Overall statistics updated: The overall statistics include additional fields showing the count of ignored files and entities

Example JSON output with ignored files:
{
  "overall": {
    "coverage_percentage": 75.0,
    "total_entities": 100,
    "documented_entities": 75,
    "undocumented_entities": 25,
    "ignored_files_count": 2,
    "ignored_entities_count": 15
  },
  "files": [
    {
      "file": "schemas/main_schema.exp",
      "ignored": false,
      "coverage": 80.0,
      "total": 50,
      "documented": 40,
      "undocumented": ["entity1", "entity2"]
    },
    {
      "file": "examples/test_schema.exp",
      "ignored": true,
      "matched_pattern": "examples/*_test_*.exp",
      "coverage": 20.0,
      "total": 10,
      "documented": 2,
      "undocumented": ["test_entity1", "test_entity2"]
    }
  ],
  "ignored_files": [
    {
      "file": "examples/test_schema.exp",
      "matched_pattern": "examples/*_test_*.exp",
      "coverage": 20.0,
      "total": 10,
      "documented": 2,
      "undocumented": ["test_entity1", "test_entity2"]
    }
  ]
}
Error handling

The ignore files functionality handles various error conditions gracefully:

  • Missing YAML file: If the specified ignore files YAML doesn’t exist, a warning is displayed and coverage analysis continues normally

  • Invalid YAML format: If the YAML file is malformed or doesn’t contain an array, a warning is displayed and the file is ignored

  • Non-matching patterns: Patterns that don’t match any files are silently ignored (no error or warning)

  • Permission errors: File access errors are reported as warnings

Use cases for ignore files

Common scenarios where ignore files are useful:

  • Test schemas: Exclude test or example schemas from production coverage metrics

  • Legacy files: Ignore old schemas that are being phased out

  • Generated files: Exclude automatically generated schemas

  • Work-in-progress: Temporarily ignore files under development

  • Different coverage standards: Apply different documentation standards to different file sets

Valid TYPE subtypes that can be excluded:

AGGREGATE

Aggregate type

ARRAY

Array type

BAG

Bag type

BINARY

Binary type

BOOLEAN

Boolean type

ENUMERATION

Enumeration type

TYPE uuid_relationship_role = ENUMERATION OF
  (supersedes,
    merge,
    split,
    derive_from,
    same_as,
    similar_to);
END_TYPE;
GENERIC

Generic type

GENERIC_ENTITY

Generic entity type

TYPE uuid_attribute_select = EXTENSIBLE GENERIC_ENTITY SELECT;
END_TYPE;
INTEGER

Integer type

LIST

List type

TYPE uuid_list_item = LIST [1:?] OF UNIQUE LIST [1:?] OF UNIQUE uuid_attribute_select;
END_TYPE;
LOGICAL

Logical type

NUMBER

Number type

REAL

Real type

SELECT

Select type

TYPE uuid_set_or_list_attribute_select = SELECT
  (uuid_list_item,
    uuid_set_item);
END_TYPE;
SET

Set type

TYPE uuid_set_item = SET [1:?] OF uuid_attribute_select;
END_TYPE;
STRING

String type

TYPE uuid = STRING (36) FIXED;
END_TYPE;

This is particularly useful since TYPE entities with certain subtypes (like SELECT) often have descriptions generated by template strings and may not require individual remark item coverage.

Note
ISO 10303 excludes documentation coverage for TYPE:SELECT and TYPE:ENUMERATION.

If you specify an invalid entity type or subtype, the command will display an error message with the list of valid options.

Example 1. Example with JSON output:
expressir coverage schemas/resources/ --format json

When using JSON or YAML output formats, all file and directory paths are displayed relative to the current working directory:

- file: "schemas/resources/action_schema/action_schema.exp"
  file_basename: action_schema.exp
  directory: "schemas/resources/action_schema"
  # ... other fields

Coverage output

The default text output displays:

  1. Directory coverage (when analyzing multiple directories)

  2. File coverage, showing:

    • File path

    • List of undocumented entities

    • Coverage percentage

  3. Overall documentation coverage statistics

This helps identify areas of your EXPRESS schemas that need documentation improvement.

Usage: Ruby

Parsing EXPRESS schema files

General

The library provides two main methods for parsing EXPRESS files.

Parsing a single file

Use the from_file method to parse a single EXPRESS schema file:

# Parse a single file
repository = Expressir::Express::Parser.from_file("path/to/schema.exp")

# With options
repository = Expressir::Express::Parser.from_file(
  "path/to/schema.exp",
  skip_references: false,  # Set to true to skip resolving references
  include_source: true,    # Set to true to include original source in the model
  root_path: "/base/path"  # Optional base path for relative paths
)

The from_file method will raise a SchemaParseFailure exception if the schema fails to parse, providing information about the specific file and the parsing error:

begin
  repository = Expressir::Express::Parser.from_file("path/to/schema.exp")
rescue Expressir::Express::Error::SchemaParseFailure => e
  puts "Failed to parse schema: #{e.message}"
  puts "Filename: #{e.filename}"
  puts "Error details: #{e.parse_failure_cause.ascii_tree}"
end

Parsing multiple files

Use the from_files method to parse multiple EXPRESS schema files:

# Parse multiple files
files = ["schema1.exp", "schema2.exp", "schema3.exp"]
repository = Expressir::Express::Parser.from_files(files)

You can provide a block to track loading progress and handle errors:

files = ["schema1.exp", "schema2.exp", "schema3.exp"]
repository = Expressir::Express::Parser.from_files(files) do |filename, schemas, error|
  if error
    puts "Error loading #{filename}: #{error.message}"
    # Skip the file with an error or take other action
  else
    puts "Successfully loaded #{schemas.length} schemas from #{filename}"
  end
end

Filtering out schemas

You can filter out specific schemas from the repository easily since Expressir::Model::Repository implements Enumerable.

schema_yaml = YAML.load_file('documents/iso-10303-41/schemas.yaml')
schema_paths = schema_yaml['schemas'].map {|x,y| y['path'].gsub("../../", "")}

repo = Expressir::Express::Parser.from_files(schema_paths)

filtered_schemas = ["action_schema", "date_time_schema"]
repo.select do |schema|
  filtered_schemas.include?(schema.name)
end.each do |schema|
  puts "Schema name: #{schema.name}"
  puts "Schema file: #{schema.file}"
  puts "Schema version: #{schema.version}"
end

Convert models to Liquid

Use to_liquid method to convert the models of Expressir::Model::* to liquid drop models (Expressir::Liquid::*).

Example:

repo = Expressir::Express::Parser.from_file("path/to/file.exp")
repo_drop = repo.to_liquid

where repo is an instance of Expressir::Model::Repository and repo_drop is an instance of Expressir::Liquid::RepositoryDrop.

The Liquid drop models of Expressir::Liquid::* have the same attributes (model_attr) as the models of Expressir::Model::*.

For example, Expressir::Model::Repository has the following attributes:

  • schemas

and each Expressir::Model::Declarations::Schema has the following attributes:

  • file

  • version

  • interfaces

  • constants

  • entities

  • subtype_constraints

  • functions

  • rules

  • procedures

Thus, Expressir::Liquid::Repository has the same attribute schemas and Expressir::Liquid::Declarations::SchemaDrop has same attribute file.

repo = Expressir::Express::Parser.from_file("path/to/file.exp")
repo_drop = repo.to_liquid
schema = repo_drop.schemas.first
schema.file = "path/to/file.exp"

Documentation coverage analysis

Expressir’s documentation coverage feature can be used programmatically to analyze and report on documentation coverage of EXPRESS schemas.

# Create a coverage report from a file
report = Expressir::Coverage::Report.from_file("path/to/schema.exp")

# Or create a report from a repository
repository = Expressir::Express::Parser.from_file("path/to/schema.exp")
report = Expressir::Coverage::Report.from_repository(repository)

# Access overall statistics
puts "Overall coverage: #{report.coverage_percentage}%"
puts "Total entities: #{report.total_entities.size}"
puts "Documented entities: #{report.documented_entities.size}"
puts "Undocumented entities: #{report.undocumented_entities.size}"

# Access file-level reports
report.file_reports.each do |file_report|
  puts "File: #{file_report[:file]}"
  puts "  Coverage: #{file_report[:coverage]}%"
  puts "  Total entities: #{file_report[:total]}"
  puts "  Documented entities: #{file_report[:documented]}"
  puts "  Undocumented entities: #{file_report[:undocumented].join(', ')}"
end

# Access directory-level reports
report.directory_reports.each do |dir_report|
  puts "Directory: #{dir_report[:directory]}"
  puts "  Coverage: #{dir_report[:coverage]}%"
  puts "  Total entities: #{dir_report[:total]}"
  puts "  Documented entities: #{dir_report[:documented]}"
  puts "  Undocumented entities: #{dir_report[:undocumented]}"
  puts "  Number of files: #{dir_report[:files]}"
end

# Generate a structured hash representation
report_hash = report.to_h  # Contains overall, directories and files sections

You can also use the core methods directly to check documentation status:

# Check if an entity has documentation
schema = repository.schemas.first
entity = schema.entities.first

if Expressir::Coverage.entity_documented?(entity)
  puts "Entity #{entity.id} is documented"
else
  puts "Entity #{entity.id} is not documented"
end

# Find all entities in a schema
all_entities = Expressir::Coverage.find_entities(schema)
puts "Found #{all_entities.size} entities in schema #{schema.id}"

Contributing

First, thank you for contributing! We love pull requests from everyone. By participating in this project, you hereby grant Ribose Inc. the right to grant or transfer an unlimited number of non exclusive licenses or sub-licenses to third parties, under the copyright covering the contribution to use the contribution by all means.

Here are a few technical guidelines to follow:

  • Open an issues to discuss a new feature.

  • Write tests to support your new feature.

  • Make sure the entire test suite passes locally and on CI.

  • Open a Pull Request.

  • Squash your commits after receiving feedback.

  • Party!

Documentation

Expressir provides detailed documentation on various aspects of its functionality:

  • Benchmarking: Learn about Expressir’s built-in capabilities for measuring schema loading performance, particularly useful for large schemas or when optimizing performance.

  • Liquid Integration: Documentation on how to use Expressir models with Liquid templates for flexible document generation.

License

Expressir is distributed under the BSD 2-clause license.

Note
Expressir originally contained some code from the NIST Reeper project but no longer contains them.

The NIST Reeper license is reproduced below:

This software was funded by NIST and developed by EuroSTEP. Pursuant to title 17 Section 105 of the United States Code this software is not subject to copyright protection and is in the public domain.

We would appreciate acknowledgment if the software is used. Links to non-Federal Government Web sites do not imply NIST endorsement of any particular product, service, organization, company, information provider, or content.

Credits

Copyright Ribose Inc.