Expressir: EXPRESS in Ruby
Purpose
Expressir (“EXPRESS in Ruby”) is a Ruby parser for EXPRESS and a set of Ruby tools for accessing ISO EXPRESS data models.
Architecture
Expressir consists of 3 parts:
-
Parsers. A parser allows Expressir to read EXPRESS files, including:
-
EXPRESS data modelling language (ISO 10303-11:2007)
-
EXPRESS data modelling language in XML (STEPmod)
-
EXPRESS XML (ISO 10303-28:2007) “Industrial automation systems and integration — Product data representation and exchange — Part 28: Implementation methods: XML representations of EXPRESS schemas and data, using XML schemas”
-
-
Data model. The data model (
lib/expressir/express
) is the Ruby data model that fully represents an EXPRESS data model. -
Converters. A converter transforms the EXPRESS Ruby data model into an interoperable export format, including:
-
EXPRESS data modelling language (ISO 10303-11:2007)
-
Installation
Add this line to your application’s Gemfile
:
gem "expressir"
And then execute:
$ bundle install
Or install it yourself as:
$ gem install expressir
Testing
Run the test suite:
bundle exec rake
Usage: CLI
This gem ships with a CLI tool. To check what’s available you can simply run
expressir
, by default it will display some usage instructions.
$ expressir
Commands:
expressir benchmark FILE_OR_YAML # Benchmark schema loading performance for a file or list of files from YAML
expressir benchmark-cache FILE_OR_YAML # Benchmark schema loading with caching
expressir clean PATH # Strip remarks and prettify EXPRESS schema at PATH
expressir format PATH # pretty print EXPRESS schema located at PATH
expressir help [COMMAND] # Describe available commands or one specific command
expressir validate *PATH # validate EXPRESS schema located at PATH
expressir coverage *PATH # List EXPRESS entities and check documentation coverage
expressir version # Expressir Version
Format schema
The format
command pretty prints an EXPRESS schema, making it more readable
while preserving its structure.
# Pretty print a schema to stdout
expressir format schemas/resources/action_schema/action_schema.exp
This command: 1. Parses the EXPRESS schema 2. Formats it with consistent indentation and spacing 3. Outputs the formatted schema to stdout
Clean schema
The clean
command strips remarks and prettifies EXPRESS schemas. This is
useful for removing all documentation comments while maintaining the schema’s
functional definition. You can optionally save the result to a file.
# Output to stdout
expressir clean schemas/resources/action_schema/action_schema.exp
# Save to file
expressir clean schemas/resources/action_schema/action_schema.exp --output clean_schema.exp
Option | Description |
---|---|
|
Path to save the cleaned schema (optional, defaults to stdout) |
Validate schema
The validate
command performs validation checks on EXPRESS schema files.
It verifies:
-
That the schema can be parsed correctly
-
That the schema includes a version string
# Validate a single schema
expressir validate schemas/resources/action_schema/action_schema.exp
# Validate multiple schemas
expressir validate schemas/resources/action_schema/action_schema.exp schemas/resources/approval_schema/approval_schema.exp
The command reports any schemas that:
-
Failed to parse
-
Are missing a version string
If all validations pass, it will display "Validation passed for all EXPRESS schemas."
Version
The version
command displays the current version of the Expressir gem.
expressir version
Benchmarking
Expressir includes powerful benchmarking capabilities for measuring schema loading performance. You can benchmark individual files or multiple files listed in a YAML configuration.
Benchmark a single file
# Basic benchmarking
expressir benchmark schemas/resources/action_schema/action_schema.exp
# With detailed output
expressir benchmark schemas/resources/action_schema/action_schema.exp --verbose
# Using benchmark-ips for more detailed statistics
expressir benchmark schemas/resources/action_schema/action_schema.exp --ips
# With specific output format
expressir benchmark schemas/resources/action_schema/action_schema.exp --format json
Benchmark multiple files from YAML
Create a YAML file with a list of schema paths:
schemas:
- schemas/resources/action_schema/action_schema.exp
- schemas/resources/approval_schema/approval_schema.exp
- schemas/resources/date_time_schema/date_time_schema.exp
Then benchmark all schemas at once:
expressir benchmark schemas.yml --verbose
Benchmark with caching
You can also benchmark schema loading with caching to measure parsing time, cache writing time, and cache reading time:
# Benchmark a single file with caching
expressir benchmark-cache schemas/resources/action_schema/action_schema.exp
# With custom cache location
expressir benchmark-cache schemas/resources/action_schema/action_schema.exp --cache_path /tmp/schema_cache.bin
# Benchmark multiple files from YAML with caching
expressir benchmark-cache schemas.yml --verbose
Benchmark options
The benchmark commands support several options:
Option | Description |
---|---|
|
Use benchmark-ips for detailed statistics |
|
Show detailed output |
|
Save benchmark results to file |
|
Output format: json, csv, or default |
|
(benchmark-cache only) Path to store the cache file |
When using the --format json
option, results will be output in JSON format,
making it easy to parse for further analysis or visualization.
Documentation coverage
Expressir can analyze EXPRESS schemas to check for documentation coverage. This helps identify which entities are properly documented with remarks and which ones require documentation.
Analyzing documentation coverage
Use the coverage
command to check documentation coverage of EXPRESS schemas:
# Analyze a single EXPRESS file
expressir coverage schemas/resources/action_schema/action_schema.exp
# Analyze multiple EXPRESS files
expressir coverage schemas/resources/action_schema/action_schema.exp schemas/resources/approval_schema/approval_schema.exp
# Analyze all EXPRESS files in a directory (recursively)
expressir coverage schemas/resources/
# Analyze files specified in a YAML file
expressir coverage schemas.yml
The output shows which entities are missing documentation, calculates coverage percentages, and provides an overall documentation coverage summary.
Coverage options
The coverage command supports different output formats and exclusion options:
Option | Description |
---|---|
|
(Default) Display a human-readable table with coverage information |
|
Output in JSON format for programmatic processing |
|
Output in YAML format for programmatic processing |
|
Comma-separated list of EXPRESS entity types to exclude from coverage analysis |
|
Path to YAML file containing array of files to ignore from overall coverage calculation |
Excluding entity types from coverage
You can exclude specific EXPRESS entity types from coverage analysis using the
--exclude
option.
This is useful when certain entity types don’t require documentation coverage, such as TYPE entities whose descriptions are generated by template strings.
# Exclude TYPE entities from coverage analysis
expressir coverage --exclude=TYPE schemas/resources/action_schema/action_schema.exp
# Exclude only SELECT type definitions (useful since TYPE descriptions are often template-generated)
expressir coverage --exclude=TYPE:SELECT schemas/resources/action_schema/action_schema.exp
# Exclude multiple entity types
expressir coverage --exclude=TYPE,CONSTANT,FUNCTION schemas/resources/action_schema/action_schema.exp
# Exclude parameters and variables (often don't require individual documentation)
expressir coverage --exclude=PARAMETER,VARIABLE schemas/resources/action_schema/action_schema.exp
# Combine with output format options
expressir coverage --exclude=TYPE:SELECT --format=json schemas/resources/action_schema/action_schema.exp
Elements checked in documentation coverage
Expressir checks documentation coverage for all EXPRESS elements (ModelElement subclasses), including:
Schema-level entities:
TYPE
-
Type definitions (supports subtype exclusion, see below)
ENTITY
-
Entity definitions
CONSTANT
-
Constant definitions
FUNCTION
-
Function definitions
RULE
-
Rule definitions
PROCEDURE
-
Procedure definitions
SUBTYPE_CONSTRAINT
-
Subtype constraint definitions
INTERFACE
-
Interface definitions
Nested entities within other constructs:
PARAMETER
-
Function and procedure parameters
VARIABLE
-
Variables within functions, rules, and procedures
ATTRIBUTE
-
Entity attributes
DERIVED_ATTRIBUTE
-
Derived attributes in entities
INVERSE_ATTRIBUTE
-
Inverse attributes in entities
UNIQUE_RULE
-
Unique rules within entities
WHERE_RULE
-
Where rules within entities and types
ENUMERATION_ITEM
-
Items within enumeration types
INTERFACE_ITEM
-
Items within interfaces
INTERFACED_ITEM
-
Interfaced items
SCHEMA_VERSION
-
Schema version information
SCHEMA_VERSION_ITEM
-
Schema version items
TYPE subtype exclusion
For TYPE elements, you can exclude specific subtypes using the TYPE:SUBTYPE
syntax:
# Exclude only SELECT types
expressir coverage --exclude=TYPE:SELECT schemas/resources/action_schema/action_schema.exp
# Exclude multiple TYPE subtypes
expressir coverage --exclude=TYPE:SELECT,TYPE:ENUMERATION schemas/resources/action_schema/action_schema.exp
FUNCTION subtype exclusion
For FUNCTION elements, you can exclude inner functions (functions nested within
other functions, rules, or procedures) using the FUNCTION:INNER
syntax:
# Exclude inner functions from coverage analysis
expressir coverage --exclude=FUNCTION:INNER schemas/resources/action_schema/action_schema.exp
# Combine with other exclusions
expressir coverage --exclude=TYPE:SELECT,FUNCTION:INNER schemas/resources/action_schema/action_schema.exp
This is useful when you want to focus documentation coverage on top-level functions while excluding nested helper functions that may not require individual documentation. The exclusion works recursively, excluding functions at any nesting level within other constructs.
Valid FUNCTION subtypes that can be excluded:
INNER
-
Inner functions nested within other functions, rules, or procedures (at any depth)
FUNCTION outer_function : BOOLEAN; -- This inner function would be excluded with FUNCTION:INNER FUNCTION inner_helper_function : BOOLEAN; -- Even deeply nested functions are excluded FUNCTION deeply_nested_function : BOOLEAN; RETURN (TRUE); END_FUNCTION; RETURN (TRUE); END_FUNCTION; RETURN (TRUE); END_FUNCTION; RULE example_rule FOR (some_entity); -- Inner functions in rules are also excluded FUNCTION inner_function_in_rule : BOOLEAN; RETURN (TRUE); END_FUNCTION; WHERE WR1: inner_function_in_rule(); END_RULE; PROCEDURE example_procedure; -- Inner functions in procedures are also excluded FUNCTION inner_function_in_procedure : BOOLEAN; RETURN (TRUE); END_FUNCTION; END_PROCEDURE;
The FUNCTION:INNER
exclusion helps maintain focus on documenting the primary
API functions while ignoring implementation details of nested helper functions.
Ignoring files from coverage calculation
You can exclude entire files from the overall coverage calculation using the
--ignore-files
option. This is useful when you have files that should not
contribute to the overall documentation coverage statistics, such as test
schemas, example files, or legacy schemas.
# Use ignore files to exclude specific files from coverage calculation
expressir coverage --ignore-files ignore_list.yaml schemas/resources/
# Combine with other options
expressir coverage --ignore-files ignore_list.yaml --exclude=TYPE:SELECT --format=json schemas/resources/
Ignore files YAML format
The ignore files YAML should contain an array of file patterns. Each pattern can be either an exact file path or use glob patterns for matching multiple files.
# Array of file patterns to ignore
- examples/test_schema.exp # Exact file path
- examples/*_test_*.exp # Glob pattern for test files
- legacy/old_*.exp # Glob pattern for legacy files
- temp/temporary_schema.exp # Another exact path
Pattern matching behavior
File patterns in the ignore files YAML support:
-
Exact paths: Match specific files exactly
-
Glob patterns: Use
*
for wildcard matching -
Relative paths: Patterns are resolved relative to the YAML file’s directory
-
Absolute paths: Full system paths are also supported
# Examples of different pattern types
- schemas/action_schema/action_schema.exp # Exact relative path
- /full/path/to/schema.exp # Absolute path
- schemas/**/test_*.exp # Recursive glob pattern
- temp/*.exp # All .exp files in temp directory
Behavior of ignored files
When files are ignored using the --ignore-files
option:
-
Excluded from overall statistics: Ignored files do not contribute to the overall coverage percentage calculation
-
Still processed and reported: Ignored files are still analyzed and appear in the output, but marked with an
ignored: true
flag -
Separate reporting section: In JSON/YAML output formats, ignored files appear in both the main
files
section (with the ignored flag) and in a separateignored_files
section -
Overall statistics updated: The overall statistics include additional fields showing the count of ignored files and entities
{
"overall": {
"coverage_percentage": 75.0,
"total_entities": 100,
"documented_entities": 75,
"undocumented_entities": 25,
"ignored_files_count": 2,
"ignored_entities_count": 15
},
"files": [
{
"file": "schemas/main_schema.exp",
"ignored": false,
"coverage": 80.0,
"total": 50,
"documented": 40,
"undocumented": ["entity1", "entity2"]
},
{
"file": "examples/test_schema.exp",
"ignored": true,
"matched_pattern": "examples/*_test_*.exp",
"coverage": 20.0,
"total": 10,
"documented": 2,
"undocumented": ["test_entity1", "test_entity2"]
}
],
"ignored_files": [
{
"file": "examples/test_schema.exp",
"matched_pattern": "examples/*_test_*.exp",
"coverage": 20.0,
"total": 10,
"documented": 2,
"undocumented": ["test_entity1", "test_entity2"]
}
]
}
Error handling
The ignore files functionality handles various error conditions gracefully:
-
Missing YAML file: If the specified ignore files YAML doesn’t exist, a warning is displayed and coverage analysis continues normally
-
Invalid YAML format: If the YAML file is malformed or doesn’t contain an array, a warning is displayed and the file is ignored
-
Non-matching patterns: Patterns that don’t match any files are silently ignored (no error or warning)
-
Permission errors: File access errors are reported as warnings
Use cases for ignore files
Common scenarios where ignore files are useful:
-
Test schemas: Exclude test or example schemas from production coverage metrics
-
Legacy files: Ignore old schemas that are being phased out
-
Generated files: Exclude automatically generated schemas
-
Work-in-progress: Temporarily ignore files under development
-
Different coverage standards: Apply different documentation standards to different file sets
Valid TYPE subtypes that can be excluded:
AGGREGATE
-
Aggregate type
ARRAY
-
Array type
BAG
-
Bag type
BINARY
-
Binary type
BOOLEAN
-
Boolean type
ENUMERATION
-
Enumeration type
TYPE uuid_relationship_role = ENUMERATION OF (supersedes, merge, split, derive_from, same_as, similar_to); END_TYPE;
GENERIC
-
Generic type
GENERIC_ENTITY
-
Generic entity type
TYPE uuid_attribute_select = EXTENSIBLE GENERIC_ENTITY SELECT; END_TYPE;
INTEGER
-
Integer type
LIST
-
List type
TYPE uuid_list_item = LIST [1:?] OF UNIQUE LIST [1:?] OF UNIQUE uuid_attribute_select; END_TYPE;
LOGICAL
-
Logical type
NUMBER
-
Number type
REAL
-
Real type
SELECT
-
Select type
TYPE uuid_set_or_list_attribute_select = SELECT (uuid_list_item, uuid_set_item); END_TYPE;
SET
-
Set type
TYPE uuid_set_item = SET [1:?] OF uuid_attribute_select; END_TYPE;
STRING
-
String type
TYPE uuid = STRING (36) FIXED; END_TYPE;
This is particularly useful since TYPE entities with certain subtypes (like SELECT) often have descriptions generated by template strings and may not require individual remark item coverage.
Note
|
ISO 10303 excludes documentation coverage for TYPE:SELECT and TYPE:ENUMERATION. |
If you specify an invalid entity type or subtype, the command will display an error message with the list of valid options.
expressir coverage schemas/resources/ --format json
When using JSON or YAML output formats, all file and directory paths are displayed relative to the current working directory:
- file: "schemas/resources/action_schema/action_schema.exp"
file_basename: action_schema.exp
directory: "schemas/resources/action_schema"
# ... other fields
Coverage output
The default text output displays:
-
Directory coverage (when analyzing multiple directories)
-
File coverage, showing:
-
File path
-
List of undocumented entities
-
Coverage percentage
-
-
Overall documentation coverage statistics
This helps identify areas of your EXPRESS schemas that need documentation improvement.
Usage: Ruby
Parsing EXPRESS schema files
General
The library provides two main methods for parsing EXPRESS files.
Parsing a single file
Use the from_file
method to parse a single EXPRESS schema file:
# Parse a single file
repository = Expressir::Express::Parser.from_file("path/to/schema.exp")
# With options
repository = Expressir::Express::Parser.from_file(
"path/to/schema.exp",
skip_references: false, # Set to true to skip resolving references
include_source: true, # Set to true to include original source in the model
root_path: "/base/path" # Optional base path for relative paths
)
The from_file
method will raise a SchemaParseFailure
exception if the schema
fails to parse, providing information about the specific file and the parsing
error:
begin
repository = Expressir::Express::Parser.from_file("path/to/schema.exp")
rescue Expressir::Express::Error::SchemaParseFailure => e
puts "Failed to parse schema: #{e.message}"
puts "Filename: #{e.filename}"
puts "Error details: #{e.parse_failure_cause.ascii_tree}"
end
Parsing multiple files
Use the from_files
method to parse multiple EXPRESS schema files:
# Parse multiple files
files = ["schema1.exp", "schema2.exp", "schema3.exp"]
repository = Expressir::Express::Parser.from_files(files)
You can provide a block to track loading progress and handle errors:
files = ["schema1.exp", "schema2.exp", "schema3.exp"]
repository = Expressir::Express::Parser.from_files(files) do |filename, schemas, error|
if error
puts "Error loading #{filename}: #{error.message}"
# Skip the file with an error or take other action
else
puts "Successfully loaded #{schemas.length} schemas from #{filename}"
end
end
Filtering out schemas
You can filter out specific schemas from the repository easily since
Expressir::Model::Repository
implements Enumerable
.
schema_yaml = YAML.load_file('documents/iso-10303-41/schemas.yaml')
schema_paths = schema_yaml['schemas'].map {|x,y| y['path'].gsub("../../", "")}
repo = Expressir::Express::Parser.from_files(schema_paths)
filtered_schemas = ["action_schema", "date_time_schema"]
repo.select do |schema|
filtered_schemas.include?(schema.name)
end.each do |schema|
puts "Schema name: #{schema.name}"
puts "Schema file: #{schema.file}"
puts "Schema version: #{schema.version}"
end
Convert models to Liquid
Use to_liquid
method to convert the models of Expressir::Model::*
to liquid
drop models (Expressir::Liquid::*
).
Example:
repo = Expressir::Express::Parser.from_file("path/to/file.exp")
repo_drop = repo.to_liquid
where repo
is an instance of Expressir::Model::Repository
and
repo_drop
is an instance of Expressir::Liquid::RepositoryDrop
.
The Liquid drop models of Expressir::Liquid::*
have the same attributes
(model_attr
) as the models of Expressir::Model::*
.
For example, Expressir::Model::Repository
has the following attributes:
-
schemas
and each Expressir::Model::Declarations::Schema
has the following attributes:
-
file
-
version
-
interfaces
-
constants
-
entities
-
subtype_constraints
-
functions
-
rules
-
procedures
Thus, Expressir::Liquid::Repository
has the same attribute schemas
and Expressir::Liquid::Declarations::SchemaDrop
has same attribute file
.
repo = Expressir::Express::Parser.from_file("path/to/file.exp")
repo_drop = repo.to_liquid
schema = repo_drop.schemas.first
schema.file = "path/to/file.exp"
Documentation coverage analysis
Expressir’s documentation coverage feature can be used programmatically to analyze and report on documentation coverage of EXPRESS schemas.
# Create a coverage report from a file
report = Expressir::Coverage::Report.from_file("path/to/schema.exp")
# Or create a report from a repository
repository = Expressir::Express::Parser.from_file("path/to/schema.exp")
report = Expressir::Coverage::Report.from_repository(repository)
# Access overall statistics
puts "Overall coverage: #{report.coverage_percentage}%"
puts "Total entities: #{report.total_entities.size}"
puts "Documented entities: #{report.documented_entities.size}"
puts "Undocumented entities: #{report.undocumented_entities.size}"
# Access file-level reports
report.file_reports.each do |file_report|
puts "File: #{file_report[:file]}"
puts " Coverage: #{file_report[:coverage]}%"
puts " Total entities: #{file_report[:total]}"
puts " Documented entities: #{file_report[:documented]}"
puts " Undocumented entities: #{file_report[:undocumented].join(', ')}"
end
# Access directory-level reports
report.directory_reports.each do |dir_report|
puts "Directory: #{dir_report[:directory]}"
puts " Coverage: #{dir_report[:coverage]}%"
puts " Total entities: #{dir_report[:total]}"
puts " Documented entities: #{dir_report[:documented]}"
puts " Undocumented entities: #{dir_report[:undocumented]}"
puts " Number of files: #{dir_report[:files]}"
end
# Generate a structured hash representation
report_hash = report.to_h # Contains overall, directories and files sections
You can also use the core methods directly to check documentation status:
# Check if an entity has documentation
schema = repository.schemas.first
entity = schema.entities.first
if Expressir::Coverage.entity_documented?(entity)
puts "Entity #{entity.id} is documented"
else
puts "Entity #{entity.id} is not documented"
end
# Find all entities in a schema
all_entities = Expressir::Coverage.find_entities(schema)
puts "Found #{all_entities.size} entities in schema #{schema.id}"
Contributing
First, thank you for contributing! We love pull requests from everyone. By participating in this project, you hereby grant Ribose Inc. the right to grant or transfer an unlimited number of non exclusive licenses or sub-licenses to third parties, under the copyright covering the contribution to use the contribution by all means.
Here are a few technical guidelines to follow:
-
Open an issues to discuss a new feature.
-
Write tests to support your new feature.
-
Make sure the entire test suite passes locally and on CI.
-
Open a Pull Request.
-
Squash your commits after receiving feedback.
-
Party!
Documentation
Expressir provides detailed documentation on various aspects of its functionality:
-
Benchmarking: Learn about Expressir’s built-in capabilities for measuring schema loading performance, particularly useful for large schemas or when optimizing performance.
-
Liquid Integration: Documentation on how to use Expressir models with Liquid templates for flexible document generation.
License
Expressir is distributed under the BSD 2-clause license.
Note
|
Expressir originally contained some code from the NIST Reeper project but no longer contains them. |
The NIST Reeper license is reproduced below:
This software was funded by NIST and developed by EuroSTEP. Pursuant to title 17 Section 105 of the United States Code this software is not subject to copyright protection and is in the public domain.
We would appreciate acknowledgment if the software is used. Links to non-Federal Government Web sites do not imply NIST endorsement of any particular product, service, organization, company, information provider, or content.
Credits
Copyright Ribose Inc.