Suma: utility for the STEP Unified Model-driven Architecture
Purpose
SUMA, the "STEP Unified Model-Based Standards Architecture", is a authoring and publication system developed for the management of EXPRESS schemas and its documentation.
This utility is used by the ISO 10303 collection to build the following artifacts:
-
the STEP Resource Library (SRL)
Features
-
EXPRESS schema management for STEP standards
-
Document collection building and compilation
-
EXPRESS links validation and extraction
-
Schema listing file generation
-
Integration with the Metanorma ecosystem
-
Progress tracking for schema loading operations
Installation
$ gem install sumaUsage: CLI
General
# Defaults to `suma help`
$ suma
Commands:
suma build METANORMA_SITE_MANIFEST # Build collection specified in site manifest (`metanorma*.yml`)
suma convert-jsdai XML_FILE IMAGE_FILE OUTPUT_DIR # Convert JSDAI XML and image files to SVG and EXP files
suma extract-terms SCHEMA_MANIFEST_FILE GLOSSARIST_OUTPUT_PATH # Extract terms from SCHEMA_MANIFEST_FILE into Glossarist v2 format
suma generate-schemas METANORMA_MANIFEST_FILE SCHEMA_MANIFEST_FILE # Generate EXPRESS schema manifest file from Metanorma site manifest
suma help [COMMAND] # Describe available commands or one specific command
suma reformat EXPRESS_FILE_PATH # Reformat EXPRESS files
suma validate SUBCOMMAND ...ARGS # Validate express documentsBuild command
The build command processes a Metanorma site manifest and generates the specified output.
$ suma build METANORMA_SITE_MANIFEST [options]Parameters:
METANORMA_SITE_MANIFEST-
This is the path to the Metanorma site manifest, typically
metanorma.yml.
Options:
--[no-]compile-
Compile or skip compilation of collection (default: true)
-
--schemas-all-path=PATH,-s PATH -
Generate file that contains all schemas in the collection
The generated schemas-*.yaml file name is derived from the input file name
with the word metanorma replaced with schemas.
$ bundle exec suma build metanorma-test.yml$ bundle exec suma build metanorma-srl.yml$ bundle exec suma build --no-compile metanorma-srl.yml
# => generates schemas-srl.ymlAll documents need to have a schemas.yaml in their document root that lists
out which schemas the document includes.
Reformat command
The reformat command provides utilities for reformatting EXPRESS files.
$ suma reformat EXPRESS_FILE_PATH [options]Parameters:
EXPRESS_FILE_PATH-
Path to an EXPRESS file or a folder containing EXPRESS files
Options:
-
--[no-]recursive,-r -
Select EXPRESS files recursively based on the specified folder path (default: false)
$ bundle exec suma reformat `pwd` -rThis command:
-
Loads the EXPRESS files specified in the
EXPRESS_FILE_PATH -
Reformats and saves the loaded EXPRESS files
Validate command
General
The validate command groups various validation utilities for EXPRESS documents.
$ suma validate SUBCOMMAND [options]Subcommands:
links-
Validate EXPRESS links
ascii-
Check for non-ASCII characters in EXPRESS files
Links subcommand
The links subcommand extracts and validates EXPRESS links without creating intermediate files.
$ suma validate links SCHEMAS_FILE DOCUMENTS_PATH [OUTPUT_FILE]Parameters:
SCHEMAS_FILE-
Path to the schemas file (default: "schemas-srl.yml")
DOCUMENTS_PATH-
Path to the documents directory (default: "documents")
OUTPUT_FILE-
Path to write validation results (default: "validation_results.txt")
$ bundle exec suma validate links schemas-srl.yml documents validation_results.txtThis command:
-
Loads the schemas specified in the
SCHEMAS_FILE -
Searches for EXPRESS links in all AsciiDoc files in the
DOCUMENTS_PATH -
Validates these links against the loaded schemas
-
Writes validation results to the
OUTPUT_FILE -
Provides progress bars to track schema loading and link validation
ASCII subcommand
The ascii subcommand detects non-ASCII characters in EXPRESS files and reports on those exact lines, providing replacement suggestions.
$ suma validate ascii EXPRESS_FILE_PATH [options]Parameters:
EXPRESS_FILE_PATH-
Path to an EXPRESS file or a folder containing EXPRESS files
Options:
-
--[no-]recursive,-r -
Select EXPRESS files recursively based on the specified folder path (default: false)
-
--[no-]yaml,-y -
Output results in YAML format for machine processing (default: false)
$ bundle exec suma validate ascii ../iso-10303/schemas -r$ bundle exec suma validate ascii ../iso-10303/schemas -r -y > validation.ymlThis command:
-
Loads the EXPRESS files specified in the
EXPRESS_FILE_PATH -
Scans each line for non-ASCII characters
-
Reports detailed information about each violation, including:
-
Filename and line number
-
The exact line content
-
Visual indication of the non-ASCII sequence location
-
Character details with hexadecimal representation
-
-
Provides specific replacement suggestions:
-
For math symbols: provides equivalent AsciiMath notation
-
For other non-ASCII characters: provides ISO 10303-11 encoded string literal format
-
-
Displays a summary table showing:
-
File path (directory/filename)
-
Each non-ASCII symbol found
-
Suggested replacement for each symbol
-
Number of occurrences of each character
-
Totals row showing unique character count and overall occurrences
-
-
Summarizes findings across all scanned files
-
Optionally outputs structured data in YAML format with detailed occurrence information
Human-readable output format example:
/path/to/file.exp:
Line 42, Column 15:
ENTITY some_entity (name: STRING, description: "résumé");
^^^^^
"é" - Hex: 0xe9, UTF-8 bytes: 0xc3 0xa9
Replacement: ISO 10303-11: "000000E9"
"s" - Hex: 0x73, UTF-8 bytes: 0x73
"u" - Hex: 0x75, UTF-8 bytes: 0x75
"m" - Hex: 0x6d, UTF-8 bytes: 0x6d
"é" - Hex: 0xe9, UTF-8 bytes: 0xc3 0xa9
Replacement: ISO 10303-11: "000000E9"
Found 1 non-ASCII sequence(s) in file.exp
Summary:
Scanned 3 EXPRESS file(s)
Found 1 non-ASCII sequence(s) in 1 file(s)
+------------------+--------------------+-----------------------------+-------------+
| File | Symbol | Replacement | Occurrences |
+------------------+--------------------+-----------------------------+-------------+
| path/to/file.exp | "é" (0xe9) | ISO 10303-11: "000000E9" | 2 |
+------------------+--------------------+-----------------------------+-------------+
| TOTAL | 1 unique | | 2 |
+------------------+--------------------+-----------------------------+-------------+Japanese Character Example
For Japanese characters like 神戸 (Kobe), the command will provide ISO 10303-11 encoded string literal replacements:
"神" - Hex: 0x795e, UTF-8 bytes: 0xe7 0xa5 0x9e
Replacement: ISO 10303-11: "0000795E"
"戸" - Hex: 0x6238, UTF-8 bytes: 0xe6 0x88 0xb8
Replacement: ISO 10303-11: "00006238"Math Symbol Example
For mathematical symbols, the command will provide equivalent AsciiMath notation:
"×" - Hex: 0xd7, UTF-8 bytes: 0xc3 0x97
Replacement: AsciiMath: xxGenerate schemas command
The suma generate-schemas command generates an EXPRESS schema manifest file
containing all schemas of documents referenced in the Metanorma manifest file,
recursively.
$ suma generate-schemas METANORMA_MANIFEST_FILE SCHEMA_MANIFEST_FILE [options]Parameters:
METANORMA_MANIFEST_FILE-
Path to the Metanorma manifest file (e.g.: "metanorma-smrl-all.yml")
Options:
-
--exclude_path,-e -
Exclude schemas by pattern (e.g.
*_lf.exp)
$ bundle exec suma generate-schemas metanorma-smrl-all.yml schemas-smrl-all.yml
# => generates schemas-smrl-all.yml*_lf.exp
$ bundle exec suma generate-schemas metanorma-smrl-all.yml schemas-smrl-all.yml -e *_lf.exp
# => generates schemas-smrl-all.yml without schemas with names like *_lf.expAll documents need to have a schemas.yaml in their document root that lists
out which schemas the document includes.
Extract terms command
The "extract terms" command is implemented for ISO 10303-2, and could also be used for other EXPRESS schema collections that require term extraction for glossary or dictionary applications.
The suma extract-terms command extracts terms from EXPRESS schemas and
generates a Glossarist v2 dataset in the output directory. This command processes
various types of STEP schemas and creates standardized terminology datasets
suitable for glossary and dictionary applications.
$ suma extract-terms SCHEMA_MANIFEST_FILE GLOSSARIST_OUTPUT_PATH [options]Parameters:
SCHEMA_MANIFEST_FILE-
Path to the schema manifest file that lists all schemas to process (e.g., "schemas-smrl-all.yml")
GLOSSARIST_OUTPUT_PATH-
Path to the output directory where the Glossarist v2 dataset will be generated
Options:
-
--language_code,-l -
Language code for the Glossarist dataset (default: "eng")
Supported schema types
The command supports extraction from the following EXPRESS schema types:
-
ARM (application reference model) - application module schemas ending with
_arm -
MIM (module implementation model) - application module schemas ending with
_mim -
Resource schemas - General resource schemas
-
BOM (business object model) - business object model schemas ending with
_bom
|
Note
|
The "Long Form" EXPRESS schemas (ending with _lf) should be excluded
from the manifest file as they do not contain definitions of ENTITY objects.
|
Extracted concepts
The command extracts ENTITY objects from EXPRESS schemas, with the following
information:
-
a unique identifier in form of
{schema_name}.{entity_name}(whereentity_nameis a slug derived from theENTITYname); -
a generated definition depending on the schema type and entity type;
-
a "Note to entry" from its first coherent textual element described in Annotated EXPRESS;
-
source information including: schema name and schema version;
-
domain in the pattern of:
{domain_type}: {schema_name}, wheredomain_typeis one ofapplication module,resource, orbusiness object model.
Output format
The command generates a Glossarist v2 compliant dataset with:
-
concept/directory containing concept definition files in YAML -
localized_concept/directory containing localized concept files in YAML
$ bundle exec suma extract-terms schemas-smrl-all.yml glossarist_output
# => generates glossarist_output/concept/*.yaml and
# glossarist_output/localized_concept/*.yaml$ bundle exec suma extract-terms schemas-activity-modules.yml terms_output
# => processes only schemas listed in the manifest fileConvert JSDAI image outputs to SVG and Annotated EXPRESS
The suma convert-jsdai command converts JSDAI EXPRESS-G diagram outputs
(XML + image file) into SVG and Annotated EXPRESS formats suitable for
Metanorma documentation.
JSDAI (Java Step Data Access Interface) Java tool is used by ISO/TC 184/SC 4 to create EXPRESS-G diagrams.
JSDAI generates two files for each diagram:
-
a raster image file (GIF or JPEG)
-
an XML file containing image metadata and clickable area definitions
This command converts these inputs into:
-
an SVG file that embeds the raster image as Base64 with clickable rectangular areas
-
an Annotated EXPRESS file with a Metanorma
svgmapblock for easy copy-paste
$ suma convert-jsdai XML_FILE IMAGE_FILE OUTPUT_DIRWhere:
XML_FILE-
Path to the JSDAI XML file (e.g., "action_schemaexpg1.xml")
IMAGE_FILE-
Path to the raster image file (GIF or JPEG format)
OUTPUT_DIR-
Path to the output directory where SVG and EXP files will be generated
$ bundle exec suma convert-jsdai \
documents/resources/action_schema/action_schemaexpg1.xml \
documents/resources/action_schema/action_schemaexpg1.gif \
output/
# => generates:
# output/action_schemaexpg1.svg
# output/action_schemaexpg1.expThis command:
-
Parses the JSDAI XML file to extract image metadata and clickable area definitions
-
Reads the raster image file and converts it to base64 format
-
Generates an SVG file with:
-
The embedded base64-encoded image
-
Clickable rectangular areas (
<a>and<rect>elements) corresponding to the XML definitions -
Proper viewBox dimensions matching the source image
-
-
Generates an Annotated EXPRESS file containing:
-
A Metanorma
svgmapblock with numbered cross-references -
Proper anchor IDs for document integration
-
Cross-reference targets extracted from the XML href attributes
-
The generated SVG and EXP files work together through a numbered mapping system:
-
In the SVG file, each clickable area is assigned a sequential number:
<a href="1"><rect .../></a> <a href="2"><rect .../></a> <a href="3"><rect .../></a>
-
In the EXPRESS file, the
svgmapblock maps these numbers to targets which are either EXPRESS or AsciiDoc anchors.Example 4. Resource schema diagram with SVG and Annotated EXPRESS cross-references(*"action_schema.__expressg" [[action_schema_expg1]] [.svgmap] ==== image::action_schemaexpg1.svg[] * <<express:basic_attribute_schema>>; 1 * <<express:action_schema>>; 2 * <<express:support_resource_schema>>; 3 ==== *)Example 5. Module schema diagram with SVG and Annotated EXPRESS cross-references(*"Activity_mim.__expressg" [[Activity_mim_expg1]] [.svgmap] ==== image::mimexpg1.svg[] * <<Activity_mim_expg2>>; 1 * <<express:action_schema>>; 2 * <<Activity_method_mim_expg1>>; 3 * <<express:basic_attribute_schema>>; 4 * <<express:management_resources_schema>>; 5 ==== *) -
When rendered in Metanorma, clicking on "area 1" in the SVG, will navigate to the
express:basic_attribute_schemaanchor, "area 2" toexpress:action_schema, and so on.
The mapping is derived from the original JSDAI XML file, where each <img.area>
element contains:
-
coordsattribute -
converted to SVG
<rect>dimensions -
hrefattribute -
converted to EXPRESS cross-reference target in the
svgmapblock - Sequential position
-
assigned as the numbered href in both SVG and
svgmaplist
Export schemas command
General
The suma export command exports EXPRESS schemas from manifest files and/or
standalone EXPRESS files to a specified output directory.
This command is useful for extracting plain or annotated EXPRESS schemas for distribution or further processing.
$ suma export -o OUTPUT_DIR FILE1 [FILE2 FILE3 ...]Parameters:
FILE1 [FILE2 FILE3 …]-
One or more files to export. Each file can be:
- An EXPRESS schema manifest file (
.yml,.yaml) -
schemas listed in the manifest will be exported
- An standalone EXPRESS file (
.exp) -
the schema will be exported directly
- An EXPRESS schema manifest file (
Options:
-
--output=PATH,-o PATH -
(required) Output directory path
--[no-]annotations-
Include annotations (remarks/comments) in exported schemas (default: false)
--[no-]zip-
Create ZIP archive of exported schemas (default: false)
Behavior
The command exports schemas based on the input file types:
For EXPRESS schema manifest files:
-
Schemas are exported while preserving the directory structure from the manifest file paths
-
Schemas under
resources/are exported toOUTPUT/resources/ -
Schemas under
modules/are exported toOUTPUT/modules/ -
Other categories (
business_object_models/,core_model/) follow the same pattern
For standalone EXPRESS files:
-
Schemas are exported directly to the output directory root
-
Output filename format:
{schema_id}.exp
By default, schemas are exported without annotations (plain EXPRESS). Use the
--annotations flag to include remarks and comments.
The --zip flag creates a ZIP archive in addition to the directory output,
named OUTPUT.zip.
$ bundle exec suma export -o express-files schemas-srl.yml
# => generates express-files/ directory with plain EXPRESS schemas$ bundle exec suma export -o express-files geometry_schema.exp
# => generates express-files/geometry_schema.exp$ bundle exec suma export -o express-files schema1.exp schema2.exp schema3.exp
# => generates express-files/{schema1,schema2,schema3}.exp$ bundle exec suma export -o express-files schemas-srl.yml additional-schemas.yml
# => exports schemas from both manifest files$ bundle exec suma export -o express-files schemas-srl.yml geometry_schema.exp
# => exports schemas from manifest with directory structure
# plus geometry_schema.exp at the root$ bundle exec suma export -o express-files --annotations schemas-srl.yml
# => generates express-files/ directory with annotated EXPRESS schemas$ bundle exec suma export -o express-files --zip schemas-srl.yml
# => generates both:
# - express-files/ directory
# - express-files.zip archive$ bundle exec suma export -o express-files \
--annotations \
--zip \
schemas-srl.yml geometry_schema.exp
# => generates annotated schemas in both directory and ZIP formatOutput structure
The exported directory structure depends on the input file types:
express-files/
├── geometry_schema.exp # standalone EXPRESS file (at root)
├── topology_schema.exp # standalone EXPRESS file (at root)
├── resources/ # from manifest files
│ ├── action_schema/
│ │ └── action_schema.exp
│ └── ...
├── modules/ # from manifest files
│ ├── activity/
│ │ ├── arm.exp
│ │ └── mim.exp
│ └── ...
├── business_object_models/ # from manifest files
│ └── ...
└── core_model/ # from manifest files
└── ...Compare schemas command
General
The suma compare command compares two EXPRESS schemas using eengine and
generates an EXPRESS Changes YAML file tracking the differences.
This command is essential for managing schema evolution across versions, particularly when working with multiple git branches of the same repository.
$ suma compare TRIAL_SCHEMA REFERENCE_SCHEMA --version VERSION [options]Parameters:
TRIAL_SCHEMA-
Path to the new/trial EXPRESS schema file
REFERENCE_SCHEMA-
Path to the old/reference EXPRESS schema file
Options:
-
--version=VERSION,-v VERSION -
(required) Version number for this change version
-
--output=PATH,-o PATH -
Output Change YAML file path (default:
{schema}.changes.yamlin trial schema directory) --mode=MODE-
Schema comparison mode:
resourceormodule(default:resource) --trial-stepmod=PATH-
Override auto-detected trial repository root
--reference-stepmod=PATH-
Override auto-detected reference repository root
--verbose-
Enable verbose output
Typical workflow
The compare command is designed for a two-repository workflow where you have different branches checked out:
# 1. Check out old version at /path/to/repo-old
# 2. Check out new version at /path/to/repo-new
# 3. Compare schemas:
$ suma compare \
/path/to/repo-new/schemas/resources/action_schema/action_schema.exp \
/path/to/repo-old/schemas/resources/action_schema/action_schema.exp \
--version 2Behavior
The command:
-
Auto-detects git repository roots from schema file paths
-
Uses detected roots as stepmod paths for eengine
-
Generates or updates
.changes.yamlfile in the trial schema directory -
Replaces existing change version if the version already exists
-
Adds new change version if the version is different
Requirements
This command requires eengine to be installed:
$ suma compare \
~/iso-10303-new/schemas/resources/support_resource_schema/support_resource_schema.exp \
~/iso-10303-old/schemas/resources/support_resource_schema/support_resource_schema.exp \
--version 2
# => generates or updates support_resource_schema.changes.yaml$ suma compare schema_new.exp schema_old.exp --version 2 --verbose
# Shows: eengine version, detected repo roots, XML generation, etc.$ suma compare schema_new.exp schema_old.exp \
--version 2 \
--output /path/to/custom.changes.yaml$ suma compare schema_new.exp schema_old.exp \
--version 2 \
--trial-stepmod ~/repo-new \
--reference-stepmod ~/repo-oldGenerated Change YAML format
The command generates EXPRESS Changes YAML files in the ISO 10303 format:
---
schema: support_resource_schema
versions:
- version: '2'
description: 'TYPE text: Underlying Type changed'
modifications:
- type: TYPE
name: textFor complete format documentation, see the EXPRESS Changes specification.
Expressir command
General
The suma expressir command provides access to all Expressir commands for
working with EXPRESS schemas. This includes documentation coverage analysis,
schema formatting, validation, and other EXPRESS-related operations.
Expressir is a comprehensive toolkit for EXPRESS schema processing that Suma integrates to provide additional functionality beyond Suma’s core features.
$ suma expressir SUBCOMMAND ...ARGS [options]Available subcommands:
coverage-
Check documentation coverage of EXPRESS schemas
format-
Pretty print EXPRESS schemas
clean-
Strip remarks and prettify EXPRESS schemas
validate-
Validate EXPRESS schemas
benchmark-
Benchmark schema loading performance
version-
Display expressir version
Coverage subcommand
The coverage subcommand analyzes EXPRESS schemas to determine documentation
coverage, reporting how many EXPRESS objects (entities, constants, functions,
types, etc.) have been documented with Annotated EXPRESS.
This addresses the need to measure and improve documentation quality in EXPRESS schema collections.
$ suma expressir coverage PATH [PATH...] [options]Parameters:
PATH-
One or more paths to analyze. Each path can be:
-
An EXPRESS schema manifest file (
.yml,.yaml) -
A single EXPRESS file (
.exp) -
A directory containing EXPRESS files
-
Options:
--format=FORMAT-
Output format:
text(default),json, oryaml --output=FILE-
Output file path for JSON/YAML formats (defaults to
coverage_report.jsonorcoverage_report.yaml) --exclude=TYPES-
Comma-separated list of EXPRESS entity types to exclude from coverage analysis (e.g.,
TYPE,CONSTANT,TYPE:SELECT) --ignore-files=FILE-
Path to YAML file containing array of file patterns to ignore from overall coverage calculation
$ suma expressir coverage schemas-srl.yml$ suma expressir coverage schemas-srl.yml --format=json --output=coverage.json$ suma expressir coverage schemas/ --exclude=TYPE:ENUMERATION,PARAMETER$ suma expressir coverage schemas-srl.yml additional-schemas.ymlThe coverage report includes:
-
Overall coverage statistics (total entities, documented entities, percentage)
-
Per-file coverage details
-
Per-directory coverage summaries
-
List of undocumented entities by type and name
Format subcommand
The format subcommand pretty-prints EXPRESS schemas.
$ suma expressir format PATHParameters:
PATH-
Path to the EXPRESS schema file to format
$ suma expressir format schema.expClean subcommand
The clean subcommand strips remarks and prettifies EXPRESS schemas.
$ suma expressir clean PATH [options]Parameters:
PATH-
Path to the EXPRESS schema file to clean
Options:
--output=FILE-
Output file path (defaults to stdout)
$ suma expressir clean schema.exp --output=cleaned_schema.expValidate subcommand
The validate subcommand validates EXPRESS schemas for syntactic correctness.
$ suma expressir validate PATH [PATH...]Parameters:
PATH-
One or more paths to EXPRESS schema files or directories to validate
$ suma expressir validate schema.exp
$ suma expressir validate schemas/Additional information
For complete documentation of all Expressir commands and options, see the Expressir documentation.
Usage: Ruby
General
Suma can be used programmatically in your Ruby applications. The following examples demonstrate common usage patterns.
Building collections
require 'suma'
# Build a collection with default settings
Suma::Processor.run(
metanorma_yaml_path: "metanorma-srl.yml",
schemas_all_path: "schemas-srl.yml",
compile: true,
output_directory: "_site"
)
# Generate schema listing without compilation
Suma::Processor.run(
metanorma_yaml_path: "metanorma-srl.yml",
schemas_all_path: "schemas-srl.yml",
compile: false,
output_directory: "_site"
)Working with schema configurations
require 'suma'
# Load schemas using SchemaConfig
schemas_file_path = "schemas-srl.yml"
schemas_config = Suma::SchemaConfig::Config.from_yaml(IO.read(schemas_file_path))
# Set the initial path to resolve relative paths
schemas_config.set_initial_path(schemas_file_path)
# Access schema information
schemas_config.schemas.each do |schema|
puts "Schema ID: #{schema.id}"
puts "Schema path: #{schema.path}"
endCopyright and license
Copyright Ribose. BSD 2-clause license.