Lyra

CRUD to Event Sourcing Transformation Engine

Part of the ORFEAS (Object-Relational to Event-Sourcing Architecture) Framework

Lyra is a Rails engine that records the writes an ActiveRecord application makes as events (RailsEventStore), and can then move the application, one checked step at a time, to treating those events as the source of truth.

The mode is application-wide. Per model you choose only whether it is monitored (monitor_with_lyra in the model, or config.models= in the initializer). Event sourcing takes one of four projection modes, which gives seven configurations:

• Monitor appends the event in after_create/after_update/after_destroy, inside the write's transaction (not after_commit).
• Hijack and event sourcing build the event in hooks prepended to ActiveRecord's persistence, after the model's before_* callbacks.
• Callback-bypassing writes (update_columns, update_column, touch, delete, update_all, delete_all, insert_all, insert_all!, upsert_all, and the updates of dependent: :nullify) on a monitored model publish bypass events; with config.strict_data_access = true they raise instead (except touch and the dependent: :nullify updates).
• Each record has its own stream, named "Model$id" (for example "Order$42").
• ES-NoProj writes no rows; it answers a query exactly from the events or raises Lyra::Projections::UnsupportedQuery.
• Switching modes after boot is gated: Lyra::ModeTransition.to! (and config.enable_hijack!, enable_monitor!, enable_event_sourcing!) checks that rows and events agree before a switch that changes the authoritative store, and the engine refuses to boot into such a switch without a certificate from rake lyra:mode:check. See Switching Modes.

• Ruby >= 4.0, Rails >= 8.0, rails_event_store ~> 3.0 (from lyra.gemspec).
• PostgreSQL. Lyra depends on the pg gem; the advisory locks behind Genesis and ES-Lazy, and id reservation in Hijack mode, need PostgreSQL.

The gem names differ from the files to require:

# Gemfile
gem "orfeas_lyra", "~> 0.9", require: "lyra"
gem "orfeas_pam_dsl", "~> 0.9", require: "pam_dsl"        # optional, privacy features
gem "orfeas_petri_flow", "~> 0.9", require: "petri_flow"  # optional, verification

Create the event store tables with the RailsEventStore 3 generator (bin/rails generate ruby_event_store:active_record:migration), then add an initializer. The Migration Guide walks through the whole path from Disabled to event sourcing.

Beyond the modes above (see the API Reference for each):

• Genesis: rows that predate Lyra get an Imported event on first use, or ahead of time with rake lyra:genesis.
• Failure policies: Hijack and event sourcing are fail-closed (the write rolls back); Monitor logs and continues, and rake lyra:repair brings the event log back in line with the tables.
• DualView: compare a record's table row with its event-sourced state; optionally sampled after commit (config.dual_view_sample_rate, off by default).
• Point-in-time state: Lyra.state_at(Model, id, time) and Model.as_of(time).
• Privacy (all opt-in; most need the pam_dsl gem): purpose-bound reads, an access log, privacy stamps in event metadata, erasure of one record's personal data from its row and its events (rake lyra:erase), and retention rules (rake lyra:retention:apply).
• Formal verification (needs petri_flow): Petri net models of each mode's write path, checked with PetriFlow (rake lyra:workflows:verify), and a check of the CRUD-to-event mapping (Lyra.verify_mapping!).
• Dashboard: mount Lyra::Engine, at: "/lyra". Outside development and test it answers 403 until config.dashboard_authorization (a proc run on the controller, e.g. ->(controller) { controller.current_user&.admin? }) says who may use it.

examples/blog_app/ is a small blog (users, posts, comments) in Monitor, on Rails 8.1, RailsEventStore 3 and the repository's PostgreSQL (docker compose up -d db, then bin/setup and bin/rails test in the app); see its README. The files examples/usage_examples.rb, examples/privacy_examples.rb and examples/privacy_policy_usage.rb show individual calls.

• Orthogonal Analysis: Compare static vs. dynamic views of system state
• Behavioral Analysis: Understand system evolution over time
• Migration Patterns: Study CRUD-to-ES transformation strategies
• Formal Verification: Check the CRUD-to-event mapping with Petri net models (petri_flow)
• Privacy Compliance: Research privacy-preserving event sourcing patterns

• Gradual Migration: Move to event sourcing one checked mode switch at a time
• Audit Trail: History of every write to a monitored model that goes through ActiveRecord
• Temporal Queries: Query state at a point in time
• Debugging: Replay events to understand issues
• CQRS Support: Commands and aggregates alongside the projected tables

• Few code changes: A model is monitored by one line or by naming it in the initializer; controllers and queries stay as they are (see Adopting Lyra for what this costs)
• Rollback: Every mode can be switched back to the previous one
• Monitoring: Compare table and event-sourced state (DualView)
• Validation: A switch that makes the events authoritative is checked first
• Performance Analysis: Measure overhead before committing

• Getting Started - Install, monitor one model, and check its events
• Migration Guide - Installation and the step-by-step move from CRUD to event sourcing
• API Reference - Configuration options, methods, rake tasks and errors
• Architecture Overview - System design and components
• Switching Modes - Who holds the mode, and how to switch safely (the deploy-time rule)
• Adopting Lyra - What "non-intrusive" means, the costs, and a checklist for a new codebase
• Performance - What each mode costs and why (measured on Lyra 0.9.0, with the SQL statements per write)
• Troubleshooting - Common problems
• Workflow Generator - The Petri net workflow models of Lyra's modes and their verification
• Changelog

• ORFEAS Framework Overview - The framework: naming, vision, components and migration path (the opening of ARCHITECTURE.md)
• Petri Nets Model - P/T nets for verification, CPNs for data modeling
• Matrix Analysis Model - Linear algebra approach

• PAM DSL Integration - Privacy policy DSL guide
• Privacy Compliance - GDPR compliance details

• PetriFlow Export - Export formats and integration
• PetriFlow Gem - Petri net library documentation
• PAM DSL Gem - Privacy DSL documentation

• Testing Guide - Testing strategy and setup

ORFEAS and Lyra are grounded in peer-reviewed research:

Pantelelis, M., & Kalloniatis, C. (2026). From ORM to OEM: Object-to-Events Mapping for the OLEP Era. IEEE Software. DOI: 10.1109/MS.2026.3734188

Pantelelis, M., & Kalloniatis, C. (2026). PAM: A Domain-Specific Language for Specifying Privacy Requirements from Regulation to Runtime. Requirements Engineering, Springer. DOI: 10.1007/s00766-026-00469-6

Pantelelis, M., & Kalloniatis, C. (2024). Create, Read, Update, Delete: Implications on Security and Privacy Principles regarding GDPR. ARES 2024, 21st International Workshop on Trust, Privacy and Security in the Digital Society (TrustBus), ACM. DOI: 10.1145/3664476.3670898

Pantelelis, M., & Kalloniatis, C. (2022). Mapping CRUD to Events: Towards an object to event-sourcing framework. 26th Pan-Hellenic Conference on Informatics (PCI 2022). DOI: 10.1145/3575879.3576006

Pantelelis, M., & Kalloniatis, C. (2022). Object Relational Mapping vs. Event-Sourcing: Systematic Review. EGOVIS 2022, Lecture Notes in Computer Science 13429, Springer, pp. 18–31. DOI: 10.1007/978-3-031-12673-4_2

Each paper's code is a tag or commit of this repository; its data and the rest of its replication material are on Zenodo. The case studies (the Aegean ePay testbed, Solidus with the Olist orders, the BPI 2017 loan application) are in the replication packages.

• Event Sourcing Patterns: Formal models for CRUD-to-event transformation
• Privacy-Preserving Systems: GDPR compliance in event-driven architectures
• Petri Net Theory: P/T nets for workflow verification, CPNs for advanced data modeling
• Matrix Analysis: Linear algebra approaches to causation and lineage
• Software Architecture: Gradual migration strategies for legacy systems

This proof-of-concept was developed using AI-assisted code generation to accelerate implementation while maintaining focus on theoretical contributions.

• Claude Code (Anthropic's agentic coding tool) - Primary development assistant for code implementation, testing, and documentation
• Claude (Anthropic) - For architectural discussions and design decisions

Important: All architectural decisions, design patterns, and theoretical foundations were specified by the researcher. AI assistance was used for:

• Code implementation following defined specifications
• Test generation based on requirements
• Documentation formatting and organization
• Code review and refactoring

This methodology enabled rapid prototyping while ensuring the theoretical rigor required for academic research.

Lyra is research software for the ORFEAS framework. Contributions and feedback are welcome!

• Bug Reports: Submit issues on GitHub
• Feature Requests: Suggest improvements or new features
• Research Collaboration: Collaborate on research extensions
• Documentation: Improve documentation and examples
• Testing: Add test cases and improve coverage

# Clone the repository
git clone https://github.com/mpantel/lyra-engine.git lyra
cd lyra

# Install dependencies
bundle install

# The engine's tests need PostgreSQL on localhost:5433 (user and password
# "postgres", database lyra_test; see test/dummy/config/database.yml).
# docker-compose.yml provides one:
bundle exec rake docker:start
docker exec lyra_postgres createdb -U postgres lyra_test

# Run the engine's tests, then pam_dsl's and petri_flow's
bundle exec rake test:all

# Build the gems
bundle exec rake build
(cd gems/pam_dsl && bundle exec rake build)
(cd gems/petri_flow && bundle exec rake build)

If you use this software in academic research, please cite:

@software{lyra2026,
  title={Lyra: CRUD to Event Sourcing Transformation Engine},
  author={Pantelelis, Michail},
  year={2026},
  note={Part of ORFEAS Framework},
  url={https://github.com/mpantel/lyra-engine},
  doi={10.5281/zenodo.17770613}
}

Lyra's object-to-events mapping, and how its modes compare in cost:

@article{pantelelis2026oem,
  title={From {ORM} to {OEM}: Object-to-Events Mapping for the {OLEP} Era},
  author={Pantelelis, Michail and Kalloniatis, Christos},
  journal={IEEE Software},
  year={2026},
  doi={10.1109/MS.2026.3734188}
}

PAM, the privacy DSL (gems/pam_dsl), from regulation to runtime:

@article{pantelelis2026pam,
  title={{PAM}: A Domain-Specific Language for Specifying Privacy Requirements from Regulation to Runtime},
  author={Pantelelis, Michail and Kalloniatis, Christos},
  journal={Requirements Engineering},
  publisher={Springer},
  year={2026},
  doi={10.1007/s00766-026-00469-6}
}

What each CRUD operation means for GDPR's security and privacy principles, the ground of PAM and Lyra's privacy events:

@inproceedings{pantelelis2024crud,
  title={Create, Read, Update, Delete: Implications on Security and Privacy Principles regarding {GDPR}},
  author={Pantelelis, Michail and Kalloniatis, Christos},
  booktitle={Proceedings of the 19th International Conference on Availability, Reliability and Security (ARES 2024), 21st International Workshop on Trust, Privacy and Security in the Digital Society (TrustBus)},
  articleno={178},
  pages={1--7},
  publisher={ACM},
  address={Vienna, Austria},
  year={2024},
  doi={10.1145/3664476.3670898}
}

The CRUD-to-events mapping as first proposed:

@inproceedings{pantelelis2022mapping,
  title={Mapping CRUD to Events: Towards an object to event-sourcing framework},
  author={Pantelelis, Michail and Kalloniatis, Christos},
  booktitle={26th Pan-Hellenic Conference on Informatics (PCI 2022)},
  year={2022},
  doi={10.1145/3575879.3576006}
}

The literature review behind it, ORM against Event Sourcing:

@inproceedings{pantelelis2022egovis,
  title={Object Relational Mapping vs. Event-Sourcing: Systematic Review},
  author={Pantelelis, Michail and Kalloniatis, Christos},
  booktitle={Electronic Government and the Information Systems Perspective (EGOVIS 2022)},
  series={Lecture Notes in Computer Science},
  volume={13429},
  pages={18--31},
  publisher={Springer},
  year={2022},
  doi={10.1007/978-3-031-12673-4_2}
}

• Rails Event Store - Event Store implementation
• Event Sourcing Pattern - Martin Fowler
• CQRS - Command Query Responsibility Segregation
• Petri Net Theory - Formal foundation
• GDPR Compliance - Privacy regulation

For questions, issues, and collaboration:

• Email: mpantel@aegean.gr
• GitHub Issues: Repository Issues
• Institution: University of the Aegean, Department of Information and Communication Systems Engineering

MIT License - see LICENSE file for details.

Built with Ruby, Petri Net Theory, and Formal Methods Part of the ORFEAS PhD Research Project