☯️ Ast::Merge
if ci_badges.map(&:color).detect { it != "green"} ☝️ let me know, as I may have missed the discord notification.
if ci_badges.map(&:color).all? { it == "green"} 👇️ send money so I can do more of this. FLOSS maintenance is now my full-time job.
I've summarized my thoughts in this blog post.
🌻 Synopsis
Ast::Merge is not typically used directly - instead, use one of the format-specific gems built on top of it.
The *-merge Gem Family
The *-merge gem family provides intelligent, AST-based merging for various file formats. At the foundation is tree_haver, which provides a unified cross-Ruby parsing API that works seamlessly across MRI, JRuby, and TruffleRuby.
| Gem | Version / CI | Language / Format |
Parser Backend(s) | Description |
|---|---|---|---|---|
| tree_haver |
|
Multi | Supported Backends: MRI C, Rust, FFI, Java, Prism, Psych, Commonmarker, Markly, Citrus, Parslet | Foundation: Cross-Ruby adapter for parsing libraries (like Faraday for HTTP) |
| ast-merge |
|
Text | internal |
Infrastructure: Shared base classes and merge logic for all *-merge gems |
| bash-merge |
|
Bash | tree-sitter-bash (via tree_haver) | Smart merge for Bash scripts |
| commonmarker-merge |
|
Markdown | Commonmarker (via tree_haver) | Smart merge for Markdown (CommonMark via comrak Rust) |
| dotenv-merge |
|
Dotenv | internal | Smart merge for .env files |
| json-merge |
|
JSON | tree-sitter-json (via tree_haver) | Smart merge for JSON files |
| jsonc-merge |
|
JSONC | tree-sitter-jsonc (via tree_haver) | ⚠️ Proof of concept; Smart merge for JSON with Comments |
| markdown-merge |
|
Markdown | Commonmarker / Markly (via tree_haver), Parslet | Foundation: Shared base for Markdown mergers with inner code block merging |
| markly-merge |
|
Markdown | Markly (via tree_haver) | Smart merge for Markdown (CommonMark via cmark-gfm C) |
| prism-merge |
|
Ruby |
Prism (prism std lib gem) |
Smart merge for Ruby source files |
| psych-merge |
|
YAML |
Psych (psych std lib gem) |
Smart merge for YAML files |
| rbs-merge |
|
RBS |
tree-sitter-rbs (via tree_haver), RBS (rbs std lib gem) |
Smart merge for Ruby type signatures |
| toml-merge |
|
TOML | Parslet + toml, Citrus + toml-rb, tree-sitter-toml (all via tree_haver) | Smart merge for TOML files |
Backend Platform Compatibility
tree_haver supports multiple parsing backends, but not all backends work on all Ruby platforms:
| Platform 👉️ TreeHaver Backend 👇️ |
MRI | JRuby | TruffleRuby | Notes |
|---|---|---|---|---|
| MRI (ruby_tree_sitter) | ✅ | ❌ | ❌ | C extension, MRI only |
| Rust (tree_stump) | ✅ | ❌ | ❌ | Rust extension via magnus/rb-sys, MRI only |
| FFI (ffi) | ✅ | ✅ | ❌ | TruffleRuby's FFI doesn't support STRUCT_BY_VALUE
|
| Java (jtreesitter) | ❌ | ✅ | ❌ | JRuby only, requires grammar JARs |
| Prism (prism) | ✅ | ✅ | ✅ | Ruby parsing, stdlib in Ruby 3.4+ |
| Psych (psych) | ✅ | ✅ | ✅ | YAML parsing, stdlib |
| Citrus (citrus) | ✅ | ✅ | ✅ | Pure Ruby PEG parser, no native dependencies |
| Parslet (parslet) | ✅ | ✅ | ✅ | Pure Ruby PEG parser, no native dependencies |
| Commonmarker (commonmarker) | ✅ | ❌ | ❓ | Rust extension for Markdown (via commonmarker-merge) |
| Markly (markly) | ✅ | ❌ | ❓ | C extension for Markdown (via markly-merge) |
Legend: ✅ = Works, ❌ = Does not work, ❓ = Untested
Why some backends don't work on certain platforms:
-
JRuby: Runs on the JVM; cannot load native C/Rust extensions (
.sofiles) -
TruffleRuby: Has C API emulation via Sulong/LLVM, but it doesn't expose all MRI internals that native extensions require (e.g.,
RBasic.flags,rb_gc_writebarrier) - FFI on TruffleRuby: TruffleRuby's FFI implementation doesn't support returning structs by value, which tree-sitter's C API requires
Example implementations for the gem templating use case:
| Gem | Purpose | Description |
|---|---|---|
| kettle-dev | Gem Development | Gem templating tool using *-merge gems |
| kettle-jem | Gem Templating | Gem template library with smart merge support |
Architecture: tree_haver + ast-merge
The *-merge gem family is built on a two-layer architecture:
Layer 1: tree_haver (Parsing Foundation)
tree_haver provides cross-Ruby parsing capabilities:
- Universal Backend Support: Automatically selects the best parsing backend for your Ruby implementation (MRI, JRuby, TruffleRuby)
- 10 Backend Options: MRI C extensions, Rust bindings, FFI, Java (JRuby), language-specific parsers (Prism, Psych, Commonmarker, Markly), and pure Ruby fallback (Citrus)
- Unified API: Write parsing code once, run on any Ruby implementation
-
Grammar Discovery: Built-in
GrammarFinderfor platform-aware grammar library discovery - Thread-Safe: Language registry with thread-safe caching
Layer 2: ast-merge (Merge Infrastructure)
Ast::Merge builds on tree_haver to provide:
-
Base Classes:
FreezeNode,MergeResultbase classes with unified constructors -
Shared Modules:
FileAnalysisBase,FileAnalyzable,MergerConfig,DebugLogger - Freeze Block Support: Configurable marker patterns for multiple comment syntaxes (preserve sections during merge)
-
Node Typing System:
NodeTypingfor canonical node type identification across different parsers -
Conflict Resolution:
ConflictResolverBasewith pluggable strategies -
Error Classes:
ParseError,TemplateParseError,DestinationParseError -
Region Detection:
RegionDetectorBase,FencedCodeBlockDetectorfor text-based analysis - RSpec Shared Examples: Test helpers for implementing new merge gems
Creating a New Merge Gem
require "ast/merge"
module MyFormat
module Merge
# Inherit from base classes and pass **options for forward compatibility
class SmartMerger < Ast::Merge::SmartMergerBase
DEFAULT_FREEZE_TOKEN = "myformat-merge"
def initialize(template, dest, my_custom_option: nil, **options)
@my_custom_option = my_custom_option
super(template, dest, **options)
end
protected
def analysis_class
FileAnalysis
end
def default_freeze_token
DEFAULT_FREEZE_TOKEN
end
def perform_merge
# Implement format-specific merge logic
# Returns a MergeResult
end
end
class FileAnalysis
include Ast::Merge::FileAnalyzable
def initialize(source, freeze_token: nil, signature_generator: nil, **options)
@source = source
@freeze_token = freeze_token
@signature_generator = signature_generator
# Process source...
end
def compute_node_signature(node)
# Return signature array for node matching
end
end
class ConflictResolver < Ast::Merge::ConflictResolverBase
def initialize(template_analysis, dest_analysis, preference: :destination,
add_template_only_nodes: false, match_refiner: nil, **options)
super(
strategy: :batch, # or :node, :boundary
preference: preference,
template_analysis: template_analysis,
dest_analysis: dest_analysis,
add_template_only_nodes: add_template_only_nodes,
match_refiner: match_refiner,
**options
)
end
protected
def resolve_batch(result)
# Implement batch resolution logic
end
end
class MergeResult < Ast::Merge::MergeResultBase
def initialize(**options)
super(**options)
@statistics = {merged_count: 0}
end
def to_my_format
to_s
end
end
class MatchRefiner < Ast::Merge::MatchRefinerBase
def initialize(threshold: 0.7, node_types: nil, **options)
super(threshold: threshold, node_types: node_types, **options)
end
def similarity(template_node, dest_node)
# Return similarity score between 0.0 and 1.0
end
end
end
endMerge Architecture: Choosing a Pattern
When building a new *-merge gem, the most important design decision is how to implement the merge logic. There are two proven patterns in the gem family. The choice depends on the structure of your format.
See also: MERGE_APPROACH.md for a detailed per-gem reference with real-world examples.
Pattern 1: Inline SmartMerger (recommended for new gems)
The SmartMerger handles all merge logic directly in perform_merge. No ConflictResolver class is needed. resolver_class returns nil.
Used by: prism-merge (Ruby), bash-merge (Bash), dotenv-merge (dotenv)
class SmartMerger < Ast::Merge::SmartMergerBase
protected
def resolver_class
nil # No ConflictResolver — merge logic is inline
end
def perform_merge
template_by_sig = build_signature_map(@template_analysis)
dest_by_sig = build_signature_map(@dest_analysis)
consumed_template_indices = Set.new
sig_cursor = Hash.new(0)
# Phase 1: template-only nodes (if add_template_only_nodes)
# Phase 2: walk dest nodes, match by signature, emit result
# Phase 3: (implicit) unmatched template nodes from Phase 1
@result
end
def build_signature_map(analysis)
map = Hash.new { |h, k| h[k] = [] }
analysis.statements.each_with_index do |node, idx|
sig = analysis.generate_signature(node)
map[sig] << {node: node, index: idx} if sig
end
map
end
endChoose this when:
- Your format has recursive nesting (classes containing methods, objects containing objects) — you'll want recursive body merging, which is easiest to control inline
- Your merge needs multi-phase output (e.g., magic comments first, then template-only nodes, then dest-order merge)
- You want simpler code with fewer classes to maintain
- The format is flat (dotenv, bash) — a ConflictResolver adds unnecessary indirection
Pattern 2: ConflictResolver Delegation
The SmartMerger delegates merge logic to a separate ConflictResolver class. The resolver receives pre-built analyses and populates a MergeResult via an emitter.
Used by: psych-merge (YAML), json-merge (JSON), jsonc-merge (JSONC), toml-merge (TOML)
class SmartMerger < Ast::Merge::SmartMergerBase
protected
def resolver_class
ConflictResolver # Delegate merge to ConflictResolver
end
def build_conflict_resolver
ConflictResolver.new(
@template_analysis,
@dest_analysis,
preference: @preference,
add_template_only_nodes: @add_template_only_nodes,
freeze_token: @freeze_token,
match_refiner: @match_refiner,
)
end
end
class ConflictResolver < Ast::Merge::ConflictResolverBase
# strategy: :batch — resolve all nodes at once
def resolve_batch(result)
merge_node_lists_to_emitter(template_nodes, dest_nodes, ...)
end
endChoose this when:
- Your format's merge logic is complex enough to warrant a separate class for testability
- You want the resolver to be independently testable with mock analyses
- The format uses an emitter pattern (building output line-by-line with structural awareness)
- Multiple merge strategies might share the same SmartMerger but differ in resolution
Signature Matching: The Core Algorithm
Both patterns use the same core algorithm. Every *-merge gem follows these steps:
-
Parse both template and destination files into ASTs via
FileAnalysis -
Generate signatures for each top-level node (e.g.,
[:def, :greet],[:pair, "name"],[:command, "echo", ['"Foo"']]) -
Build a signature map:
signature → [{node:, index:}, ...]— stores all occurrences, not just the first - First pass (destination order): Walk destination nodes, find matching template nodes by signature
-
Second pass: Add any remaining unmatched template nodes (if
add_template_only_nodes: true)
Cursor-Based Positional Matching
Design principle: Two distinct lines in the input must remain two distinct lines in the output. Signatures identify what a node is, not where it is. The AST provides the structural context that prevents false merging.
When multiple nodes share the same signature, they are matched 1:1 in order using a per-signature cursor:
Template: Destination:
echo "Foo" ←→ echo "Foo" (1st ←→ 1st)
echo "Foo" ←→ echo "Foo" (2nd ←→ 2nd)
echo "Bar" echo "Bar"
echo "Baz" (dest-only, preserved)
This uses two data structures:
-
consumed_template_indices(Set): tracks which template nodes have been matched -
sig_cursor(Hash): tracks the next candidate index per signature
The old approach used a processed_signatures Set which would collapse all nodes sharing a signature into a single match — losing legitimate duplicates. All gems in the family now use cursor-based matching.
Recursive Body Merging
For formats with nested structure (Ruby, YAML, JSON, TOML), containers are merged recursively:
module Foo
class Bar
attr_accessor :fizz # ← scoped to Bar's body
end
class Buzz
attr_accessor :fizz # ← scoped to Buzz's body (NOT collapsed with Bar's)
end
endWhen class Bar matches between template and destination, their bodies are extracted and merged in a separate recursive call. The recursion itself provides tree-path scoping — signatures are only compared within the same container.
Implementing recursive merge (inline pattern):
def perform_merge
# ... signature matching ...
if should_merge_recursively?(template_node, dest_node)
body_merger = self.class.new(
extract_body(template_node),
extract_body(dest_node),
preference: @preference,
# ... pass through all options ...
)
merged_body = body_merger.merge
# Reassemble: opening line + merged body + closing line
end
endImplementing recursive merge (ConflictResolver pattern):
def merge_node_lists_to_emitter(template_nodes, dest_nodes, template_analysis, dest_analysis)
# ... signature matching ...
if can_merge_recursively?(template_node, dest_node)
# Extract children, build new signature maps, recurse
merge_node_lists_to_emitter(
template_children, dest_children,
template_analysis, dest_analysis,
)
end
endDecision Guide: Which Pattern for Your Format?
| Format Characteristic | Inline SmartMerger | ConflictResolver |
|---|---|---|
| Flat structure (no nesting) | ✅ | ✅ |
| Deep recursive nesting | ✅ | ✅ |
| Multi-phase output ordering | ✅ | ➖ |
| Magic comments / prefix lines | ✅ | ➖ |
| Independent resolver testability needed | ➖ | ✅ |
| Emitter-based output construction | ➖ | ✅ |
| Simple, fewer classes | ✅ | ➖ |
Legend: ✅ = natural fit, ➖ = possible but not the natural fit
Rules of thumb:
- If your format has prefix metadata that must appear first regardless of merge preference (magic comments, shebangs, frontmatter), use the inline pattern — it gives you direct control over output ordering
- If your format's merge is purely structural (matching keys/nodes and choosing which version to keep), the ConflictResolver pattern keeps the SmartMerger clean
- When in doubt, start with the inline pattern — it's simpler and you can always extract a ConflictResolver later
Forward Compatibility: **options
All constructors and public API methods must include **options as the final parameter:
def initialize(source, freeze_token: nil, signature_generator: nil, **options)
# **options captures future parameters for forward compatibility
endWhen SmartMergerBase adds new standard options (like node_typing, regions), all FileAnalysis classes automatically accept them without code changes. Without **options, every gem would need updating whenever a new option is added to the base class.
Base Classes Reference
| Base Class | Purpose | Key Methods to Implement |
|---|---|---|
SmartMergerBase |
Main merge orchestration |
analysis_class, perform_merge
|
ConflictResolverBase |
Resolve node conflicts |
resolve_batch or resolve_node_pair
|
MergeResultBase |
Track merge results |
to_s, format-specific output |
MatchRefinerBase |
Fuzzy node matching | similarity |
ContentMatchRefiner |
Text content fuzzy matching | Ready to use |
FileAnalyzable |
File parsing/analysis | compute_node_signature |
ContentMatchRefiner
Ast::Merge::ContentMatchRefiner is a built-in match refiner for fuzzy text content matching using Levenshtein distance. Unlike signature-based matching which requires exact content hashes, this refiner allows matching nodes with similar (but not identical) content.
# Basic usage - match nodes with 70% similarity
refiner = Ast::Merge::ContentMatchRefiner.new(threshold: 0.7)
# Only match specific node types
refiner = Ast::Merge::ContentMatchRefiner.new(
threshold: 0.6,
node_types: [:paragraph, :heading],
)
# Custom weights for scoring
refiner = Ast::Merge::ContentMatchRefiner.new(
threshold: 0.7,
weights: {
content: 0.8, # Levenshtein similarity (default: 0.7)
length: 0.1, # Length similarity (default: 0.15)
position: 0.1, # Position in document (default: 0.15)
},
)
# Custom content extraction
refiner = Ast::Merge::ContentMatchRefiner.new(
threshold: 0.7,
content_extractor: ->(node) { node.text_content.downcase.strip },
)
# Use with a merger
merger = MyFormat::SmartMerger.new(
template,
destination,
preference: :template,
match_refiner: refiner,
)This is particularly useful for:
- Paragraphs with minor edits (typos, rewording)
- Headings with slight changes
- Comments with updated text
- Any text-based node that may have been slightly modified
Namespace Reference
The Ast::Merge module is organized into several namespaces, each with detailed documentation:
| Namespace | Purpose | Documentation |
|---|---|---|
Ast::Merge::Detector |
Region detection and merging | lib/ast/merge/detector/README.md |
Ast::Merge::Recipe |
YAML-based merge recipes | lib/ast/merge/recipe/README.md |
Ast::Merge::Comment |
Comment parsing and representation | lib/ast/merge/comment/README.md |
Ast::Merge::Text |
Plain text AST parsing | lib/ast/merge/text/README.md |
Ast::Merge::RSpec |
Shared RSpec examples | lib/ast/merge/rspec/README.md |
Key Classes by Namespace:
-
Detector:
Region,Base,Mergeable,FencedCodeBlock,YamlFrontmatter,TomlFrontmatter -
Recipe:
Config,Runner,ScriptLoader -
Comment:
Line,Block,Empty,Parser,Style -
Text:
SmartMerger,FileAnalysis,LineNode,WordNode,Section -
RSpec: Shared examples and dependency tags for testing
*-mergeimplementations
💡 Info you can shake a stick at
| Tokens to Remember |
|
|---|---|
| Works with JRuby |
|
| Works with Truffle Ruby |
|
| Works with MRI Ruby 3 |
|
| Support & Community |
|
| Source |
|
| Documentation |
|
| Compliance |
|
| Style |
|
| Maintainer 🎖️ |
|
... 💖 |
|
Compatibility
Compatible with MRI Ruby 3.2.0+, and concordant releases of JRuby, and TruffleRuby.
| 🚚 Amazing test matrix was brought to you by | 🔎 appraisal2 🔎 and the color 💚 green 💚 |
|---|---|
| 👟 Check it out! | ✨ github.com/appraisal-rb/appraisal2 ✨ |
Federated DVCS
| Federated DVCS Repository | Status | Issues | PRs | Wiki | CI | Discussions |
|---|---|---|---|---|---|---|
| 🧪 kettle-rb/ast-merge on GitLab | The Truth | 💚 | 💚 | 💚 | 🐭 Tiny Matrix | ➖ |
| 🧊 kettle-rb/ast-merge on CodeBerg | An Ethical Mirror (Donate) | 💚 | 💚 | ➖ | ⭕️ No Matrix | ➖ |
| 🐙 kettle-rb/ast-merge on GitHub | Another Mirror | 💚 | 💚 | 💚 | 💯 Full Matrix | 💚 |
| 🎮️ Discord Server | Let's | talk | about | this | library! |
Available as part of the Tidelift Subscription.
The maintainers of this and thousands of other packages are working with Tidelift to deliver commercial support and maintenance for the open source packages you use to build your applications. Save time, reduce risk, and improve code health, while paying the maintainers of the exact packages you use.
- 💡Subscribe for support guarantees covering all your FLOSS dependencies
- 💡Tidelift is part of Sonar
- 💡Tidelift pays maintainers to maintain the software you depend on!
📊@Pointy Haired Boss: An enterprise support subscription is "never gonna let you down", and supports open source maintainers Alternatively:
✨ Installation
Install the gem and add to the application's Gemfile by executing:
bundle add ast-mergeIf bundler is not being used to manage dependencies, install the gem by executing:
gem install ast-merge🔒 Secure Installation
This gem is cryptographically signed, and has verifiable SHA-256 and SHA-512 checksums by stone_checksums. Be sure the gem you install hasn’t been tampered with by following the instructions below.
Add my public key (if you haven’t already, expires 2045-04-29) as a trusted certificate:
gem cert --add <(curl -Ls https://raw.github.com/galtzo-floss/certs/main/pboling.pem)You only need to do that once. Then proceed to install with:
gem install ast-merge -P HighSecurityThe HighSecurity trust profile will verify signed gems, and not allow the installation of unsigned dependencies.
If you want to up your security game full-time:
bundle config set --global trust-policy MediumSecurityMediumSecurity instead of HighSecurity is necessary if not all the gems you use are signed.
NOTE: Be prepared to track down certs for signed gems and add them the same way you added mine.
⚙️ Configuration
ast-merge provides base classes and shared interfaces for building format-specific merge tools.
Each implementation (like prism-merge, psych-merge, etc.) has its own SmartMerger with format-specific configuration.
Common Configuration Options
All SmartMerger implementations share these configuration options:
merger = SomeFormat::Merge::SmartMerger.new(
template,
destination,
# When conflicts occur, prefer template or destination values
preference: :template, # or :destination (default), or a Hash for per-node-type
# Add nodes that only exist in template (Boolean or callable filter)
add_template_only_nodes: true, # default: false, or ->(node, entry) { ... }
# Custom node type handling
node_typing: {}, # optional, for per-node-type preference
)Signature Match Preference
Control which source wins when both files have the same structural element:
-
:template- Template values replace destination values -
:destination(default) - Destination values are preserved - Hash - Per-node-type preference (see Advanced Configuration)
Template-Only Nodes
Control whether to add nodes that only exist in the template:
-
true- Add all template-only nodes -
false(default) - Skip template-only nodes - Callable - Filter which template-only nodes to add
Callable Filter
When you need fine-grained control over which template-only nodes are added, pass a callable (Proc/Lambda) that receives (node, entry) and returns truthy to add or falsey to skip:
# Only add nodes with gem_family signatures
merger = SomeFormat::Merge::SmartMerger.new(
template,
destination,
add_template_only_nodes: ->(node, entry) {
sig = entry[:signature]
sig.is_a?(Array) && sig.first == :gem_family
},
)
# Only add link definitions that match a pattern
merger = Markly::Merge::SmartMerger.new(
template,
destination,
add_template_only_nodes: ->(node, entry) {
entry[:template_node].type == :link_definition &&
entry[:signature]&.last&.include?("gem")
},
)The entry hash contains:
-
:template_node- The node being considered for addition -
:signature- The node's signature (Array or other value) -
:template_index- Index in the template statements -
:dest_index- Alwaysnilfor template-only nodes
🔧 Basic Usage
Using Shared Examples in Tests
# spec/spec_helper.rb
require "ast/merge/rspec/shared_examples"
# spec/my_format/merge/freeze_node_spec.rb
RSpec.describe(MyFormat::Merge::FreezeNode) do
it_behaves_like "Ast::Merge::FreezeNode" do
let(:freeze_node_class) { described_class }
let(:default_pattern_type) { :hash_comment }
let(:build_freeze_node) do
lambda { |start_line:, end_line:, **opts|
# Build a freeze node for your format
}
end
end
endAvailable Shared Examples
-
"Ast::Merge::FreezeNode"- Tests for FreezeNode implementations -
"Ast::Merge::MergeResult"- Tests for MergeResult implementations -
"Ast::Merge::DebugLogger"- Tests for DebugLogger implementations -
"Ast::Merge::FileAnalysisBase"- Tests for FileAnalysis implementations -
"Ast::Merge::MergerConfig"- Tests for SmartMerger implementations
🎛️ Advanced Configuration
Freeze Blocks
Freeze blocks are special comment-delimited regions in your files that tell the merge tool to preserve content exactly as-is, preventing any changes from the template. This is useful for hand-edited customizations you never want overwritten.
A freeze block consists of:
- A start marker comment (e.g.,
# mytoken:freeze) - The protected content
- An end marker comment (e.g.,
# mytoken:unfreeze)
# In a Ruby file with prism-merge:
class MyApp
# prism-merge:freeze
# Custom configuration that should never be overwritten
CUSTOM_SETTING = "my-value"
# prism-merge:unfreeze
VERSION = "1.0.0" # This can be updated by template
endThe FreezeNode class represents these protected regions internally.
Each format-specific merge gem (like prism-merge, psych-merge, etc.) configures its own
freeze token (the token in token:freeze), which defaults to the gem name (e.g., prism-merge).
Supported Comment Patterns
Different file formats use different comment syntaxes. The merge tools detect freeze markers using the appropriate pattern for each format:
| Pattern Type | Start Marker | End Marker | Languages |
|---|---|---|---|
:hash_comment |
# token:freeze |
# token:unfreeze |
Ruby, Python, YAML, Bash, Shell |
:html_comment |
<!-- token:freeze --> |
<!-- token:unfreeze --> |
HTML, XML, Markdown |
:c_style_line |
// token:freeze |
// token:unfreeze |
C (C99+), C++, JavaScript, TypeScript, Java, C#, Go, Rust, Swift, Kotlin, PHP, JSONC |
:c_style_block |
/* token:freeze */ |
/* token:unfreeze */ |
C, C++, JavaScript, TypeScript, Java, C#, Go, Rust, Swift, Kotlin, PHP, CSS |
| 📍 NOTE |
|---|
CSS only supports block comments (/* */), not line comments. |
| JSON does not support comments; use JSONC for JSON with comments. |
Per-Node-Type Preference with node_typing
The node_typing option allows you to customize merge behavior on a per-node-type basis.
When combined with a Hash-based preference, you can specify different merge
preferences for different types of nodes (e.g., prefer template for linter configs but destination for everything else).
How It Works
-
Define a
node_typing: A Hash mapping node type symbols to callables that receive a node and return either:- The original node (no special handling)
- A wrapped node with a
merge_typeattribute (viaAst::Merge::NodeTyping::Wrapper)
-
Use a Hash-based preference: Instead of a simple
:destinationor:templateSymbol, pass a Hash with:-
:defaultkey for the fallback preference - Custom keys matching the
merge_typevalues from yournode_typing
-
# Example: Prefer template for lint gem configs, destination for everything else
node_typing = {
call_node: ->(node) {
if node.name == :gem && node.arguments&.arguments&.first&.unescaped&.match?(/rubocop|standard|reek/)
Ast::Merge::NodeTyping::Wrapper.new(node, :lint_gem)
else
node
end
},
}
merger = Prism::Merge::SmartMerger.new(
template_content,
dest_content,
node_typing: node_typing,
preference: {
default: :destination,
lint_gem: :template,
},
)NodeTyping::Wrapper
The Ast::Merge::NodeTyping::Wrapper class wraps an AST node and adds a merge_type attribute.
It delegates all method calls to the wrapped node, so it can be used transparently in place of the original node.
# Wrap a node with a custom merge_type
wrapped = Ast::Merge::NodeTyping::Wrapper.new(original_node, :special_config)
wrapped.merge_type # => :special_config
wrapped.class # => Ast::Merge::NodeTyping::Wrapper
wrapped.location # => delegates to original_node.locationNodeTyping Utility Methods
# Process a node through the node_typing configuration
processed = Ast::Merge::NodeTyping.process(node, node_typing_config)
# Check if a node has been wrapped with a merge_type
Ast::Merge::NodeTyping.typed_node?(node) # => true/false
# Get the merge_type from a wrapped node (or nil)
Ast::Merge::NodeTyping.merge_type_for(node) # => Symbol or nil
# Unwrap a node type wrapper to get the original
Ast::Merge::NodeTyping.unwrap(wrapped_node) # => original_nodeHash-Based Preference (without node_typing)
Even without node_typing, you can use a Hash-based preference to set a default
and document your intention for future per-type customization:
# Simple Hash preference (functionally equivalent to preference: :destination)
merger = MyMerger.new(
template_content,
dest_content,
preference: {default: :destination},
)MergerConfig Factory Methods
The MergerConfig class provides factory methods that support all options:
# Create config preferring destination
config = Ast::Merge::MergerConfig.destination_wins(
freeze_token: "my-freeze",
signature_generator: my_generator,
node_typing: my_typing,
)
# Create config preferring template
config = Ast::Merge::MergerConfig.template_wins(
freeze_token: "my-freeze",
signature_generator: my_generator,
node_typing: my_typing,
)📋 YAML Merge Recipes
ast-merge includes a YAML-based recipe system for defining portable, distributable merge configurations. Recipes allow any project to ship merge knowledge as data — a YAML file (and optionally small companion Ruby scripts) — that consumers can load and execute without writing merge instrumentation.
Preset vs Config (Recipe)
The recipe system provides two levels of configuration:
-
Ast::Merge::Recipe::Preset— Merge configuration only (preference, signature generator, node typing, freeze token). Use when you have your own template/destination handling and just need the merge settings. -
Ast::Merge::Recipe::Config— Full recipe extending Preset with template file, target glob patterns, injection point configuration, and when_missing behavior. Use for standalone merge operations that know their own inputs and outputs.
Minimal Recipe (Preset)
A simple preset recipe is just a YAML file — no companion folder or Ruby scripts required:
name: my_config
description: Merge YAML config files with destination preference
parser: psych
merge:
preference: destination
add_missing: true
freeze_token: my-projectLoad and use it:
preset = Ast::Merge::Recipe::Preset.load("path/to/my_config.yml")
merger = Psych::Merge::SmartMerger.new(template, destination, **preset.to_h)
result = merger.mergeFull Recipe (Config)
A full recipe adds template, targets, and injection point configuration:
name: gem_family_section
description: Update gem family section in README files
# Template file (relative to recipe file)
template: GEM_FAMILY_SECTION.md
# Target files (supports globs)
targets:
- README.md
- vendor/*/README.md
# Where to inject/replace content
injection:
anchor:
type: heading
text: "/Gem Family/"
position: replace
boundary:
type: heading
same_or_shallower: true
# Merge settings
merge:
preference: template
add_missing: true
# When anchor is not found in a target
when_missing: skipExecute it:
recipe = Ast::Merge::Recipe::Config.load("path/to/gem_family_section.yml")
runner = Ast::Merge::Recipe::Runner.new(recipe, dry_run: true, parser: :markly)
results = runner.run
puts runner.summary
# => { total: 10, updated: 5, unchanged: 3, skipped: 2 }Or via CLI:
bin/ast-merge-recipe path/to/gem_family_section.yml --dry-run --parser=marklyRecipe YAML Schema
Preset Fields (used by both Preset and Config)
| Field | Required | Description |
|---|---|---|
name |
Yes | Recipe identifier |
description |
No | Human-readable description |
parser |
No | Parser to use (prism, markly, psych, etc.). Default: prism
|
merge.preference |
No |
:template or :destination. Default: :template
|
merge.add_missing |
No |
true, false, or path to a Ruby script returning a callable filter. Default: true
|
merge.signature_generator |
No | Path to companion Ruby script (relative to recipe folder) |
merge.node_typing |
No | Hash mapping node class names to companion Ruby script paths |
merge.match_refiner |
No | Path to companion Ruby script for match refinement |
merge.normalize_whitespace |
No |
true to collapse excessive blank lines |
merge.rehydrate_link_references |
No |
true to convert inline links to reference style |
freeze_token |
No | Token for freeze block preservation (e.g., "my-project") |
Config-Only Fields (full recipes)
| Field | Required | Description |
|---|---|---|
template |
Yes | Path to template file (relative to recipe file or absolute) |
targets |
No | Array of glob patterns for target files. Default: ["*.md"]
|
injection.anchor.type |
No | Node type to match (e.g., heading, paragraph) |
injection.anchor.text |
No | Text pattern — string for exact match, /regex/ for pattern |
injection.anchor.level |
No | Heading level (for heading anchors) |
injection.position |
No |
replace, before, after, first_child, last_child. Default: replace
|
injection.boundary.type |
No | Node type that marks the end of the section |
injection.boundary.same_or_shallower |
No |
true to end at next same-level-or-higher heading |
when_missing |
No |
skip, add, or error. Default: skip
|
Companion Scripts (Optional)
When a recipe needs custom signature matching or node categorization beyond the defaults, it can reference Ruby scripts in an optional companion folder. The folder name must match the recipe name (without .yml):
my-project/
recipes/
my_format.yml # The recipe
my_format/ # Optional companion folder
signature_generator.rb # Returns a lambda for node matching
typing/
call_node.rb # Returns a lambda for node categorization
Each script must return a callable (the last expression is the return value):
# signature_generator.rb
lambda do |node|
return node unless node.is_a?(Prism::CallNode)
case node.name
when :gem
first_arg = node.arguments&.arguments&.first
[:gem, first_arg.unescaped] if first_arg.is_a?(Prism::StringNode)
when :source
[:source]
else
node
end
endScripts are loaded on demand via Ast::Merge::Recipe::ScriptLoader and cached for the lifetime of the preset.
Text Matching in Anchor Patterns
When matching nodes by text content (e.g., heading anchors), the .text method returns plain text without formatting:
| Markdown Source |
.text Returns |
|---|---|
### The `*-merge` Gem Family |
The *-merge Gem Family |
**Bold text** |
Bold text |
[link text](url) |
link text |
Write patterns that match the plain text:
# ❌ WRONG - backticks won't appear in .text
anchor:
text: "/`\\*-merge` Gem Family/"
# ✅ CORRECT - match plain text
anchor:
text: "/\\*-merge Gem Family/"Distributing Recipes
Recipes are designed to be portable. A project can ship recipes in its gem or repository:
-
Minimal recipes (YAML only) need no companion folder — consumers only need
ast-merge - Advanced recipes (YAML + scripts) ship the companion folder alongside the YAML
- Consumers load recipes with
Ast::Merge::Recipe::Preset.load(path)orConfig.load(path)— no dependency onkettle-jemor any specific tool - The kettle-jem gem provides a collection of built-in recipes for common file types (Gemfile, gemspec, Rakefile, Appraisals, Markdown)
See lib/ast/merge/recipe/README.md for additional details and examples.
🦷 FLOSS Funding
While kettle-rb tools are free software and will always be, the project would benefit immensely from some funding. Raising a monthly budget of... "dollars" would make the project more sustainable.
We welcome both individual and corporate sponsors! We also offer a wide array of funding channels to account for your preferences (although currently Open Collective is our preferred funding platform).
If you're working in a company that's making significant use of kettle-rb tools we'd appreciate it if you suggest to your company to become a kettle-rb sponsor.
You can support the development of kettle-rb tools via GitHub Sponsors, Liberapay, PayPal, Open Collective and Tidelift.
| 📍 NOTE |
|---|
| If doing a sponsorship in the form of donation is problematic for your company from an accounting standpoint, we'd recommend the use of Tidelift, where you can get a support-like subscription instead. |
Open Collective for Individuals
Support us with a monthly donation and help us continue our activities. [Become a backer]
NOTE: kettle-readme-backers updates this list every day, automatically.
No backers yet. Be the first!
Open Collective for Organizations
Become a sponsor and get your logo on our README on GitHub with a link to your site. [Become a sponsor]
NOTE: kettle-readme-backers updates this list every day, automatically.
No sponsors yet. Be the first!
Another way to support open-source
I’m driven by a passion to foster a thriving open-source community – a space where people can tackle complex problems, no matter how small. Revitalizing libraries that have fallen into disrepair, and building new libraries focused on solving real-world challenges, are my passions. I was recently affected by layoffs, and the tech jobs market is unwelcoming. I’m reaching out here because your support would significantly aid my efforts to provide for my family, and my farm (11 🐔 chickens, 2 🐶 dogs, 3 🐰 rabbits, 8 🐈 cats).
If you work at a company that uses my work, please encourage them to support me as a corporate sponsor. My work on gems you use might show up in bundle fund.
I’m developing a new library, floss_funding, designed to empower open-source developers like myself to get paid for the work we do, in a sustainable way. Please give it a look.
Floss-Funding.dev: 👉️ No network calls. 👉️ No tracking. 👉️ No oversight. 👉️ Minimal crypto hashing. 💡 Easily disabled nags
🔐 Security
See SECURITY.md.
🤝 Contributing
If you need some ideas of where to help, you could work on adding more code coverage, or if it is already 💯 (see below) check reek, issues, or PRs, or use the gem and think about how it could be better.
We so if you make changes, remember to update it.
See CONTRIBUTING.md for more detailed instructions.
🚀 Release Instructions
See CONTRIBUTING.md.
Code Coverage
🪇 Code of Conduct
Everyone interacting with this project's codebases, issue trackers,
chat rooms and mailing lists agrees to follow the .
🌈 Contributors
Made with contributors-img.
Also see GitLab Contributors: https://gitlab.com/kettle-rb/ast-merge/-/graphs/main
📌 Versioning
This Library adheres to .
Violations of this scheme should be reported as bugs.
Specifically, if a minor or patch version is released that breaks backward compatibility,
a new version should be immediately released that restores compatibility.
Breaking changes to the public API will only be introduced with new major versions.
dropping support for a platform is both obviously and objectively a breaking change
—Jordan Harband (@ljharb, maintainer of SemVer) in SemVer issue 716
I understand that policy doesn't work universally ("exceptions to every rule!"), but it is the policy here. As such, in many cases it is good to specify a dependency on this library using the Pessimistic Version Constraint with two digits of precision.
For example:
spec.add_dependency("ast-merge", "~> 4.0", ">= 4.0.0") # ruby >= 3.2.0SemVer should, IMO, but doesn't explicitly, say that dropping support for specific Platforms is a breaking change to an API, and for that reason the bike shedding is endless.
To get a better understanding of how SemVer is intended to work over a project's lifetime, read this article from the creator of SemVer:
See CHANGELOG.md for a list of releases.
📄 License
The gem is available as open source under the terms of
the MIT License .
See LICENSE.txt for the official Copyright Notice.
© Copyright
-
Copyright (c) 2025-2026 Peter H. Boling, of
Galtzo.com
, and ast-merge contributors.
🤑 A request for help
Maintainers have teeth and need to pay their dentists. After getting laid off in an RIF in March, and encountering difficulty finding a new one, I began spending most of my time building open source tools. I'm hoping to be able to pay for my kids' health insurance this month, so if you value the work I am doing, I need your support. Please consider sponsoring me or the project.
To join the community or get help 👇️ Join the Discord.
To say "thanks!" ☝️ Join the Discord or 👇️ send money.
Please give the project a star ⭐ ♥.
Thanks for RTFM. ☺️