Ammitto

Purpose

Ammitto is a Ruby gem for retrieving sanctioned entities (persons, organizations, vessels, and aircraft) from various international sources. It provides a unified JSON-LD based data model for querying sanctions data from multiple jurisdictions.

Features

Unified Data Model - Consistent JSON-LD schema across all sources
Multiple Entity Types - Supports persons, organizations, vessels, and aircraft
15 Data Sources - EU, UN, US (OFAC), World Bank, UK, Australia, Canada, Switzerland, China, Russia, Japan, New Zealand, Turkey, EU Vessels, UN Vessels
Knowledge Graph Export - Individual JSON-LD node files with @id references
Lightweight Search Index - Optimized search-index.json for client-side search
Ontology Browser Data - Classes, properties, hierarchy for schema exploration
Rich Metadata - Legal bases, effects, reasons, status history, and official announcements
Local Caching - Downloads and caches data locally for fast queries
Search API - Simple and powerful search interface

Installation

Add this line to your application’s Gemfile:

gem 'ammitto'

And then execute:

bundle install

Or install it yourself as:

gem install ammitto

Usage

Configuration

require 'ammitto'

Ammitto.configure do |config|
  config.api_base_url = "https://www.ammitto.com/api/v1"  # default
  config.cache_dir = File.expand_path("~/.ammitto")       # default
  config.cache_ttl = 3600                                  # 1 hour, default
  config.verbose = true                                    # default: false
end

Refreshing Cache

# Refresh specific sources
Ammitto.refresh_cache(sources: [:eu, :un, :us])

# Refresh all sources
Ammitto.refresh_cache(all: true)

# Force refresh even if cache is fresh
Ammitto.refresh_cache(all: true, force: true)

Searching

# Search across all sources
results = Ammitto.search("Kim Jong Un")

# Search specific sources
results = Ammitto.search("Kim Jong Un", sources: [:un, :us])

# Search with pagination
results = Ammitto.search("Putin", limit: 50, offset: 0)

# Iterate results
results.each do |entry|
  puts "#{entry.display_name}"
  puts "  Authority: #{entry.authority.name}"
  puts "  Status: #{entry.status}"
  puts "  Listed: #{entry.period.listed_date}"
end

Checking Cache Status

status = Ammitto.cache_status
status.each do |source, info|
  if info[:cached]
    puts "#{source}: cached, updated #{info[:updated_at]}"
  else
    puts "#{source}: not cached"
  end
end

Getting JSON-LD Output

results = Ammitto.search("test")
json_ld = results.to_json_ld
puts JSON.pretty_generate(json_ld)

Available Sources

Ammitto.sources
# => [:au, :ca, :ch, :cn, :eu, :gb, :ru, :un, :us, :wb]

Data Model

Entity Types

PersonEntity - Individual persons with birth info, nationalities, identifications
OrganizationEntity - Companies and organizations with registration info
VesselEntity - Ships with IMO numbers, flag states, and ownership
AircraftEntity - Aircraft with serial numbers, manufacturers, and registrations

Sanction Entry

A SanctionEntry represents a single sanction against an entity, including:

Authority (EU, UN, US, etc.)
Regime (DPRK, Russia/Ukraine, etc.)
Legal bases (resolutions, regulations, executive orders)
Effects (asset freeze, travel ban, etc.)
Reasons (terrorism, proliferation, etc.)
Status (active, suspended, delisted, etc.)
Status history (tracking changes over time)
Official announcement (link to source document)
Raw source data (original XML/JSON preserved)

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    Ammitto Ruby Gem                              │
├─────────────────────────────────────────────────────────────────┤
│  Ammitto.search() ──► QueryBuilder ──► Sources ──► ResultSet    │
│                                                                  │
│  Ammitto.refresh_cache() ──► CacheManager ──► ApiClient         │
│                                   │                              │
│                                   ▼                              │
│                              Cache (~/.ammitto/)                 │
└─────────────────────────────────────────────────────────────────┘
                              │
                              │ Downloads from
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                    www.ammitto.com/api/v1/                       │
│                                                                  │
│  /sources/eu.jsonld    /sources/un.jsonld    /sources/us.jsonld │
│  /sources/gb.jsonld    /sources/cn.jsonld    /all.jsonld        │
└─────────────────────────────────────────────────────────────────┘

Data Sources

Code	Authority	Source
EU	European Union	webgate.ec.europa.eu
UN	United Nations	scsanctions.un.org
US	United States (OFAC)	treasury.gov
WB	World Bank	worldbank.org
UK	United Kingdom (OFSI)	gov.uk
AU	Australia (DFAT)	dfat.gov.au
CA	Canada (SEFO)	international.gc.ca
CH	Switzerland (SECO)	seco.admin.ch
CN	China (MOFCOM/MFA)	mofcom.gov.cn, mfa.gov.cn
RU	Russia (MID)	mid.ru
JP	Japan (MOFA)	mofa.go.jp
NZ	New Zealand (MFAT)	mfat.govt.nz
TR	Turkey (Ministry of Interior)	icisleri.gov.tr
EU_VESSELS	EU Vessels List	webgate.ec.europa.eu
UN_VESSELS	UN Vessels List	scsanctions.un.org

For Ammitto Operators

This section documents the complete data pipeline for operators maintaining the Ammitto data infrastructure.

Architecture Overview

The Ammitto data pipeline consists of four phases:

┌─────────────────────────────────────────────────────────────────────────────────────┐
│                          AMMITTO DATA PIPELINE ARCHITECTURE                           │
├─────────────────────────────────────────────────────────────────────────────────────┤
│                                                                                      │
│  PHASE 1: SOURCE DATA REPOS (GitHub: ammitto/data-{source})                         │
│  ══════════════════════════════════════════════════════════════════════             │
│                                                                                      │
│   GHA cron (daily 6:00-8:00 UTC)                                                     │
│         │                                                                            │
│         ▼                                                                            │
│   ammitto fetch {source} --format yaml --output-dir processed                       │
│         │                                                                            │
│         │  1. Download raw XML/JSON/HTML from source API                             │
│         │  2. Parse using Lutaml::Model (Ammitto::Sources::{Source})                │
│         │  3. Save as individual YAML files per entity                               │
│         │  4. Create _index.yaml with metadata                                       │
│         ▼                                                                            │
│   ┌───────────────────────────────────────────────────────────────────────────┐     │
│   │  data-uk/  data-eu/  data-un/  data-us/  data-wb/                          │     │
│   │  data-au/  data-ca/  data-ch/  data-cn/  data-ru/                          │     │
│   │  └── processed/*.yaml  (one file per sanctioned entity)                    │     │
│   └───────────────────────────────────────────────────────────────────────────┘     │
│                                                                                      │
│  ══════════════════════════════════════════════════════════════════════             │
│                                                                                      │
│  PHASE 2: CENTRAL DATA REPO (GitHub: ammitto/data)                                   │
│  ══════════════════════════════════════════════════════════════════════             │
│                                                                                      │
│   GHA schedule (daily 9:00 UTC)                                                      │
│         │                                                                            │
│         ▼                                                                            │
│   1. Checkout all source repos                                                       │
│         │                                                                            │
│         ▼                                                                            │
│   2. ammitto harmonize --all --sources-dir ./sources --output-dir ./api/v1          │
│         │                                                                            │
│         │  For each source:                                                          │
│         │    - Read YAML files from sources/data-{source}/processed/                │
│         │    - Parse using Lutaml::Model source models                               │
│         │    - Transform using Transformers (UkTransformer, etc.)                    │
│         │    - Convert to harmonized ontology models                                 │
│         │    - Export as JSON-LD with @context and @graph                            │
│         │                                                                            │
│         ▼                                                                            │
│   3. Create combined all.jsonld + stats.json                                         │
│         │                                                                            │
│         ▼                                                                            │
│   ┌───────────────────────────────────────────────────────────────────────────┐     │
│   │  api/v1/                                                                   │     │
│   │  ├── sources/uk.jsonld  sources/eu.jsonld  ...                            │     │
│   │  ├── all.jsonld            (combined data)                                │     │
│   │  └── stats.json            (entity counts, last updated)                  │     │
│   └───────────────────────────────────────────────────────────────────────────┘     │
│                                                                                      │
│  ══════════════════════════════════════════════════════════════════════             │
│                                                                                      │
│  PHASE 3: WEBSITE (GitHub: ammitto/ammitto.github.io)                                │
│  ══════════════════════════════════════════════════════════════════════             │
│                                                                                      │
│   Triggered by repository_dispatch from data repo                                    │
│         │                                                                            │
│         ▼                                                                            │
│   https://www.ammitto.com/api/v1/*.jsonld                                            │
│                                                                                      │
│  ══════════════════════════════════════════════════════════════════════             │
│                                                                                      │
│  PHASE 4: GEM USERS                                                                  │
│  ══════════════════════════════════════════════════════════════════════             │
│                                                                                      │
│   Ammitto.refresh_cache → Downloads from www.ammitto.com/api/                       │
│   ~/.ammitto/cache/ → Local JSON-LD cache                                            │
│   Ammitto.search("query") → Queries against cached data                              │
│                                                                                      │
└─────────────────────────────────────────────────────────────────────────────────────┘

Repository Structure

Repository	Purpose
`ammitto/ammitto`	Core Ruby gem with CLI, source models, and transformers
`ammitto/data-uk`	UK (OFSI) sanctions data in YAML format
`ammitto/data-eu`	EU sanctions data in YAML format
`ammitto/data-eu-vessels`	EU vessels sanctions data in YAML format
`ammitto/data-un`	UN sanctions data in YAML format
`ammitto/data-un-vessels`	UN vessels sanctions data in YAML format
`ammitto/data-us`	US (OFAC) sanctions data in YAML format
`ammitto/data-wb`	World Bank debarments data in YAML format
`ammitto/data-au`	Australia (DFAT) sanctions data in YAML format
`ammitto/data-ca`	Canada (SEFO) sanctions data in YAML format
`ammitto/data-ch`	Switzerland (SECO) sanctions data in YAML format
`ammitto/data-cn`	China (MOFCOM/MFA) sanctions data in YAML format
`ammitto/data-ru`	Russia (MID) sanctions data in YAML format
`ammitto/data-jp`	Japan (MOFA) sanctions data in YAML format
`ammitto/data-nz`	New Zealand (MFAT) sanctions data in YAML format
`ammitto/data-tr`	Turkey sanctions data in YAML format
`ammitto/data`	Central repo with harmonized JSON-LD output
`ammitto/ammitto.github.io`	Website serving the API with search and ontology browser
`ammitto/schemas`	JSON-LD schema definitions

Source Information

Code	Authority	Source URL	Format
UK	United Kingdom (OFSI)	sanctionslist.fcdo.gov.uk	XML
EU	European Union	webgate.ec.europa.eu	XML
UN	United Nations	scsanctions.un.org	XML
US	United States (OFAC)	treasury.gov	XML
WB	World Bank	worldbank.org	JSON
AU	Australia (DFAT)	dfat.gov.au	CSV
CA	Canada (SEFO)	international.gc.ca	XML
CH	Switzerland (SECO)	seco.admin.ch	XML
CN	China (MOFCOM/MFA)	mofcom.gov.cn, mfa.gov.cn	HTML
RU	Russia (MID)	mid.ru	HTML

CLI Commands for Operators

Fetch Data from Source

# Fetch UK data as YAML
ammitto fetch uk --format yaml --output-dir ./processed --verbose

# Fetch multiple sources
ammitto fetch uk eu un --format yaml --output-dir ./processed

# Fetch all sources
ammitto fetch --all --format yaml --output-dir ./processed

# Dry run (show what would be fetched)
ammitto fetch uk --dry-run

Harmonize Data (YAML → JSON-LD)

# Auto-detect and harmonize all data-* repositories
ammitto harmonize --scan --sources-dir /path/to/repos --output-dir ./api/v1 --verbose

# Harmonize specific source
ammitto harmonize uk --input-dir ../data-uk/processed --output-dir ./api/v1

# Create combined all.jsonld and all.ttl
ammitto harmonize --scan --combine --output-dir ./api/v1

The harmonization process generates:

Output	Description
`api/v1/node/entity/{source}/{id}.jsonld`	Individual entity nodes
`api/v1/node/entry/{source}/{id}.jsonld`	Individual entry nodes
`api/v1/node/authority/{code}.jsonld`	Authority nodes (deduplicated)
`api/v1/node/regime/{code}.jsonld`	Regime nodes (deduplicated)
`api/v1/search-index.json`	Lightweight search index (~8MB vs 69MB full)
`api/v1/facets/*.json`	Facet counts for filters
`api/v1/ontology/classes.jsonld`	Ontology class definitions
`api/v1/ontology/properties.jsonld`	Ontology property definitions
`api/v1/ontology/hierarchy.json`	Class hierarchy tree
`api/v1/all.jsonld`	Combined JSON-LD graph
`api/v1/all.ttl`	Combined Turtle/RDF graph
`api/v1/stats.json`	Statistics and metadata

Check Status

# Show cache and data status
ammitto status

# List available sources
ammitto sources

# Export to different formats
ammitto export jsonld --output-dir ./data
ammitto export ttl --output-dir ./data

GitHub Actions Workflows

Source Repo Workflows (`data-{source}/.github/workflows/fetch.yml`)

Each source repository has a workflow that:

Runs daily at a scheduled time (6:00-8:00 AM UTC, staggered by source)
Installs the ammitto gem from GitHub
Downloads raw data from the source API
Parses using Lutaml::Model source models
Saves individual YAML files to processed/
Commits and pushes changes

Table 1. Schedule Times (UTC)

Source	Cron Time	Description
UK	6:00	UK OFSI sanctions
EU	6:15	EU sanctions
UN	6:30	UN consolidated list
US	6:45	US OFAC SDN list
WB	7:00	World Bank debarments
AU	7:15	Australia DFAT
CA	7:30	Canada SEFO
CH	7:45	Switzerland SECO
CN	8:00	China (HTML parsing)
RU	8:15	Russia (HTML parsing)

Central Repo Workflow (`data/.github/workflows/harmonize.yml`)

Runs daily at 9:00 AM UTC (after all fetches complete)
Checks out all source repositories
Runs ammitto harmonize --all --combine
Saves JSON-LD output to api/v1/
Triggers website deployment

Adding a New Source

To add a new sanctions source:

Create source models in lib/ammitto/sources/{source}/ using Lutaml::Model

# lib/ammitto/sources/xx/designation.rb
module Ammitto
  module Sources
    module Xx
      class Designation < Lutaml::Model::Serializable
        attribute :id, :string
        attribute :name, :string
        # ... other attributes

        xml do
          root "Designation"
          map_element "ID", to: :id
          map_element "Name", to: :name
        end
      end
    end
  end
end

Create transformer in lib/ammitto/transformers/xx_transformer.rb

module Ammitto
  module Transformers
    class XxTransformer < BaseTransformer
      def initialize
        super(:xx)
      end

      def transform(source)
        # Transform source model to ontology models
      end
    end
  end
end

Register the source in lib/ammitto/config/defaults.rb:

ALL_SOURCES = %i[eu un us wb uk au ca ch cn ru xx].freeze

Create data repo on GitHub:

gh repo create ammitto/data-xx --public --description "Ammitto Data repository for XX Source"

Add fetch workflow to .github/workflows/fetch.yml in the new repo
Update harmonize workflow in ammitto/data to checkout the new source

Environment Variables

Variable	Description	Default
`AMMITTO_CACHE_DIR`	Local cache directory	`~/.ammitto`
`AMMITTO_API_BASE_URL`	API base URL	`https://www.ammitto.com/api/v1`
`AMMITTO_LOG_LEVEL`	Logging level (debug, info, warn, error)	`info`
`AMMITTO_SOURCES`	Comma-separated sources to process	all sources

Troubleshooting

Manual Fetch and Harmonize

# Fetch a specific source manually
cd /path/to/data-uk
ammitto fetch uk --format yaml --output-dir processed --verbose

# Check the output
ls -la processed/ | head -20
cat processed/_index.yaml

# Harmonize manually
cd /path/to/data
ammitto harmonize uk --sources-dir ./sources --output-dir ./api/v1 --verbose

# Check JSON-LD output
jq '.["@graph"] | length' ./api/v1/sources/uk.jsonld

Common Issues

Empty YAML files: Check that source models correctly parse the XML/JSON structure
Missing entities in JSON-LD: Verify transformers are registered in the Registry
Workflow failures: Check the GitHub Actions logs for error messages

Development

After checking out the repo, run bin/setup to install dependencies. Then, run bundle exec rake spec to run the tests.

# Run tests
bundle exec rake spec

# Run linter
bundle exec rubocop

# Interactive console
bin/console

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/ammitto/ammitto.

License

The gem is available as open source under the terms of the BSD-2-Clause License.

ammitto

Runtime