Kataba

Kataba (片刃) provides XML Schema Definition (XSD) mirroring and offline validation for Nokogiri

• Configuration to enable optional mirror list for XSD files
• Configuration to alter offline storage location
• Recursive XSD search to ensure total depth processing (i.e. XSD -> import -> etc.)

The cache layout — every transitively-imported XSD stored as <md5(URL)>.xsd in one flat directory, with each schemaLocation attribute in the cached files rewritten to those MD5'd filenames — is a deliberate workaround for two stacked constraints in the Nokogiri / libxml / JRuby stack. Capturing the rationale here so the gem's "weirdness" stays explained even if the upstream links rot.

1. libxml resolves xs:import schemaLocation against the working directory at schema-parse time. If the schemaLocation is an absolute URL, libxml goes off and fetches it over the network — slow, fragile, and the reason this gem exists in the first place. The classic workaround (documented by Ktulu in 2011) is to put every imported XSD in one directory, rewrite each import to a relative filename, and Dir.chdir to that directory at schema-load time. Kataba automates that recipe: fetch_schema wraps the Nokogiri::XML::Schema(...) call in a Dir.chdir(offline_storage) block.

2. Under JRuby + nokogiri-java, Dir.chdir doesn't propagate to the Java code that handles imports (reproduction repo; JRuby's lead maintainer has stated on JIRA that Dir.chdir is fragile under JRuby and full paths should be preferred). The portable fix is to remove path resolution from the picture entirely: store every cached XSD as a flat sibling, and rewrite every schemaLocation to a bare filename with no directory components. With nothing to resolve, neither MRI's libxml nor nokogiri-java has anything to get wrong.

MD5(URL) is the collision-free flat-name function. In real-world schemas, imports collide on basename — http://www.loc.gov/mods/xml.xsd and http://www.w3.org/2001/xml.xsd both end in xml.xsd. Hashing the full URL gives a stable, deterministic per-source filename without inventing a path-flattening scheme. (MD5 here is a content-addressable name, not a security primitive — collision resistance against random URLs is more than enough.)

download_xsd is recursive because xs:import chains aren't always one level deep: mods-3-5.xsd imports xlink.xsd and xml.xsd, and other schemas chain further. After each fetch, every schemaLocation in the file is captured and rewritten in place, then the loop runs again on any new URIs that surfaced.

A relative schemaLocation resolves against the schema that imports it. The TDWG Darwin Core schemas import their siblings by bare filename (tdwg_dwc_extensions.xsd), and Dublin Core's dcterms.xsd imports dc.xsd the same way. Kataba resolves each one against the importing schema's canonical URI, never a mirror or a redirect target, and caches it under the MD5 of the result. Two relative dc.xsd imports from different directories therefore get different cache files. An absolute location is kept exactly as written, because its MD5 is both the cache filename and the mirror_list key.

Each URI is fetched once per fetch_schema call. Schemas may import each other — tdwg_dwc_simple.xsd and tdwg_dwc_extensions.xsd do — so the recursion keeps a record of what it has fetched, and stops on a cycle.

gem install kataba

Kataba.configuration.offline_storage = "/tmp/kataba"

Kataba.configuration.mirror_list = File.join(Rails.root, 'config', 'mirror.yml')

Kataba.configuration.open_timeout = 3

Kataba.configuration.read_timeout = 10

open_timeout and read_timeout bound the wait on a schema host, in seconds, and default to 3 and 10. Kataba sets them because Net::HTTP defaults both to 60 and a fetch pays them per redirect hop and again on the alternate-scheme attempt, so an unbounded fetch of one schema can run for minutes — long enough to hold a web request thread if you validate inside one.

Raise them for a slow mirror. Lower them when the caller must not be held. A host that goes quiet raises Kataba::Fetcher::FetchTimeout, a subclass of Kataba::Fetcher::FetchError, so rescue Kataba::Fetcher::FetchError catches a timeout alongside a 404 and a redirect loop.

The mirror list is a YAML map from a canonical schema URI to where Kataba should fetch it instead. The key is the URI exactly as a document or an importing schema names it, after a relative import has been resolved.

"http://www.loc.gov/standards/mods/v3/mods-3-5.xsd": "https://mirror.example.org/mods-3-5.xsd"
"http://rs.tdwg.org/dwc/xsd/tdwg_dwcterms.xsd": "schemas/tdwg_dwcterms.xsd"

A value that is not an http(s) URL is a file. A relative path resolves against the directory that holds the mirror list, so a project can ship a corrected copy of a broken published schema beside its configuration. The file's own relative imports still resolve against the canonical URI.

Kataba never refreshes a cached schema, and that includes one that a new mirror entry would now replace. Clear the offline storage after you change the mirror list.

The fetch_schema method returns a Nokogiri::XML::Schema object

xsd = Kataba.fetch_schema("http://www.loc.gov/standards/mods/v3/mods-3-5.xsd")