Project

arxiv-dl

0.0
The project is in a healthy, maintained state
arxiv-dlxoengineering/arxiv-dlHomepageDocumentationSource CodeBug TrackerWiki
Command line tool and Ruby library for archiving arxiv.org papers as PDFs with sidecar metadata.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
 Popularity
Downloads
3,268
Stars
0
Forks
0
Watchers
0
 Releases
Current version
0.1.1
Total releases
2
First release
2026-07-22
Latest release
2026-07-23
 Development
Primary Language
Ruby
Licenses
MIT
Average date of last 50 commits
2026-05-16
Reverse Dependencies
0

 Dependencies

Runtime

feedjira
~> 4.0
http
~> 6.0
nokogiri
~> 1.19
ostruct
~> 0.6
stringex
~> 2.8
 Project Readme

arxiv-dl

Ruby gem to download papers and metadata from arxiv.org for offline archives.

For each paper it produces a self-contained per-paper folder with:

  • The PDF
  • The single-page abstract HTML
  • The full HTML version (with images and a deduplicated, cross-paper shared assets cache)
  • The TeX/LaTeX source tarball, extracted
  • Four sidecar metadata files: metadata.md, metadata.yaml, metadata.json, metadata.bib

Installation

gem install arxiv-dl

Or in a Gemfile:

gem 'arxiv-dl'

CLI usage

arxiv-dl <ARXIV_ID_OR_URL> [<ARXIV_ID_OR_URL>...]

Accepted input forms:

Form Example
Bare ID 2508.16190
Prefixed ID arXiv:2508.16190
Legacy ID cs/0002001, alg-geom/9708001
Versioned ID 2508.16190v1
Abstract URL https://arxiv.org/abs/2508.16190
PDF URL https://arxiv.org/pdf/2508.16190.pdf
HTML URL https://arxiv.org/html/2508.16190

Flags

Flag Description
-p PATH, --path PATH Root download directory
--rate-limit SECONDS Seconds between HTTP requests; 0 disables throttling
-v, --verbose Print step lines and per-request URL/byte logs to stdout
-q, --quiet Print nothing; success/failure conveyed via exit code
--version Print the gem version and exit
-h, --help Print help and exit

-v and -q are mutually exclusive.

Environment variables

Variable Effect
ARXIV_DOWNLOAD_PATH Root download directory (default: $HOME/Downloads/ArXiv_Papers)
ARXIV_RATE_LIMIT Seconds between HTTP requests (default: 3, per arxiv etiquette; 0 disables)

Precedence: CLI flag > ENV var > default.

Examples

Download a single paper to the default location:

arxiv-dl 2508.16190

Download to a custom directory:

arxiv-dl --path ~/Archives/arxiv 2508.16190

Download multiple papers, verbose:

arxiv-dl -v 2508.16190 1207.7214 cs/0002001

Disable rate limiting (when running against a local mirror, etc):

arxiv-dl --rate-limit 0 2508.16190

Output layout

$ARXIV_DOWNLOAD_PATH/                   # default: $HOME/Downloads/ArXiv_Papers
  _shared/                              # cross-paper deduplicated static assets
    arxiv.org/static/...
    cdn.jsdelivr.net/...
  YYYY/MM/DD/<primary_category>/<arxiv-id>-<slug>/
    <arxiv-id>.pdf
    <arxiv-id>-abstract.html
    metadata.md                         # YAML frontmatter + Markdown body
    metadata.yaml
    metadata.json
    metadata.bib                        # upstream BibTeX, falls back to synthesized
    html/
      <arxiv-id>.html                   # path-rewritten to local assets
      x1.png, x2.png, ...               # paper-specific images
    src/
      *.tex, *.bbl, ...                 # extracted from /src/<id> tarball

YYYY/MM/DD is the original submission date. <primary_category> is from the paper's metadata (cs.CL, math.NT, etc). <slug> is derived from the paper title (Unicode → ASCII, hyphenated, truncated to 80 chars at a word boundary).

For legacy IDs containing / (e.g. cs/0002001), the slash is replaced with - in the directory name (cs-0002001-...).

Library usage

require 'arxiv/downloader'

identifier = Arxiv::Downloader::Identifier.new '2508.16190'
client     = Arxiv::Downloader::Client.new                # 3-second rate limit by default
path       = Arxiv::Downloader::Archive.new(identifier, root: '/tmp/papers', client: client).run
# => "/tmp/papers/2025/08/22/cs.CL/2508.16190-comicscene154-a-scene-dataset-for-comic-analysis"

Development

script/setup    # install dependencies
script/test     # run specs and rubocop
script/console  # interactive prompt

License

MIT — see LICENSE.md.

Code of Conduct

See CODE_OF_CONDUCT.md.