Esirb

Esirb is a light wrapper around Faraday for the purposes of interacting with EVE Online's ESI API.

I wanted something low maintenance, so this gem makes a few assumptions to let you dynamically generate any path that might end up on the ESI.

• Installation
• Usage
  • Quick start
  • Create a client
  • Build the path
  • Make the request
  • Pagination
  • Configuration
  • Customizing the connection
  • Caching
  • Errors
  • Retries
  • Rate limits
• Development
• Contributing
• License

Install the gem and add to the application's Gemfile by executing:

bundle add esirb

If bundler is not being used to manage dependencies, install the gem by executing:

gem install esirb

client = Esirb::Client.new
alliance_ids = client.alliances.get.body # => [99000006, 99000008, ...]

client = Esirb::Client.new

A client holds no state from building paths or making requests, so one client can be shared between threads.

client = Esirb::Client.new(token: "...")

ESI access tokens expire after about 20 minutes. If a client lives longer than that, pass something that returns the current token instead, such as a lambda. esirb calls it before every request:

client = Esirb::Client.new(token: -> { character.access_token_refreshing_if_expired })

esirb doesn't refresh tokens itself: your app keeps the refresh token and does the refresh, and esirb only asks for the access token. If the callable returns nil or a blank string, the request raises Esirb::Error instead of being sent.

Treat ESI's static path segments as method names, and path parameters as the arguments for those methods. Chain these methods to build the path to the desired ESI route.

path = client.route(30000142, 30002187)
path.to_s # => "/route/30000142/30002187"

path = client.ui.autopilot.waypoint
path.to_s # => "/ui/autopilot/waypoint"

path = client.characters(123456789).location
path.to_s # => "/characters/123456789/location"

Each call returns a new path, so you can reuse part of one:

character = client.characters(123456789)
character.location.to_s # => "/characters/123456789/location"
character.skills.to_s   # => "/characters/123456789/skills"

Path params can be strings, symbols or integers, and are URL-encoded. Query params don't go in the path; pass them to the HTTP verb with params:.

Some ESI segments, like freelance-jobs, use hyphens, which Ruby method names can't contain. Call them with underscores instead:

client.characters(123456789).freelance_jobs.to_s # => "/characters/123456789/freelance-jobs"

esirb keeps a list of ESI's hyphenated segments, and only those names are converted. Segments that really use underscores, like system_kills, stay as they are. For a segment esirb doesn't know about yet, add it exactly as written with segment:

client.characters(123456789).segment("new-thing", 42).to_s # => "/characters/123456789/new-thing/42"

Call an HTTP verb on the path: get, head, post, put, patch or delete. Each returns a Faraday::Response, with JSON bodies already parsed.

client = Esirb::Client.new(token: -> { character.access_token }) # needs the esi-location.read_location.v1 scope
resp = client.characters(123456789).location.get
resp.status # => 200
resp.body   # => { "solar_system_id" => 30000142 }

Pass query parameters with params:, a body with body: (sent as JSON), and extra headers with headers:. The API Explorer lists what each route takes.

client.markets(10000002).orders.get(params: { type_id: 34, order_type: "sell" })
client.universe.names.post(body: [30000142, 30002187])

Headers passed to a verb are added to the client's default headers, and replace a default with the same name.

ESI splits long lists into pages, using one of three styles depending on the route. You can tell which from the route's query parameters in ESI's API Explorer:

Each helper makes as many requests as it takes to collect the pages, so one call can be dozens of requests, each counting toward ESI's rate limit. They're sent one at a time, never in parallel. Each request goes through esirb's cache, retries and error-limit guard like any other, and keeps the params: and headers: you pass. If any request fails, the error is raised and nothing is returned.

orders = client.markets(10000002).orders.get_all_pages(params: { type_id: 34 })
# => one array with every order from every page

Under the hood, that's the same as fetching the pages yourself:

orders = client.markets(10000002).orders

first = orders.get(params: { type_id: 34, page: 1 })
total_pages = first.headers["X-Pages"].to_i
records = first.body

(2..total_pages).each do |page|
  response = orders.get(params: { type_id: 34, page: })
  # get_all_pages also checks this, and starts over from page 1 if it doesn't match:
  raise "data changed between pages" unless response.headers["Last-Modified"] == first.headers["Last-Modified"]

  records.concat(response.body)
end

Every page of one list should have the same Last-Modified. A different one means ESI refreshed the data partway through, so the pages would mix old and new records. get_all_pages then starts over from page 1 once, and raises Esirb::PagesChanged if it happens again. Fetching the whole list takes as many requests as there are pages, or up to twice that if it has to start over.

These responses aren't a plain list. They're an object holding the records under a key that depends on the route (freelance_jobs, projects, listings...), plus a cursor with two values: before, pointing at older records, and after, pointing at newer ones. get_by_cursor finds the records' key for you. There are two ways to call it.

Everything, for a first sync:

jobs, after = client.freelance_jobs.get_by_cursor
# jobs:  every record, oldest first
# after: save this to fetch only changes later

Done by hand, that's:

freelance_jobs = client.freelance_jobs

response = freelance_jobs.get # no cursor: the most recent records
jobs = response.body["freelance_jobs"]
after = response.body.dig("cursor", "after") # where the list ends now; save it for later
before = response.body.dig("cursor", "before")

while before
  response = freelance_jobs.get(params: { before: })
  older = response.body["freelance_jobs"]
  break if older.empty?

  jobs = older + jobs # get_by_cursor also skips records it already has, matched by id
  before = response.body.dig("cursor", "before")
end

Only what changed since a saved after, to keep up to date:

changed, after = client.freelance_jobs.get_by_cursor(after: saved_after)

Done by hand:

changed = []
after = saved_after

loop do
  response = freelance_jobs.get(params: { after: })
  newer = response.body["freelance_jobs"]
  break if newer.empty?

  changed += newer # get_by_cursor also replaces records it already has with these newer copies
  after = response.body.dig("cursor", "after") || break
end
# Save `after` again for next time.

after and before values stay valid for days, according to CCP. Pass them back exactly as you got them, without trying to decode them. A record can show up on more than one page, for example when it changed while the pages were being fetched. get_by_cursor returns each record once, matched by its id: when following before toward older records it keeps the copy it already had, and when following after toward newer records it keeps the newest copy.

ESI returns the newest records first. To get older ones, you pass the oldest ID you've seen as from_id. get_from_id does that for you, one request after another, so you never pass from_id yourself.

Everything, newest first:

transactions = client.characters(123456789).wallet.transactions
all = transactions.get_from_id

Done by hand:

records = transactions.get.body # the newest records, newest first

loop do
  from_id = records.last["transaction_id"]
  # The response starts with the from_id record again, followed by older ones.
  older = transactions.get(params: { from_id: }).body.reject { |record| record["transaction_id"] == from_id }
  break if older.empty?

  records.concat(older)
end

Only records newer than one you already have:

newer = transactions.get_from_id(stop_at: newest_known_id)

Done by hand:

newer = []
from_id = nil

loop do
  params = from_id ? { from_id: } : {}
  page = transactions.get(params:).body.reject { |record| record["transaction_id"] == from_id }
  stop_index = page.index { |record| record["transaction_id"] == newest_known_id }
  break newer.concat(page.first(stop_index)) if stop_index
  break if page.empty?

  newer.concat(page)
  from_id = page.last["transaction_id"]
end

stop_at is esirb's option, not an ESI parameter: an ID you already have, usually the newest one you stored last time. The walk ends when it reaches that record, which isn't included. If it's on the first page, that's one request. If ESI no longer has it, the walk fetches everything.

Records are identified by transaction_id, which is what wallet transactions use. Pass id_key: if a route uses a different field.

In a Rails app, generate an initializer with each setting and a note on what it does:

bin/rails generate esirb:install

That writes config/initializers/esirb.rb, with your app's name in the User-Agent, esirb's current compatibility date pinned, and Rails.cache as the cache store. Edit the User-Agent's contact email before deploying.

Outside Rails, or to write it yourself, set defaults for every new client once:

Esirb.configure do |config|
  config.user_agent = "mappy/1.2 (admin@example.com)"
  config.compatibility_date = "2026-08-18"
  config.cache_store = Rails.cache
end

Options passed to Esirb::Client.new override these for that client:

Setting tenant or language to nil leaves that header out. token is set per client only.

ESI changes a route's responses only under a new compatibility date, and a request without a date gets ESI's oldest behavior, so esirb always sends one. New fields and enum values can still appear within a date, so don't fail on ones you don't recognize. To move to a newer date, check ESI's changes since your current one against your code, then set compatibility_date. A new esirb version can move the default date; the CHANGELOG says when.

esirb uses Faraday. To add middleware, change the adapter or set other connection options, give Esirb.configure a faraday callable, or pass a block to a client. Both receive the Faraday connection builder after esirb has set up its own middleware:

Esirb.configure do |config|
  config.faraday = ->(f) { f.response :logger, Rails.logger }
end

client = Esirb::Client.new { |f| f.adapter :net_http_persistent }

(:net_http_persistent comes from the faraday-net_http_persistent gem, which you'd add to your Gemfile.) The configured callable runs first, then the client's block. Middleware added this way sits closest to the network, so a logger sees the requests that actually go to ESI, not responses served from the cache.

esirb caches responses only when you give it a cache_store, usually once in Esirb.configure so every client shares it. Any object with read, write and delete works, such as Rails.cache or another ActiveSupport::Cache store.

Without a cache_store, every request goes to ESI, even when the last response is still fresh. CCP asks apps not to request data again before its Expires time or work around ESI's cache, and doing that can get an app banned, so set a store in production. The store also lets esirb pause before the error limit runs out.

Esirb.configure do |config|
  config.cache_store = ActiveSupport::Cache::RedisCacheStore.new(url: ENV["REDIS_URL"], namespace: "esirb", expires_in: 1.day)
end

Esirb::Client.new(cache_store: nil) # no caching for this client

Use a store that expires entries. Cache keys include the access token, and tokens change about every 20 minutes, so entries for old tokens are never read again. A store without expiry, including Faraday::HttpCache::MemoryStore, keeps them forever.

esirb follows ESI's cache headers:

• Responses are kept separate per URL, tenant, language, compatibility date and access token, so a shared store never serves one character's data to another.
• Authenticated responses are cached too. ESI marks them private, max-age=5, must-revalidate, so they're reused for 5 seconds and then revalidated, which returns a cheap 304 Not Modified when nothing has changed.

Error statuses from ESI raise a subclass of Esirb::ResponseError, with ESI's own message:

begin
  client.universe.systems(1).get
rescue Esirb::NotFound => e
  e.message # => "404 System not found (GET https://esi.evetech.net/universe/systems/1)"
  e.status  # => 404
  e.body    # => { "error" => "System not found" }
  e.headers # => response headers, e.g. e.headers["X-ESI-Error-Limit-Remain"]
end

Error responses aren't free: a 4xx costs 5 rate limit tokens and counts toward ESI's error limit, so don't keep repeating a request that fails.

Everything esirb raises is an Esirb::Error. When an error status is a normal answer, rescue its class. For example, ESI returns 404 for a character's fleet when they aren't in one:

fleet = begin
  client.characters(123456789).fleet.get.body
rescue Esirb::NotFound
  nil
end

ESI often answers with a 502, 503 or 504 that clears up within a second or two, so esirb retries those, along with timeouts and connection failures. By default it retries twice, waiting about half a second and then a second, with some randomness so many clients don't retry at the same moment.

• Only GET, HEAD, PUT and DELETE are retried. POST isn't, since repeating it might do something twice, like sending a mail.
• If ESI sends Retry-After, esirb waits that long instead. If it asks for more than 10 seconds, esirb raises straight away rather than tie up your process.
• A 429 is retried after its Retry-After, if that's 10 seconds or less. A 429 without Retry-After, which some routes send from inside EVE's servers, or one asking for longer, is raised straight away.
• When a failed response shows fewer than 20 errors left in ESI's error limit, esirb stops retrying. Running out blocks every ESI route for your app until the window resets, and each failed retry counts as another error.
• Other errors, including 500, other 4xx and 420, aren't retried.

retries is the total for one request, whatever the reasons: with the default of 2, a request is sent at most 3 times. Set retries in Esirb.configure or per client to change the number, or to 0 to turn retries off. Worst case, a request takes about (retries + 1) * timeout seconds plus the waits, so lower timeout or retries together if that's too long.

ESI limits requests in one of two ways, depending on the route (details). Esirb::RateLimit reads either from a response or an error:

response = client.characters(123456789).location.get
limit = Esirb::RateLimit.new(response)
limit.group     # => "char-location"
limit.limit     # => 1200 tokens...
limit.window    # => ...per 900 seconds
limit.remaining # => 1195
limit.used      # => 2, what this request cost

# On routes still using the older error limit:
limit.error_limit_remaining # => 100 errors left in this window
limit.error_limit_reset     # => 46 seconds until it resets

# Errors carry the limits too:
begin
  client.characters(123456789).location.get
rescue Esirb::ResponseError => e
  e.rate_limit.remaining
end

Values a route doesn't send are nil. Errors cost more than successful requests (5 tokens for a 4xx against 2 for a 2xx), so avoid repeating requests that fail. CCP also recommends slowing down as remaining nears zero, and staggering scheduled work rather than running it all at the same minute.

ESI's older error limit allows about 100 error responses a minute. Once it's used up, every ESI route returns 420 until the window resets, and repeatedly ignoring it can get an app banned. With a cache_store, esirb keeps track of it for you: when a response shows fewer than 20 errors left, every client sharing that store stops sending requests until the window resets, and raises Esirb::ErrorLimitReached instead.

begin
  client.universe.systems(30000142).get
rescue Esirb::ErrorLimitReached => e
  e.retry_after # => 37, seconds until ESI's error limit resets
end

ESI counts the error limit per IP address, so esirb keeps one count per ESI host and tenant, shared by every client and process using the store. If several servers with different IP addresses share one store, an error burst on one pauses the others too, which errs on the side of sending less.

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

After moving esirb's default compatibility date, run bundle exec rake esi:check_segments to compare esirb's list of hyphenated segments with ESI's OpenAPI spec.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and the created tag, and push the .gem file to rubygems.org.

Bug reports and pull requests are welcome on GitHub at https://github.com/sicks/esirb.

The gem is available as open source under the terms of the MIT License.