The fulfil_api Ruby gem

The fulfil_api is a simple, powerful HTTP client written in Ruby to interact with Fulfil's API. It takes learnings from many years of working with Fulfil's APIs and turns it into an easy to use HTTP client.

Install the gem and add to the application's Gemfile by executing:

  $ bundle add fulfil_api

If bundler is not being used to manage dependencies, install the gem by executing:

  $ gem install fulfil_api

There are two ways of configuring the HTTP client:

1. Staticly through an initializer file (typically used in a Rails application)
2. Dynamically through calling the FulfilApi.with_config method.

The configuration of the FulfilApi client is thread-safe and therefore you can even combine both the static and dynamic configuration of Fulfil in case you need to.

# config/initializers/fulfil_api.rb

FulfilApi.configure do |config|
  config.access_token = FulfilApi::AccessToken.new(ENV["FULFIL_API_KEY"])
  config.merchant_id = "the-id-of-the-merchant"
end

with_config temporarily applies options on top of the currently active configuration (per thread) and reverts when the block returns. The options you pass are merged over the active config, so you only need to specify what changes — credentials and other settings are inherited rather than reset to their defaults.

FulfilApi.with_config(
  access_token: FulfilApi::AccessToken.new(ENV["FULFIL_API_KEY"]),
  merchant_id: "the-id-of-the-merchant"
) do
  # Query the Fulfil API
end

This makes it easy to use different settings in contexts with different constraints. For example, a web request bound by a 30s timeout can keep tight defaults globally, while a background job (which has more time) overrides just the timeouts and retries without re-passing credentials:

FulfilApi.with_config(
  request_options: { open_timeout: 5, read_timeout: 60, write_timeout: 30 },
  retry_options: { max_retries: 3 }
) do
  # Long-running work against the Fulfil API
end

The following configuration options are (currently) available throught both configuration methods:

• access_token (FulfilApi::AccessToken): The access_token is required to authenticate with Fulfil's API endpoints. Fulfil supports two types of access tokens: "OAuth" and "Personal" access tokens. The gem supports both tokens and defaults to the personal access token.

NOTE: To use an OAuth access token, use FulfilApi::AccessToken.new(oauth_token, type: :oauth). Typically, you would use the OAuth access token only when using the dynamic configuration mode of the gem.

• merchant_id (String): The merchant_id is the subdomain that the Fulfil instance is hosted on. This configuration option is required to be able to query Fulfil's API endpoints.

• request_options (Hash): The request_options are the per-request timeout options for the HTTP client. See https://lostisland.github.io/faraday/#/customization/request-options in faraday.

NOTE: With the persistent (keep-alive) adapter there is no single whole-request timeout; Faraday resolves read_timeout, open_timeout, and write_timeout independently. read_timeout is the value that governs a slow or stalled response.

• retry_options (Hash): Which requests faraday-retry retries when they fail before Fulfil answers them (a dropped keep-alive socket, a refused connection or a timeout). An error response from Fulfil is never retried. Supported keys:
  • max_retries (default 1): The maximum number of retries per request.
  • requests (default: every GET, and PUT requests to search_read and search_count): A hash of HTTP verbs, each mapped to true for every endpoint or to a list of patterns matched against the request path. Fulfil reads through PUT as well as writes through it, so only the read endpoints are retried by default. Retrying a write that timed out could apply it twice.

FulfilApi.configure do |config|
  config.retry_options = {
    max_retries: 2,
    requests: { get: true, put: [%r{/search_read\z}, %r{/search_count\z}, %r{/model/sale\.sale/\d+\z}] }
  }
end

• circuit_breaker (Faulty, optional): A Faulty instance to stop sending requests to a Fulfil instance that keeps failing. Off by default. See Stopping requests to a failing Fulfil instance.

• connection_options (Hash): Tuning for the persistent (keep-alive) connection. Supported keys:

  • max_retries (optional): Re-enables Ruby's built-in retry, which retries every idempotent request (GET/HEAD/PUT/DELETE/OPTIONS) whatever its endpoint, writes through PUT included. Off by default; prefer retry_options.
  • idle_timeout (Integer, optional): Seconds a pooled socket may sit idle before it is recycled. Lower this towards your server's keep-alive window to shrink the stale-socket window for non-idempotent requests.
  • pool_size (Integer, optional): Maximum number of concurrent connections kept in the pool.

NOTE: When retries are enabled, the worst-case time for a request is roughly (max_retries + 1) × read_timeout. On platforms with a hard request cap (e.g. Heroku's 30s router limit), keep read_timeout low enough that this product stays under the cap.

• oauth (FulfilApi::OAuth::Configuration): The credentials and options of the OAuth app used by the OAuth flow. Assign it a Hash, or set its options one by one on config.oauth.

An application that talks to several Fulfil instances shouldn't let one struggling instance tie up all of its threads. The gem can run every request through a Faulty circuit breaker. Faulty isn't a dependency of the gem, so add it to your own Gemfile first:

gem "faulty"

Then hand the gem a Faulty instance. Its circuit_defaults set the thresholds, and its storage decides whether the state is kept per process (in memory) or shared between processes (e.g. Redis):

FulfilApi.configure do |config|
  config.circuit_breaker = Faulty.new(
    storage: Faulty::Storage::Redis.new(client: Redis.new),
    circuit_defaults: { rate_threshold: 0.5, sample_threshold: 5, cool_down: 60 }
  )
end

Every Fulfil instance gets its own circuit, so one failing instance never stops requests to another. The Client and the TplClient of the same instance share a circuit. Timeouts, dropped connections and 5xx responses count as failures. A 4xx response is an answer and doesn't count, and a request that failed after retrying counts once.

While a circuit is closed, failed requests raise the same FulfilApi::HttpError they always do. Once it opens, requests raise a FulfilApi::CircuitOpen without touching the network, until Faulty's cool-down lets a test request through.

Fulfil caps the lifetime of personal access tokens, so an application that has to keep working unattended needs an OAuth token instead. The gem ships the whole flow as a Rails engine: a user is sent to Fulfil's consent screen, comes back with an authorization code, and the resulting token is stored as a FulfilApi::Installation.

The Rails parts of the gem only load when the gem is used from within a Rails application. Everywhere else, fulfil_api stays the plain HTTP client it has always been.

NOTE: The engine requires Rails 8.0 or newer.

1. Create an app in Fulfil's authentication dashboard and whitelist https://your-app.example.com/fulfil/callback as a redirection URL.
2. Run the install generator and its migration:

  $ bin/rails generate fulfil_api:install
  $ bin/rails db:migrate

The generator writes config/initializers/fulfil_api.rb, adds the migration for the fulfil_api_installations table, and mounts the engine at /fulfil. None of that changes how the application behaves: the routes have no callers, and the flow refuses to run until step 3 below. An application still on personal access tokens can install the gem and move to OAuth later.

3. Fill in the app's credentials, and name the controller the flow runs under:

# config/initializers/fulfil_api.rb

FulfilApi.configure do |config|
  config.merchant_id = ENV.fetch("FULFIL_MERCHANT_ID", nil)

  config.oauth.client_id = ENV.fetch("FULFIL_OAUTH_CLIENT_ID", nil)
  config.oauth.client_secret = ENV.fetch("FULFIL_OAUTH_CLIENT_SECRET", nil)
  config.oauth.scopes = %w[sale.sale]

  config.oauth.parent_controller = "AuthenticatedController"
end

NOTE: parent_controller has no default on purpose. The engine's controllers inherit from it, so it has to be a controller that authenticates the request — and, in an application serving many merchants, one that establishes the current tenant. An unauthenticated flow lets a stranger walk through it and record their own Fulfil workspace over yours; a flow outside the application's tenancy records installations against the wrong owner. Until it is set, the flow raises FulfilApi::OAuth::ConfigurationError.

NOTE: The access tokens are encrypted at rest with Active Record Encryption. Run bin/rails db:encryption:init and store the keys in your credentials if the application does not use it yet.

• client_id / client_secret (String): The app's credentials, found under App Credentials in Fulfil's authentication dashboard.
• scopes (Array<String>, default []): The scopes to request. See https://developers.fulfil.io for the full list.
• access_type (Symbol, default :offline_access): :offline_access grants a permanent token that keeps working when no user is around, which is what background jobs need. :user_session grants a token that expires with the user's Fulfil session. Fulfil has no refresh token, so an expired :user_session token can only be replaced by walking through the flow again.
• after_install_path (String or callable, default "/"): Where the user ends up after installing the app, unless they were sent to the flow from somewhere else. A callable receives the FulfilApi::Installation that was created.
• parent_controller (String, required): The controller the engine's own controllers inherit from, so the flow picks up the application's authentication, layout, and multi-tenancy. No default — see the note above.
• redirect_uri (String, optional): Defaults to the engine's callback URL, derived from the incoming request. Set it when the application sits behind a proxy that rewrites the host.

Include FulfilApi::Authenticated in a controller. Every action then runs with a client authenticated as the current installation, and a user without a usable token is sent through the flow and returned to where they were headed.

class SalesOrdersController < ApplicationController
  include FulfilApi::Authenticated

  def index
    @sales_orders = FulfilApi::Resource.set(model_name: "sale.sale").limit(50)
  end
end

By default the installation belongs to the application as a whole, which is what an application built for a single merchant needs. An application serving many merchants ties the installation to a record instead:

class Shop < ApplicationRecord
  include FulfilApi::Installable
end

A record is connected to a single Fulfil workspace. Installing it on another one replaces what came before rather than leaving the record connected to two, and drops the token the previous workspace granted.

Point the flow at the current tenant by overriding two methods in ApplicationController:

class ApplicationController < ActionController::Base
  private

  def fulfil_installation_owner
    Current.shop
  end

  def fulfil_merchant_id
    Current.shop.fulfil_merchant_id
  end
end

Outside of a request — in a background job, for instance — go through the record:

shop.with_fulfil_config do |client|
  client.get("model/sale.sale")
end

FulfilApi::Installation is a plain Active Record model, so an application-wide token works the same way:

FulfilApi::Installation.global.sole.with_config do |client|
  client.get("model/sale.sale")
end

NOTE: Currently, the gem is under heavy development. The querying interface of the gem is really basic at the moment. In the future, we will closer match the querying interface of ActiveRecord.

The gem uses an ActiveRecord like query interface to query the Fulfil API.

# Find one specific resource
sales_order = FulfilApi::Resource.set(model_name: "sale.sale").find_by(["id", "=", 100])
p sales_order["id"] # => 100

# Find a list of resources
sales_orders = FulfilApi::Resource.set(model_name: "sale.sale").where(["channel", "=", 4])
p sales_orders.size # => 500 (standard number of resources returned by Fulfil)
p sales_orders.first["id"] # => 10 (an example of an ID returned by Fulfil)

# Find a limited list of resources
sales_orders = FulfilApi::Resource.set(model_name: "sale.sale").where(["channel", "=", 4]).limit(50)
p sales_orders.size # => 50

# Include more resource details than the ID only
sales_orders = FulfilApi::Resource.set(model_name: "sale.sale").select("reference").where(["channel", "=", 4])
p sales_orders.first["reference"] # => SO1234

# Fetch nested data from a relation
line_items = FulfilApi::Resource.set(model_name: "sale.line").select("sale.reference")
p line_items.first["sale"]["reference"] # => SO1234

# Query nested data from a relation
line_items = FulfilApi::Resource.set(model_name: "sale.line").where(["sale.reference", "=", "SO1234"])
p line_items.first["id"] # => 10

NOTE: It's important to note that the results from the Fulfil API are cached. This prevents you from accidentally overasking the Fulfil API. To reload the resources from the Fulfil API after you've already fetchted them, use the .reload on the returned relation (e.g. line_items.reload).

Any data returned through the FulfilApi gem returns a list or a single FulfilApi::Resource. The data of the API resource is accessible through a Hash-like method.

sales_order = FulfilApi::Resource.set(model_name: "sale.sale").find_by(["id", "=", 100])
p sales_order["id"] # => 100

When you're requesting relational data for an API resource, you can access it in a similar manner.

sales_order = FulfilApi::Resource.set(model_name: "sale.sale").select("channel.name").find_by(["id", "=", 100])
p sales_order["channel"]["name"] # => Shopify

NOTE: Fulfil is not able to return nested data from Array-like API resources. If you want to find all line items of a sales order, it's typically better to query the line item resource directly.

# You can't do this
FulfilApi::Resource.set(model_name: "sale.sale").select("lines.reference").find_by(["id", "=", 100])

# You can do this (BUT it's not recommended)
sales_order = FulfilApi::Resource.set(model_name: "sale.sale").select("lines").find_by(["id", "=", 100])
line_items = FulfilApi::Resource.set(model_name: "sale.line").where(["id", "in", sales_order["lines"]])

# You can do this (recommended)
line_items = FulfilApi::Resource.set(model_name: "sale.line").find_by(["sale.id", "=", 100])

Whenever a request to Fulfil fails, the gem raises a FulfilApi::HttpError. Every HTTP status code has its own subclass, named after the status code it represents, so you can rescue the exact failure you care about:

begin
  FulfilApi::Resource.set(model_name: "sale.sale").find_by(["id", "=", 100])
rescue FulfilApi::HttpError::TooManyRequests => exception
  puts exception.message    # => "This user has exceeded an allotted request count. Try again later."
  puts exception.status_code # => 429

  sleep exception.response_headers["retry-after"].to_i
  retry
end

The message is the description reported by Fulfil. The raw response is available through #response_body, #response_headers and #status_code.

To catch anything that went wrong, rescue the base class instead:

rescue FulfilApi::HttpError => exception
  Rails.logger.error("Fulfil responded with #{exception.status_code}: #{exception.message}")

A request that never reached Fulfil — a connection reset, a DNS failure, a timeout — has no status code to name and raises a FulfilApi::HttpError itself.

FulfilApi::HttpError inherits from FulfilApi::Error, the base class of every error in this gem, so code that already rescues FulfilApi::Error keeps working unchanged.

Rescuing tells you about the one call you wrapped. To report every failure of the Fulfil API — to count rate limit hits in your APM, to page on authentication failures, to log what Fulfil actually said — subscribe once instead.

Every FulfilApi::Error publishes an ActiveSupport::Notifications event named error.fulfil_api when it is raised. FulfilApi.on_error is the shorthand for listening to it:

# config/initializers/fulfil_api.rb

FulfilApi.on_error do |error|
  Appsignal.increment_counter("fulfil_api_errors", 1, error: error.class.name)
end

Because every error carries its own class, you can report only the failures you care about:

FulfilApi.on_error do |error|
  next unless error.is_a?(FulfilApi::HttpError::TooManyRequests)

  Appsignal.increment_counter("fulfil_api_rate_limit_exceeded")
end

Subscribing through ActiveSupport::Notifications directly gives you the full payload, which carries the HTTP details of a FulfilApi::HttpError as separate keys — handy as metric dimensions:

ActiveSupport::Notifications.subscribe(FulfilApi::Error::EVENT_NAME) do |event|
  event.payload[:exception]        # => ["FulfilApi::HttpError::TooManyRequests", "Try again later."]
  event.payload[:exception_object] # => the FulfilApi::HttpError::TooManyRequests instance
  event.payload[:status_code]      # => 429
  event.payload[:response_headers] # => { "retry-after" => "5", ... }
  event.payload[:response_body]    # => the raw response of Fulfil
end

Both methods return the subscriber, so you can stop listening again with ActiveSupport::Notifications.unsubscribe(subscriber).

A few things worth knowing:

• The event is published when an error is raised, not when it is built. Rescuing it afterwards does not suppress the notification, and retrying a request publishes one event per attempt — which is exactly what you want when counting rate limit hits.
• The subscriber runs on the thread that raised the error, while the error travels up the stack. Keep it cheap and hand anything slow to a background job.
• A subscriber cannot change the behaviour of your application. If it raises, the exception is swallowed and reported on $stderr rather than replacing the FulfilApi::Error on its way up.

The gem also includes a client for Fulfil's 3PL Integration API. This is a separate API that allows third-party logistics providers to interact with Fulfil on behalf of a merchant.

Configure the 3PL client through the tpl option in the configuration block:

FulfilApi.configure do |config|
  config.merchant_id = "the-id-of-the-merchant"

  config.tpl = {
    auth_token: ENV["FULFIL_3PL_AUTH_TOKEN"], # required
    merchant_id: "a-different-merchant-id",   # optional, falls back to config.merchant_id
    api_version: "v1"                         # optional, defaults to "v1"
  }
end

The 3PL client is accessible via FulfilApi.tpl_client and supports the standard HTTP methods:

# GET request with optional URL parameters
FulfilApi.tpl_client.get("inbound-transfers", page: 1, per_page: 25)

# POST request with a request body
FulfilApi.tpl_client.post("inbound-transfers/receive.json", { tracking_number: "ABC123" })

# PUT request with a request body
FulfilApi.tpl_client.put("inbound-transfers/receive.json", { status: "received" })

# PATCH request with a request body
FulfilApi.tpl_client.patch("inbound-transfers/receive.json", { status: "received" })

NOTE: For the full list of available 3PL API endpoints, refer to the Fulfil 3PL Integration API documentation.

After checking out the repo, run bin/setup to install dependencies. Then, run rake test to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bin/rake install.

To release a new version, run the bin/release script. This will update the version number in version.rb, create a git tag for the version, push git commits and the created tag, and push the .gem file to rubygems.org.

Bug reports and pull requests are welcome on GitHub at https://github.com/codeturebv/fulfil_api. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.

The gem is available as open source under the terms of the MIT License.

Everyone interacting in the Fulfil project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.