PackAPI
Building blocks for implementing APIs around domain models.
Overview
PackAPI provides a comprehensive set of tools for building robust API layers on top of domain models. It includes utilities for:
- Data transformation - Elements for passing data out of the API
- Filter definitions - Elements for describing the filters supported by query endpoints in the API
- Attribute mapping - Elements for building the mapping between domain models and API models
- Query building - Elements for building query endpoints based on user inputs (sort, filter, pagination)
- Batch operations - Elements for retrieving multiple pages of data from other query endpoints
Motivation
Separation of concerns and information hiding within the module.
In order to separate the public interface from the (private) implementation of our subsystems (modules), we needed to create an API that did not depend on our ActiveRecord models. We chose to implement this separation using value objects (built using dry-types, but could have been built using Ruby Data type). Beyond the API methods, the public interface definition is then captured in the explicit attribute list of these value objects.
However, building the mapping between the domain models and the API value objects can be tedious and error-prone. We created this gem to provide reusable building blocks to make this mapping easier to define and maintain.
Installation
Add this line to your application's Gemfile:
gem 'pack_api'And then execute:
bundle installOr install it yourself as:
gem install pack_apiRequirements
- Ruby >= 3.0.0
- ActiveRecord >= 7.0
- dry-types ~> 1.8
Features
Mapping
The mapping module provides tools for transforming data between domain models and API representations:
-
AttributeMap- Define bidirectional mappings between model and API attributes -
AttributeMapRegistry- Centralized registry for attribute mappings -
ModelToAPIAttributesTransformer- Transform model attributes to API format -
APIToModelAttributesTransformer- Transform API attributes to model format -
ValueObjectFactory- Create value objects from raw data
Querying
Build flexible query interfaces with support for filtering, sorting, and pagination:
-
ComposableQuery- Build complex queries from simpler components -
CollectionQuery- Query ActiveRecord collections based on arguments for pagination, filtering and sorting -
AbstractFilter- Base class for custom filters -
FilterFactory- Create filters dynamically based on query method arguments -
SortHash- Handle sorting parameters - Base class filter implementations for boolean, enum, numeric, and range filters
Pagination
Enable paginated access to resources across the API:
-
Paginator- Standard pagination implementation -
PaginatorBuilder- Build paginators with custom configurations -
SnapshotPaginator- Enable record iteration (one by one) across results in a page, even when the underlying records change state (and may no longer be at the same position in the result set)
Types
Type definitions and validation using dry-types:
-
BaseType- Base type for value objects -
CollectionResultMetadata- Metadata for paginated collections -
Result- Generic result type to be returned from your API methods -
AggregateType- Composite types made of attributes from other types - Filter definition types for various data types
Batch Operations
Utilities for processing large datasets efficiently:
-
ValuesInBatches- Process values in batches -
ValuesInBackgroundBatches- Process values in background batches
Usage
Basic Example
See the test files for more detailed examples, but here's a simple usage example.
Let's assume your system has Author, Comment and BlogPost ActiveRecord models.
- Define value objects to contain the data passed out of the API:
# public/author_type.rb
class AuthorType < PackAPI::Types::BaseType
attribute :id, ::Types::String
attribute :name, ::Types::String
end
# public/comment_type.rb
class CommentType < PackAPI::Types::BaseType
attribute :text, ::Types::String
end
# public/blog_post_type.rb
class BlogPostType < PackAPI::Types::BaseType
attribute :id, ::Types::String
attribute :legacy_id, ::Types::String
attribute :title, ::Types::String
attribute :persisted, ::Types::Bool
attribute :contents, ::Types::String.optional
optional_attribute :associated, AuthorType
optional_attribute :notes, ::Types::Array.of(CommentType)
optional_attribute :earnings_float, ::Types::Coercible::Float
end- Define the rules for mapping between the domain models and the API value objects:
# api/author_attribute_map.rb
class AuthorAttributeMap < PackAPI::Mapping::AttributeMap
api_type AuthorType
model_type Author
map :name, to: :name
map :id, to: :external_id
map :blog_posts
end
# api/comment_attribute_map.rb
class CommentAttributeMap < PackAPI::Mapping::AttributeMap
api_type CommentType
model_type Comment
map :text, to: :txt
end
# api/blog_post_attribute_map.rb
class BlogPostAttributeMap < PackAPI::Mapping::AttributeMap
api_type BlogPostType
model_type BlogPost
# example API attribute mapped to a model attribute of the same name
map :title
map :contents, from_model_attribute: ->(attachment) { attachment&.blob }
# example API attribute mapped to a model attribute of a different name
map :id, to: :external_id
# example of API attribute ending in "_id"
map :legacy_id
# example of API attribute mapped to a model method (unidirectional)
map :persisted, to: :persisted?, readonly: true
# example of API association mapped to a model association
# (the association_id can also be passed in, and reported on during error cases)
map :associated, to: :author,
from_api_attribute: ->(author_id) { Author.find_by(external_id: author_id) }
map :notes, to: :comments, transform_nested_attributes_with: CommentAttributeMap
# example of OPTIONAL API attribute (association) mapped to a model method (bidirectional)
map :earnings_float, to: :earnings_float
end-
Implement filters.
-
Implement a query endpoint using the attribute map:
def query_blog_posts(cursor = nil, search = nil, sort = nil, page_size = 50, filters = {}, optional_attributes = [])
collection = BlogPost.all
# avoid N+1 queries for optional attributes that are associations
if optional_attributes.include?(:associated)
collection = collection.includes(:author)
end
# convert the search terms to something used by the CollectionQuery to perform searches (hash of model attributes to search terms)
if search.present?
# search through blog post title and comments
collection = collection.includes(:comments)
model_search = {
'title' => search,
"#{Comment.table_name}.txt" => search,
}
end
# convert the API sort to model sort
model_sort = BlogPostAttributeMap.model_attribute_keys(PackAPI::Querying::SortHash.new(sort))
# convert the API filters to model filters
model_filters = BlogPostFilterMap.new.from_api_filters(filters)
# build and execute the query
query = PackAPI::Querying::CollectionQuery.new(collection:)
query.filter_factory = Filters::BlogPost::FilterFactory.new
query.call(cursor:, per_page: page_size, sort: model_sort, search: model_search, filters: model_filters)
# build and return the result
PackAPI::Types::Result.from_collection(models: query.results,
value_object_factory: ValueObjectFactory.new,
optional_attributes:,
sort: BlogPostAttributeMap.api_attribute_keys(query.sort),
paginator: query.paginator)
endTesting with Shared Examples
PackAPI includes RSpec shared examples to help test your API query methods. These are opt-in and only need to be loaded if you're using RSpec.
Loading Shared Examples
In your spec_helper.rb or rails_helper.rb, require the shared examples you need:
# Load all shared examples
require 'pack_api/rspec/shared_examples_for_api_query_methods'
require 'pack_api/rspec/shared_examples_for_paginated_results'
# Or load them individually as needed
require 'pack_api/rspec/shared_examples_for_api_query_methods'Using the Shared Examples
Testing API Query Methods:
RSpec.describe 'query_blog_posts' do
let(:api_query_method) { method(:query_blog_posts) }
let(:resources) { BlogPost.all }
it_behaves_like 'an API query method'
# With custom options
it_behaves_like 'an API query method',
model_id_attribute: :uuid,
supports_search: true do
let(:search_terms) { "searchable text" }
let(:matched_resources) { BlogPost.where("title LIKE ?", "%searchable%") }
end
endTesting Paginated Methods:
RSpec.describe 'paginated query' do
let(:paginated_api_query_method) { method(:query_blog_posts) }
let(:paginated_resources) { BlogPost.all }
it_behaves_like 'a paginated API method', model_id_attribute: :external_id
endDevelopment
After checking out the repo, run:
bundle installRun the test suite:
bundle exec rspecContributing
Bug reports and pull requests are welcome on GitHub at https://github.com/flytedesk/pack_api.
License
The gem is available as open source under the terms of the MIT License.