Project

ar_shard

0.01
The project is in a healthy, maintained state
Isolated Multibase Support for ActiveRecord Models with dynamic config
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
 Project Readme

AR Shard

Gem Version Build Status

AR Shard is non-intrusive extenstion for ActiveRecord to add threadsafe Model based sharding or just multidatabase connection. As this gem uses connection_handlers it can work only with ActiveRecord version 5 and above.

Features

  • Multiple Databases support for Rails5
  • Define config as a Proc
  • Shard only part of you ActiveRecord w/o sharing connection
  • Work with you Sharded Model and AR model at the same time
  • Threadsafe. Battle-tested with Sidekiq and 4 mixed Databases

Limitations

  • Doesn't care about migrations
  • Due to nature of connection_handlers all DB connections are established at boot time
  • if data structure is different between databases you should use ModelName.reset_column_information

Installation

Add this line to your application’s Gemfile:

gem 'ar_shard', '~> 1.0'

Usage

Define your separate AR like:

class ShardRecord < ActiveRecord::Base
  self.abstract_class = true

  connected_shards shards: [
    { shard_1: { adapter: 'sqlite3', database: 'shard_1' } },
    { shard_2: { adapter: 'postgresql', database: 'shard_2' } },
    { shard_3: { adapter: 'sqlite3', database: 'shard_3' } }
  ]
end

or with Proc

class ShardRecord < ActiveRecord::Base
  self.abstract_class = true

  connected_shards shards: -> { YAML.load_file('path/to/config/shards.yml') }
end

Where shards.yml is like:

- shard_1:
    adapter: mysql2
    host: blabla
    username: blabla
    password: secret
    database: remote_database_name
    port: 3306
    pool: 10
- shard_2:
    adapter: postgresql
    host: blabla
    username: blabla
    password: secret
    database: shard_name
    port: 5432
    pool: 10

given we have models like:

class User < ActiveRecord::Base

end

class Sharded::User < ShardRecord

end

AR Shard is isolating and rotating connections. Same like Rails Multibase support but without overlap. Now we can do crazy things like:

# Connecting to another DB
ShardRecord.with_shard(:shard_name) do
  Sharded::User.find_each do |user|
    # Working with our DB and another at the same time!

    u = User.find_by(email: user.email)
    u.update(name: user.name)
  end
end

Contributing

To get started with development:

git clone https://github.com/noma4i/ar_shard.git
cd ar_shard
bundle install
bundle exec appraisal rake test

License

The gem is available as open source under the terms of the MIT License.