Project

querrel

0.0
No release in over 3 years
Low commit activity in last 3 years
Parallel query for ActiveRecord
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
 Dependencies

Development

Runtime

 Project Readme

Querrel

Querrel makes it easy to query multiple databases in parallel (threads) using ActiveRecord, as if you were querying one database.

Installation

Add this line to your application's Gemfile:

gem 'querrel'

And then execute:

$ bundle

Or install it yourself as:

$ gem install querrel

Usage

Basic

You can use with Querrel directly via it's top level namespace, or by creating an instance:

all_brands = Querrel.query(Brand.all, on: ['db1', 'db2', 'db3'])
# is the same as
all_brands = Querrel.new(['db1', 'db2', 'db3']).query(Brand.all)

Either of the above will give you an array of all the Brand objects from all the given databases. The records will be marked as readonly.

Advanced

query will yield a block with the passed in ActiveRecord::Relation, this allows you to do additional operations on the results before they are merged, for example you could use pluck:

all_brand_names = q.query(Brand.all) do |s|
  s.pluck(:name)
end

There is also a run method which instead of running a preprescribed scope and merging, will take a block which allows you do anything you want with the specified database connection, for instance:

require 'thread'
all_brands = []
b_s = Mutex.new
all_products = []
p_s = Mutex.new

Querrel.run(on: dbs) do |db|
  b_s.synchronize { all_brands += Brand.all.to_a }
  p_s.synchronize { all_products += Product.all.to_a }
end

Databases

There are three ways in which you can instruct Querrel which databases to use:

  1. Pass in an array of environments, e.g. Querrel.new([:customer1, :customer2])

  2. Pass in an array of database names, e.g. Querrel.new(['dbs/customer1.sqlite3', 'dbs/customer2.sqlite3'], db_names: true)

  3. Pass in a hash of named connection configurations, e.g.:

    Querrel.new({
      one: {
        adapter: "sqlite3",
        database: "test/dbs/test_db_1.sqlite3"
      },
      two: {
        adapter: "sqlite3",
        database: "test/dbs/test_db_2.sqlite3"
      }
    })

Configuration

By default Querrel will use a maximum of 20 threads, but you can adjust this using the :threads option:

q = Querrel.new(dbs, threads: 50)

Contributing

  1. Fork it ( https://github.com/meritec/querrel/fork )
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request