Project

work_queue

0.03
No commit activity in last 3 years
No release in over 3 years
A tunable work queue, designed to coordinate work between a producer and a pool of worker threads.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
 Dependencies
 Project Readme

Description¶ ↑

A work queue is designed to coordinate work between a producer and a pool of worker threads. When some task needs to be performed, the producer adds an object containing the task routine to the work queue. If the work queue is full, the producer will block until a worker thread removes an object from the queue. Eventually, one of the worker threads removes the object from the work queue and executes the routine. If the work queue is empty, a worker thread will block until an object is made available by the producer.

Work queues are useful for several reasons:

  • To easily perform tasks asynchronously and concurrently in your application;

  • To let you focus on the work you actually want to perform without having to worry about the thread creation and management;

  • To minimize overhead, by reusing previously constructed threads rather than creating new ones;

  • To bound the resources used, by setting a limit on the maximum number of simultaneously executing threads;

Usage¶ ↑

Install the gem:

gem install work_queue

Run the code:

require 'rubygems'
require 'work_queue'
wq = WorkQueue.new
wq.enqueue_b { puts "Hello from the WorkQueue" }
wq.join

It’s generally recommended to bound the resources used:

# Limit the maximum number of simultaneous worker threads
WorkQueue.new 10, nil
# Limit the maximum number of queued tasks
WorkQueue.new nil, 20

Example(s)¶ ↑

Download multiple files in parallel:

require 'open-uri'
require 'rubygems'
require 'work_queue'

wq = WorkQueue.new 5

(1..6605).each do |number|
  wq.enqueue_b do
    open("rfc#{number}.txt", "wb") do |file|
      file.write open("http://www.rfc-editor.org/rfc/rfc#{number}.txt").read
      puts "rfc#{number} downloaded"
    end
  end
end

wq.join

License¶ ↑

Released under the MIT license.