LineUp
With LineUp you can enqueue Resque jobs in arbitrary Redis namespaces.
Resque uses a Set and a List to keep track of all Queues and their Jobs.
- The Set is usually located at
resque:queuesand contains a list of (lower-cased, underscored) Strings, each representing a queue name - Each queue is a List located at
resque:queue:my_job(withmy_jobas queue name forMyJob-jobs in this example) - Each job inside of a queue is a JSON or Marshal'ed Hash with the keys
classandargs, for example{ class: 'MyJob', args: [123, some: thing] }.to_json
Depending on how you configure the Redis backend for Resque, you will end up in a different namespace:
- If you use
Resque.redis = Redis::Namespace.new(:bob, ...), Resque detects that you passed in anRedis::Namespaceobject and will not add any additional namespace. So the queue Set in this example will be located atbob:queues - Any other
Redis.new-compatible object will get theresque-namespace added. SoResque.redis = Redis.new(...)will cause the queue Set to be located atresque:queues
If you use multiple applications, you should make sure that each of them has its own namespace. You would normally achieve that with Resque.redis = Redis::Namespace.new('myapp:resque', ...) so that the queue Set would be located at myapp:resque:queues.
So far so good, but there is no way to enqueue a Job for an application from inside another namespace, say otherapp:resque:queues, without maintaining an additional connection to Redis in the other app's namespace. So far, the only solution has been to share the resque:queues namespace between all applications and have separate queue-names, such as myapp-myjob and otherapp-myjob, but that is not really separating namespaces.
That's where LineUp comes in, it doesn't even need Resque. It goes right into Redis (scary huh?), just as Resque does internally.
Examples
Setup
If you use the Raidis gem, you do not need any setup. Otherwise, a manual setup would look like this:
redis = Redis::Namespace.new 'myapp:resque', redis: Redis.new(...)
Resque.redis = redis
LineUp.redis = redisUsage
With the setup above, Resque lies in the myapp:resque-namespace. So you can enqeueue jobs to the very same application by using Resque.enqueue(...).
This is how you can enqueue a job for another applications:
LineUp.push :otherApp, :SomeJob, 12345, some: thingThis will enqueue to other_app:resque:some_job with arguments [12345, { 'some' => 'thing' }] and make sure that the other_app:resque:queues Set references the queue List.
Gotchas
-
Resque.redisMUST respond to a method called#namespacewhich takes a block and yields a new namespace. See this commit. Currently LineUp requires a non-rubygems fork ofRedis::Namespacein order to be able to use this cutting-edge method. - Currently the jobs are encoded using
MultiJsononly, notMarshal, feel free to commit a patch if you need the latter - You cannot share the
resqueroot namespace. LineUp defaults to theapplication:resquenamespace (because that's the only scenario I can think of that would make you want to use LineUp in the first place :)