Allows you to define pipelines in Sidekiq, an ordered chain of jobs.
class WorkerOne
include Sidekiq::Worker
include SidekiqPipeline
def pipeline_perform(params)
puts params["key"]
end
end
class WorkerTwo
include Sidekiq::Worker
include SidekiqPipeline
def pipeline_perform(params)
puts params["data"]
end
end
tasks = {
pipeline: [{
class: "WorkerOne",
params: {key: 'foo'}
},{
class: "WorkerTwo",
params: {data: 'bar'}
}]
}
SidekiqPipeline.execute(tasks)
# WorkerOne will output: 'foo'
# WorkerTwo will output: 'bar'
Tasks will be run in the order they are defined. If a task fails, it will retry on the step it failed on, and it will continue down the pipeline if/when it retires successfully. Each worker can have it's own queue or share queues. Tries to be as unobtrusive as possible
Add this line to your application's Gemfile:
gem 'sidekiq_pipeline'
And then execute:
$ bundle
Or install it yourself as:
$ gem install sidekiq_pipeline
- Fork it
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create new Pull Request