Active Job

Active Job is the Rails standard interface for interacting with job runners. Active Job can be configured to work with Sidekiq.

Active Job Setup

The Active Job adapter must be set to :sidekiq or else it will use the default value provided by Rails, which is :async. This can be done in config/application.rb:

class Application < Rails::Application
  # ...
  config.active_job.queue_adapter = :sidekiq
end

We can use the generator to create a new job.

rails generate job Example

This above command will create app/jobs/example_job.rb

class ExampleJob < ActiveJob::Base
  # Set the Queue as Default
  queue_as :default

  def perform(*args)
    # Perform Job
  end
end

Usage

Jobs can be added to the job queue from anywhere. We can add a job to the queue by:

ExampleJob.perform_later args

At this point, Sidekiq will run the job for us. If the job for some reason fails, Sidekiq will retry as normal.

Note: This can be confusing when comparing Sidekiq and ActiveJob documentation, as ActiveJob does not provide a retry mechanism on its own, but failed ActiveJob jobs will retry.

Queues

Sidekiq requires you to define the queues to process when it starts. You do this by listing the queue names in config/sidekiq.yml or using the -q argument in order of priority: bundle exec sidekiq -q critical -q high -q default -q low

Before Rails 6.1, different types of jobs were put in different queues. Unfortunately this leads to an increase in queues that need to be enabled in Sidekiq with each Rails version upgrade or these jobs will silently sit in Redis and not execute. I've always recommended using just a few prioritized queues like low, default, high and critical. Thus I recommend disabling the standard Rails queues; instead let all jobs go to default or prioritize them:

# config/application.rb
module Myapp
  class Application < Rails::Application
    config.load_defaults "6.0"

    # nil will use the "default" queue
    # some of these options will not work with your Rails version
    # add/remove as necessary
    config.action_mailer.deliver_later_queue_name = nil # defaults to "mailers"
    config.action_mailbox.queues.routing    = nil       # defaults to "action_mailbox_routing"
    config.active_storage.queues.analysis   = nil       # defaults to "active_storage_analysis"
    config.active_storage.queues.purge      = nil       # defaults to "active_storage_purge"
    config.active_storage.queues.mirror     = nil       # defaults to "active_storage_mirror"
    # config.active_storage.queues.purge    = :low      # alternatively, put purge jobs in the `low` queue
  end
end

Since Rails 6.1 you don't need to worry about this.

Customizing error handling

Before Rails 6.0.1, Active Job does not support the full richness of Sidekiq's retry feature out of the box. Instead, it has a simpler abstraction for encoding retries upon encountering specific exceptions.

class ExampleJob < ActiveJob::Base
  retry_on ErrorLoadingSite, wait: 5.minutes, queue: :low_priority 
  def perform(*args)
    # Perform Job
  end
end

The default Active Job retry scheme—when using retry_on—is 5 retries, 3 seconds apart. Once this is done (after 15-30 seconds), Active Job will kick the job back to Sidekiq, where Sidekiq's retries with exponential backoff will take over.

As of Sidekiq 6.0.1 you can use sidekiq_options with your Rails 6.0.1+ Active Jobs and configure the standard Sidekiq retry mechanism.

class ExampleJob < ActiveJob::Base
  sidekiq_options retry: 5

  def perform(*args)
    # Perform Job
  end
end

Sidekiq supports sidekiq_retries_exhausted and sidekiq_retry_in blocks on an ActiveJob job as of 7.1.3.

Action Mailer

Action Mailer now comes with a method named #deliver_later which will send emails asynchronously (your emails send in a background job). As long as Active Job is setup to use Sidekiq we can use #deliver_later. Unlike Sidekiq, using Active Job will serialize any activerecord instance with Global ID. Later the instance will be deserialized.

Mailers are queued in the queue mailers before Rails 6.1. Remember to start sidekiq processing that queue:

bundle exec sidekiq -q default -q mailers

To send a basic message to the Job Queue we can use:

UserMailer.welcome_email(@user).deliver_later

If you would like to bypass the job queue and perform the job synchronously you can use:

UserMailer.welcome_email(@user).deliver_now

With Sidekiq we had the option to send emails with a set delay. We can do this through Active Job as well.

Old syntax for delayed message in Sidekiq:

UserMailer.delay_for(1.hour).welcome_email(@user.id)
UserMailer.delay_until(5.days.from_now).welcome_email(@user.id)

New syntax to send delayed message through Active Job:

UserMailer.welcome_email(@user).deliver_later(wait: 1.hour)
UserMailer.welcome_email(@user).deliver_later(wait_until: 5.days.from_now)

Using Global ID

Rails's Global ID feature allows passing ActiveRecord models as arguments to #perform, so that

def perform(user_id)
  user = User.find(user_id)
  user.send_welcome_email!
end

can be replaced with:

def perform(user)
  user.send_welcome_email!
end

Unfortunately, this means that if the User record is deleted after the job is enqueued but before the perform method is called, exception handling is different. With regular Sidekiq, you could handle this with

def perform(user_id)
  user = User.find_by(id: user_id)

  if user
    user.send_welcome_email!
  else
    # handle a deleted user record
  end
end

With Active Job, the perform(user) will instead raise for a missing record exception as part of deserializing the User instance.

You can work around this with

class MyJob < ActiveJob::Base
  rescue_from ActiveJob::DeserializationError do |exception|
    # handle a deleted user record
  end

  # ...
end

Job ID

Active Job has its own Job ID which is not used by Sidekiq. You can get Sidekiq's JID by using provider_job_id:

job = SomeJob.perform_later
jid = job.provider_job_id

Performance

See https://gist.github.com/mperham/42307b8b135cd546ed68550e9af8a631 for a Rails 8.0 benchmark test between Sidekiq and Solid Queue. Active Job adds about 30% overhead versus Sidekiq's native Sidekiq::Job API.

Sidekiq is roughly 15x faster to enqueue jobs and 15x faster to execute jobs than Solid Queue due to transactional overhead.

Time to execute 500,000 no-op jobs:

System	Time
Sidekiq/native	19.0
Sidekiq/AJ	25.6
Solid Queue/AJ	293

Queue Prefixes

Active Job allows you to configure a queue prefix. Don't use environment-specific prefixes. Each environment should use a separate Redis database altogether, otherwise all of your environments will share the same retry and scheduled sets and chaos will likely ensue.

Commercial Features

A few Sidekiq Pro and Sidekiq Enterprise features will break in unpredictable ways if you try to use Active Job with those features. For instance, creating Active Jobs within a Batch will work in the base case but fail if you use Active Job's retry mechanism in those jobs.