Enhance trimming to be more performant #15
Conversation
app/jobs/solid_cable/trim_job.rb
Outdated
| def perform(id = nil) | ||
| id ||= ::SolidCable::Message.maximum(:id) | ||
|
|
||
| return unless (id % (trim_batch_size / 2)).zero? |
There was a problem hiding this comment.
I think it would be better to do this check before creating the job, as we'll end up with a lot of no-op jobs otherwise.
We might be better using a random value rather than id, I could imagine cases say where there are not a lot of message coming through and you have a bunch of processes getting the same value for id.
There was a problem hiding this comment.
I think it would be better to do this check before creating the job, as we'll end up with a lot of no-op jobs otherwise.
The job is running as perform_now so we shouldn't be enqueuing anything into the queue and we dont pay a penalty for no-oping.
We might be better using a random value rather than id, I could imagine cases say where there are not a lot of message coming through and you have a bunch of processes getting the same value for id
Ahh ok ill check that out
There was a problem hiding this comment.
The job is running as perform_now so we shouldn't be enqueuing anything into the queue and we dont pay a penalty for no-oping.
Oh sorry, I missed that 🙈
app/jobs/solid_cable/trim_job.rb
Outdated
|
|
||
| return unless (id % (trim_batch_size / 2)).zero? | ||
|
|
||
| ::SolidCable::Message.where( |
There was a problem hiding this comment.
This won't be valid on MySQL, you'll get something like:
You can't specify target table 'solid_cable_messages' for update in FROM clause.
I think something like this will work:
::SolidCable::Message.transaction do
ids = ::SolidCable::Message.trimmable.non_blocking_lock.limit(trim_batch_size).pluck(:id)
::SolidCable::Message.where(id: ids).delete_all
end
As an aside, it would be good to run the tests against MySQL and PostgreSQL.
There was a problem hiding this comment.
Ha yea working on updating the test suite to run through all of them.
This won't be valid on MySQL, you'll get something like:
You can't specify target table 'solid_cable_messages' for update in FROM clause.
Whoops thanks
No description provided.