Fixed race condition when updating member's last_seen_at timestamp
#20389
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
So far, have managed to fix the race condition, but may have broken a few other things — cleaning things up and adding tests now. |
cmraible
force-pushed
the
chris-eng-1240-race-condition-when-updating-members-last_seen_at-timestamp
branch
from
59fbbf6
to
e76fc01
Compare
ref https://linear.app/tryghost/issue/ENG-1240/race-condition-when-updating-members-last-seen-at-timestamp When members click a link in an email, Ghost updates the member's `last_seen_at` timestamp, but it should only update the timestamp if the member hasn't been seen in the current day (based on the publication's timezone). Currently there is a race condition present where multiple simultaneous requests from the same member (if e.g. an email link checker is following all links in an email) can cause the `last_seen_at` timestamp to be updated multiple times in the same day for the same member. These additional queries add a significant load on Ghost and its database, which can contribute to the exhaustion of the connection pool and eventually requests may time out. The primary motivation for this change is to avoid that race condition by adding a lock to the member row, checking if `last_seen_at` has already been updated in the current day, and only updating it if it hasn't. Another beneficial side-effect of this change is that it avoids locking the `labels` and `newsletters` tables, which are locked when we update the `last_seen_at` timestamp in the `members` table currently. This should improve Ghost's ability to handle a large influx of requests to redirect endpoints, which tend to happen immediately after a publisher sends an email.
cmraible
force-pushed
the
chris-eng-1240-race-condition-when-updating-members-last_seen_at-timestamp
branch
from
a13e0bb
to
614cc7d
Compare
cmraible
added a commit
that referenced
this pull request
Jun 19, 2024
#20389) ref https://linear.app/tryghost/issue/ENG-1240/race-condition-when-updating-members-last-seen-at-timestamp When members click a link in an email, Ghost updates the member's `last_seen_at` timestamp, but it should only update the timestamp if the member hasn't yet been seen in the current day (based on the publication's timezone). Currently there is a race condition present where multiple simultaneous requests from the same member (if e.g. an email link checker is following all links in an email) can cause the `last_seen_at` timestamp to be updated multiple times in the same day for the same member. These additional queries add a significant load on Ghost and its database, which can contribute to the exhaustion of the connection pool and eventually requests may time out. The primary motivation for this change is to avoid that race condition by adding a lock to the member row, checking if `last_seen_at` has already been updated in the current day, and only updating it if it hasn't. Another beneficial side-effect of this change is that it avoids locking the `labels` and `newsletters` tables, which are locked when we update the `last_seen_at` timestamp in the `members` table currently. This should improve Ghost's ability to handle a large influx of requests to redirect endpoints (confirmed with load tests), which tend to happen immediately after a publisher sends an email.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ref https://linear.app/tryghost/issue/ENG-1240/race-condition-when-updating-members-last-seen-at-timestamp
When members click a link in an email, Ghost updates the member's
last_seen_attimestamp, but it should only update the timestamp if the member hasn't yet been seen in the current day (based on the publication's timezone).Currently there is a race condition present where multiple simultaneous requests from the same member (if e.g. an email link checker is following all links in an email) can cause the
last_seen_attimestamp to be updated multiple times in the same day for the same member. These additional queries add a significant load on Ghost and its database, which can contribute to the exhaustion of the connection pool and eventually requests may time out.The primary motivation for this change is to avoid that race condition by adding a lock to the member row, checking if
last_seen_athas already been updated in the current day, and only updating it if it hasn't.Another beneficial side-effect of this change is that it avoids locking the
labelsandnewsletterstables, which are locked when we update thelast_seen_attimestamp in thememberstable currently. This should improve Ghost's ability to handle a large influx of requests to redirect endpoints (confirmed with load tests), which tend to happen immediately after a publisher sends an email.