Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lightning updates triggering full rebakes #3107

Closed
mlbrgl opened this issue Jan 17, 2024 · 4 comments
Closed

Lightning updates triggering full rebakes #3107

mlbrgl opened this issue Jan 17, 2024 · 4 comments

Comments

@mlbrgl
Copy link
Member

mlbrgl commented Jan 17, 2024

Description

Lightning updates are currently triggering full rebakes.

Expected behaviour

Lightning updates should trigger partial/lightning rebakes. They are correctly identified on the front-end (see ⚡️ icon in "Republish button), but the check seems to fail on the backend.

Steps to reproduce

Steps to reproduce the behavior:

  1. Create a test Gdoc and publish it
  2. Change its content only
  3. Confirm there is a ⚡️ in the Republish button
  4. ⚡️ Republish
  5. Check the content of .queue (or .pending depending on how fast you are)

The "slug" key - a marker of a lightning deploy - is missing, e.g.:

{"timeISOString":"2024-01-17T14:26:58.166Z","authorName":"Admin User","authorEmail":"admin@example.com","message":"Updating test-article"}

@ikesau
Copy link
Member

ikesau commented Feb 6, 2024

Probably something wrong in checkIsLightningUpdate with a Date/string coercion problem.

This time when we fix it, create unit tests to make it easier to catch if/when it breaks again.

@rakyi
Copy link
Contributor

rakyi commented Aug 27, 2024

We tested that the lightning updates get correctly put in the queue file. Needs to check that this bakes atomically.

@rakyi
Copy link
Contributor

rakyi commented Aug 28, 2024

I also verified this works on staging, but only sometimes. AFAICT, the problem seems to be that we update the GDoc and enqueue the update in the same DB transaction, which creates a race condition.

If the deploy-queue process reads the change from the file system and starts its own transaction before the previous one commits, it will bake the GDoc with the old content. In my testing, it worked when the lighting update was the only change in the queue, but it failed when it was enqueued after a full bake that was in progress. Not sure if this makes a difference.

On prod, we trigger the bake in Buildkite instead of running it on the deploy-queue process, which is another code path I didn't verify, but presumably the race condition would be much less likely there.

@rakyi
Copy link
Contributor

rakyi commented Aug 28, 2024

Closing this since we confirmed lightning updates don't trigger full rebakes. Opened #3908 for the new issue.

@rakyi rakyi closed this as completed Aug 28, 2024
@larsyencken larsyencken closed this as not planned Won't fix, can't repro, duplicate, stale Oct 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants