Lightning update sometimes doesn't update the article content

[rakyi]

Description

When lightning update bakes a single GDoc, it sometimes doesn't update the content.

Expected behaviour

Lightning update should always result in updated content.

Steps to reproduce

Steps to reproduce the behavior:

1. Publish a new GDoc
2. While the change is deploying, update its body
3. Click Republish button in its admin (it should have a lightning icon) and confirm the change
4. There should be two changes in the deploy queue in admin:
   • Full deploy triggerend by the publication of the article
   • Lightning update of the article
5. Check the content of the article after both changes complete - it should have the original content before update

Environment

Staging server

Additional context

AFAICT, the problem seems to be that we update the GDoc and enqueue the update in the same DB transaction, which creates a race condition.

If the deploy-queue process reads the change from the file system and starts its own transaction before the previous one commits, it will bake the GDoc with the old content. In my testing, it worked when the lighting update was the only change in the queue, but it failed when it was enqueued after a full bake that was in progress. Not sure if this makes a difference.

On prod, we trigger the bake in Buildkite instead of running it on the deploy-queue process, which is another code path I didn't verify, but presumably the race condition would be much less likely there.

[rakyi]

What we should probably do is to write custom transaction logic where we write to the deployment queue only after we commit the transaction.

[rakyi]

I tried fixing it #4449, and it seems the problem isn't in the transaction isolation after all.