Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Downgrade 'Closing scope' error to a warning to prevent hammering sentry #13552

Closed
wants to merge 2 commits into from

Conversation

anoadragon453
Copy link
Member

We noticed an error repeatedly coming up in Sentry after deploying e91a929 to matrix.org yesterday: https://sentry.tools.element.io/organizations/element/issues/725/ (internal link for those with access).

In summary, we're seeing hundreds of thousands of the following on incoming GET /_matrix/federation/v1/event/{eventId} requests:

Closing scope %s which is not the currently-active one %s
<synapse.logging.scopecontextmanager._LogContextScope object at 0x7f181f60e4f0>

@squahtx proposed that this was due to changes in #13489.

I imagine the messed up logging contexts have been there for a long time and it's only complaining extra hard now because of the new spans.

This PR reduces the error to a warning so that Sentry data is usable again. We should still separately fix the issue.

@anoadragon453 anoadragon453 marked this pull request as ready for review August 18, 2022 10:58
@anoadragon453 anoadragon453 requested a review from a team as a code owner August 18, 2022 10:58
Copy link
Contributor

@babolivier babolivier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm as long as we commit to properly fixing it in the not too distant future.

@squahtx
Copy link
Contributor

squahtx commented Aug 18, 2022

May be related to #13366 or #10855.

@richvdh
Copy link
Member

richvdh commented Aug 18, 2022

oh man, I'm not very keen on this. can't we find the actual problem rather than just ignoring it?

@richvdh
Copy link
Member

richvdh commented Aug 18, 2022

We've investigated further and found that #13499 is the cause of the problems here. We're investigating a fix, but otherwise we'll just revert #13499 for now.

@richvdh richvdh closed this Aug 18, 2022
@MadLittleMods MadLittleMods added T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. A-Metrics metrics, measures, stuff we put in Prometheus labels Aug 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A-Metrics metrics, measures, stuff we put in Prometheus T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants