Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: bump golang-jwt to v5 #1171

Merged
merged 1 commit into from
Sep 23, 2024

Conversation

David-Wobrock
Copy link
Contributor

@David-Wobrock David-Wobrock commented Jun 27, 2024

The rationale behind this update is that we see an ever increasing memory usage on our Oathkeeper instances, until they OOM and we start a new one.

From some pprof analysis in our prod environment, seems to indicate that it's in MutatorIDToken.Mutate that the issue lies.
More specifically when signing the token string.

  • in MutatorIDToken.Mutate => signed, err := a.r.CredentialsSigner().Sign(r.Context(), jwks, claims)
  • in Sign => signed, err := token.SignedString(key.Key)
  • in SignedString => return strings.Join([]string{sstr, sig}, "."), nil

Screenshot 2024-06-25 at 17 26 37

In golang-jwt v5, this method does not use strings.Join anymore.
See golang-jwt/jwt#115

I'm entirely sure to understand why string joining could cause this issue, perhaps some reference that Oathkeeper is keeping and therefore leads to this memory leak 🤷
But in any case, we believe it's a good thing to keep dependencies up-to-date and use the latest version.

Let me know what you think :)

Checklist

  • I have read the contributing guidelines.
  • I have referenced an issue containing the design document if my change
    introduces a new feature.
  • I am following the
    contributing code guidelines.
  • I have read the security policy.
  • I confirm that this pull request does not address a security
    vulnerability. If this pull request addresses a security vulnerability, I
    confirm that I got the approval (please contact
    security@ory.sh) from the maintainers to push
    the changes.
  • I have added tests that prove my fix is effective or that my feature
    works.
  • I have added or changed the documentation.

Further Comments

@David-Wobrock David-Wobrock force-pushed the chore/upgrade-golang-jwt-v5 branch 2 times, most recently from 4424a76 to 90dc53e Compare June 27, 2024 11:41
@David-Wobrock David-Wobrock marked this pull request as ready for review June 27, 2024 11:54
@David-Wobrock David-Wobrock requested a review from aeneasr as a code owner June 27, 2024 11:54
Copy link
Member

@aeneasr aeneasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! I don't think it will solve the mem leak issue but good to see the update.

The mem leak might come from an incorrect cache configuration for the ID token mutator.

@David-Wobrock David-Wobrock force-pushed the chore/upgrade-golang-jwt-v5 branch from 90dc53e to 700303f Compare July 10, 2024 12:48
@David-Wobrock David-Wobrock requested a review from aeneasr August 5, 2024 08:52
@David-Wobrock David-Wobrock force-pushed the chore/upgrade-golang-jwt-v5 branch from 700303f to 5d0fd3e Compare August 9, 2024 14:15
@David-Wobrock
Copy link
Contributor Author

Very nice! I don't think it will solve the mem leak issue but good to see the update.

The mem leak might come from an incorrect cache configuration for the ID token mutator.

You're right, profiling locally I have the same memory usage on master and this branch.

What obviously helps, is to disable the cache 🙄
But generally, I think that making the id_token mutator cache configurable is probably a good move.
Both enable/disable, but also configure the max cost/tokens - like the authn OAuth2 ClientCredentials and introspection handlers.

I'll try to suggest a PR that adds these options.

@aeneasr aeneasr force-pushed the chore/upgrade-golang-jwt-v5 branch from 5d0fd3e to b50a90b Compare September 20, 2024 14:23
@aeneasr aeneasr merged commit 361177a into ory:master Sep 23, 2024
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants