Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

catalog/lease: reduce lock contention on descVersionState #111397

Merged
merged 1 commit into from
Oct 2, 2023

Conversation

fqazi
Copy link
Collaborator

@fqazi fqazi commented Sep 28, 2023

Recently, a regression was noticed on master related to AcquireByName being slower between builds. Using a profiler, this was tracked down to contention on descriptorVersionState's mutex. To address this, this patch avoids acquiring this lock twice in the hot path, which resolves the regression.

After: BenchmarkLeaseAcquireByNameCached-24 2378610 451.9 ns/op
Before: BenchmarkLeaseAcquireByNameCached-24 1493899 727.9 ns/op
f13c021: BenchmarkLeaseAcquireByNameCached-24 2129446 568.1 ns/op

Fixes: #111094

Release note: None

@fqazi fqazi requested a review from a team as a code owner September 28, 2023 00:48
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Contributor

@annrpom annrpom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 3 files at r1, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @fqazi)


pkg/sql/catalog/lease/name_cache.go line 38 at r1 (raw file):

// Resolves a (qualified) name to the descriptor's ID.
// Returns a valid descriptorVersionState for descriptor with that name,

small nit: could you update this comment to reflect your change

@annrpom annrpom requested a review from a team September 28, 2023 01:29
Recently, a regression was noticed on master related to AcquireByName
being slower between builds. Using a profiler, this was tracked down to
contention on descriptorVersionState's mutex.  To address this, this
patch avoids acquiring this lock twice in the hot path, which resolves
the regression.

After:   BenchmarkLeaseAcquireByNameCached-24    	 2378610	       451.9 ns/op
Before:  BenchmarkLeaseAcquireByNameCached-24    	 1493899	       727.9 ns/op
f13c021: BenchmarkLeaseAcquireByNameCached-24    	 2129446	       568.1 ns/op

Fixes: cockroachdb#111094

Release note: None
Copy link
Collaborator Author

@fqazi fqazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @annrpom)


pkg/sql/catalog/lease/name_cache.go line 38 at r1 (raw file):

Previously, annrpom (annie pompa) wrote…

small nit: could you update this comment to reflect your change

Done.

@fqazi fqazi requested a review from annrpom September 28, 2023 15:49
@@ -47,14 +48,15 @@ func (c *nameCache) get(
parentSchemaID descpb.ID,
name string,
timestamp hlc.Timestamp,
) *descriptorVersionState {
) (desc *descriptorVersionState, expiration hlc.Timestamp) {
c.mu.RLock()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as described in As described in #109443 (comment), RWMutex performs badly under high concurrency. Should we just change to a normal mutex?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me try this out and see how it behaves without this change.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's similar to the old one:
BenchmarkLeaseAcquireByNameCached-24 1624814 734.4 ns/op

Also, there are no writes to the name cache in this scenario, so it should hit the fast path on the RWMutex (outside of system table activity).

Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for checking!

@fqazi
Copy link
Collaborator Author

fqazi commented Oct 2, 2023

@rafiss TFTR!

bors r+

@craig
Copy link
Contributor

craig bot commented Oct 2, 2023

Build succeeded:

@craig craig bot merged commit 0be8311 into cockroachdb:master Oct 2, 2023
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

lease: regression in BenchmarkLeaseAcquireByNameCached from 23.1
4 participants