Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: narrow down use of SingleDel to avoid anomalies #69923

Merged
merged 1 commit into from
Sep 10, 2021

Conversation

tbg
Copy link
Member

@tbg tbg commented Sep 8, 2021

Fixes #69414.

Only use SingleDel if

  • the meta tracking allows it (i.e. the sole previous condition), and
  • epoch zero, and
  • no ignored txn seqno ranges.

These three together imply that the engine history of the metadata key
is a single Set, so we can safely use SingleDel.

Release justification: fix for a release blocker
Release note: None

@tbg tbg requested a review from sumeerbhola September 8, 2021 13:36
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@tbg tbg force-pushed the storage-singledel branch 2 times, most recently from 50960e6 to 3b12b44 Compare September 8, 2021 14:57
Copy link
Collaborator

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

nit: why is this "below-Raft" given this is being done by the leaseholder when evaluating the proposal?

Reviewed 5 of 5 files at r1, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @tbg)


pkg/storage/mvcc.go, line 2957 at r1 (raw file):

// to change with #69891):
//
// - Set (Set' Del) Set'' SingleDel

nit: are the parentheses supposed to represent what was in a single compaction?
If so, the example is not quite accurate. We won't compact (Set' Del) into nothing. The Del will live because there is an older Set.
A bad sequence (which happens even if the first Del is a SingleDel) is
Set Set' (Del Set'') SingleDel
Set Set' (Set'' SingleDel)
Set Set'


pkg/storage/mvcc.go, line 2971 at r1 (raw file):

// https://github.com/cockroachdb/cockroach/issues/69891
type singleDelOptimizationHelper struct {
	_didNotUpdateMeta *bool

why underscore prefixes in the names?


pkg/storage/mvcc.go, line 2982 at r1 (raw file):

	}
	// TODO(sumeerbhole): changing this to `return true` does not
	// yield any test failures. This seems problematic.

I didn't understand this comment. We currently don't populate this optional field unless it is true, but that is just an implementation detail.


pkg/storage/mvcc.go, line 2996 at r1 (raw file):

}

// onClearIntent returns true if the SingleDel optimization is available

nit: how about calling thisonAbortIntent with a code comment saying that this should not be mistaken to mean txn abort, since it only applies to the intent? onClearIntent seems a bit confusing since we clear the intent even in the case of committing (intentDemuxWriter.ClearIntent)

Fixes cockroachdb#69414.

Only use SingleDel if

- the meta tracking allows it (i.e. the sole previous condition), and
- epoch zero, and
- no ignored txn seqno ranges.

These three together imply that the engine history of the metadata key
is a single `Set`, so we can safely use `SingleDel`.

Release justification: fix for a release blocker
Release note: None
Copy link
Member Author

@tbg tbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: why is this "below-Raft" given this is being done by the leaseholder when evaluating the proposal?

🙈 it's not, fixed, thank you.

RFAL.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @sumeerbhola)


pkg/storage/mvcc.go, line 2971 at r1 (raw file):

Previously, sumeerbhola wrote…

why underscore prefixes in the names?

It's my attempt to make it less likely that they will be accessed instead of the getters by accident.


pkg/storage/mvcc.go, line 2982 at r1 (raw file):

Previously, sumeerbhola wrote…

I didn't understand this comment. We currently don't populate this optional field unless it is true, but that is just an implementation detail.

Ah, that makes sense, thank you for clarifying.

@tbg tbg marked this pull request as ready for review September 10, 2021 10:20
@tbg tbg requested a review from a team as a code owner September 10, 2021 10:20
Copy link
Collaborator

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 1 of 1 files at r3, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @sumeerbhola and @tbg)


-- commits, line 13 at r3:
btw, I'd feel better if we also fix the compatibility issue with 21.1 before beta #69891 (comment) (I realize we don't expect to upgrade from 21.1 to 21.2 beta for serverless, so it isn't strictly necessary, but remembering why partial fixes are ok make my head hurt).


pkg/storage/mvcc.go, line 2937 at r3 (raw file):

//   BEGIN;
//     SAVEPOINT foo;
//       INSERT INTO kv VALUES(1, 1);

nit: can you add back the UPDATE in the example that was there in your previous version. That is more consistent with the use of Del below.

@tbg
Copy link
Member Author

tbg commented Sep 10, 2021

bors r=sumeerbhola
TFTR!
Going to leave the comments as is since I don't have time for another CI turnaround tonight. I expect there'll be another chance to touch these up with progress on #69891 (comment).

@craig
Copy link
Contributor

craig bot commented Sep 10, 2021

Build succeeded:

@craig craig bot merged commit 773ec30 into cockroachdb:master Sep 10, 2021
@tbg tbg deleted the storage-singledel branch September 13, 2021 07:00
@tbg
Copy link
Member Author

tbg commented Sep 13, 2021

blathers backport 21.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

roachtest: tpccbench/nodes=9/cpu=4/multi-region failed [replica inconsistency]
3 participants