vendor: bump Pebble to e567fec84c6e #81389

jbowens · 2022-05-17T17:46:56Z

This commit includes non-trivial changes to account for the change
in the vfs.WithDiskHealthChecks function signature change.
Additionally, this PR uncovered #81413 and necessitated some changes to
work around it.

e567fec8 db: ensure Open closes opened directories on error
b8c9a560 internal/metamorphic: overwrite unused bounds buffers
7c5f0cbb db: copy user-provided bounds and optimize unchanged bounds
f8897076 *: Add IterOption to optionally read L6 filter blocks.
d79f9617 vfs: extend disk-health checking to filesystem metadata operations
5ae21746 db: remove newRangeKeyIter closure
6d975489 db: add BenchmarkIteratorScan
782d102e sstable: fix invariant check for sstable size estimation
738a7f0b db: fix NewIter regression resulting in extra memtable levels
37558663 *: Use keyspan.LevelIter for rangedels in compactions
e6c60c71 db: use sublevel level iters for all compactions out of L0
d8f63bef db: extend documentation on ingested file overlap; improve test cases
b9e970a8 internal/keyspan: correct and document MergingIter key stability
498177bb internal/keyspan: collapse fragmentBoundIterator into MergingIter

Release note (bug fix): Fix a gap in disk-stall detection. Previously,
disk stalls during filesystem metadata operations could go undetected,
inducing deadlocks. Now stalls during these types of operations will
correctly fatal the process.

cockroach-teamcity · 2022-05-17T17:47:04Z

This change is

nicktrav

Reviewed 9 of 9 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @jbowens)

jbowens

Still wrangling with two issues:

kv/kvserver.TestStoreMetrics is showing reduced bloom filter lookups.
sql/colexec.TestExternalDistinct has a goroutine leak from the disk-health checking. It appears to be file-level goroutines (not the new filesystem-wide goroutine) that are leaking, but it's a new leak with this commit. Still digging.

Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @jbowens)

nicktrav

Reviewed 4 of 4 files at r3, 1 of 1 files at r4, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @jbowens)

nicktrav

Reviewed 2 of 2 files at r5, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @jbowens)

yuzefovich · 2022-05-18T02:54:18Z

Thanks for filing #81413 - I think I have a fix in #81419, do you mind rebasing and checking whether it addresses the issue?

jbowens · 2022-05-18T13:51:24Z

Thanks for filing #81413 - I think I have a fix in #81419, do you mind rebasing and checking whether it addresses the issue?

That fixes it, thanks for the quick turnaround!

jbowens · 2022-05-18T17:32:13Z

TestLogic is pointing to another open file leak.

[TestLogic] gotest://github.com/cockroachdb/cockroach/pkg/sql/logictest#TestLogic
[12:51:58]
[TestLogic] [Test Output]
=== RUN   TestLogic
    test_log_scope.go:79: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic1197699296
    test_log_scope.go:80: use -show-logs to present logs inline
=== CONT  TestLogic
    logic.go:4414: -- test log scope end --
    logic_test.go:33: Leaked goroutine: goroutine 11937409 [select]:
        github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFile).startTicker.func1()
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/vfs/disk_health.go:72 +0xe8
        created by github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFile).startTicker
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/vfs/disk_health.go:67 +0x65
        Leaked goroutine: goroutine 11937584 [select]:
        github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFile).startTicker.func1()
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/vfs/disk_health.go:72 +0xe8
        created by github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFile).startTicker
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/vfs/disk_health.go:67 +0x65
--- FAIL: TestLogic (1937.22s)

nicktrav

Thanks for persevering with this yak shave!

Reviewed 9 of 13 files at r7, 10 of 10 files at r8, all commit messages.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @jbowens)

yuzefovich · 2022-05-19T00:09:16Z

I think I found the other problem - #81490 and will open a fix shortly.

nicktrav

Reviewed 9 of 9 files at r9, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @jbowens)

jbowens · 2022-05-19T16:42:40Z

Hrm, recent pushes aren't tickling CI

nicktrav · 2022-05-19T19:13:47Z

Isn't the lint commit already on master?

jbowens · 2022-05-19T20:41:12Z

Yes, it was a bad rebase for poking CI. This doesn't need any review until there's a green CI.

This commit includes non-trivial changes to account for the change in the `vfs.WithDiskHealthChecks` function signature change. ``` e567fec8 db: ensure Open closes opened directories on error b8c9a560 internal/metamorphic: overwrite unused bounds buffers 7c5f0cbb db: copy user-provided bounds and optimize unchanged bounds f8897076 *: Add IterOption to optionally read L6 filter blocks. d79f9617 vfs: extend disk-health checking to filesystem metadata operations 5ae21746 db: remove newRangeKeyIter closure 6d975489 db: add BenchmarkIteratorScan 782d102e sstable: fix invariant check for sstable size estimation 738a7f0b db: fix NewIter regression resulting in extra memtable levels 37558663 *: Use keyspan.LevelIter for rangedels in compactions e6c60c71 db: use sublevel level iters for all compactions out of L0 d8f63bef db: extend documentation on ingested file overlap; improve test cases b9e970a8 internal/keyspan: correct and document MergingIter key stability 498177bb internal/keyspan: collapse fragmentBoundIterator into MergingIter ``` Release note (bug fix): Fix a gap in disk-stall detection. Previously, disk stalls during filesystem metadata operations could go undetected, inducing deadlocks. Now stalls during these types of operations will correctly fatal the process.

jbowens

We're green, so I'll merge this afternoon.

Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @nicktrav and @stevendanna)

jbowens · 2022-05-20T17:16:10Z

TFTR!

bors r+

craig · 2022-05-20T18:16:21Z

Build succeeded:

GitHub CI (Cockroach)

jbowens requested a review from a team as a code owner May 17, 2022 17:46

jbowens force-pushed the jackson/pebble-master-e567fec84c6e branch from 07cfcd6 to 685e9ce Compare May 17, 2022 17:56

jbowens requested a review from a team as a code owner May 17, 2022 17:56

nicktrav approved these changes May 17, 2022

View reviewed changes

jbowens commented May 17, 2022

View reviewed changes

jbowens mentioned this pull request May 17, 2022

sql/colcontainer: TestExternalDistinct leaks Queues, file descriptors #81413

Closed

jbowens force-pushed the jackson/pebble-master-e567fec84c6e branch from 685e9ce to e38c096 Compare May 17, 2022 23:09

jbowens requested a review from a team as a code owner May 17, 2022 23:09

jbowens force-pushed the jackson/pebble-master-e567fec84c6e branch from e38c096 to 047685b Compare May 17, 2022 23:20

nicktrav approved these changes May 17, 2022

View reviewed changes

jbowens force-pushed the jackson/pebble-master-e567fec84c6e branch from 047685b to 9743844 Compare May 17, 2022 23:48

nicktrav reviewed May 17, 2022

View reviewed changes

jbowens force-pushed the jackson/pebble-master-e567fec84c6e branch from 9743844 to 804ed95 Compare May 18, 2022 15:27

jbowens requested a review from a team as a code owner May 18, 2022 15:27

jbowens force-pushed the jackson/pebble-master-e567fec84c6e branch from 804ed95 to 77acc3f Compare May 18, 2022 15:57

erikgrinaker mentioned this pull request May 18, 2022

db: standardize behavior of indexed batch mutations during iteration cockroachdb/pebble#1640

Merged

jbowens force-pushed the jackson/pebble-master-e567fec84c6e branch from 77acc3f to 52d1182 Compare May 18, 2022 20:38

jbowens requested a review from a team as a code owner May 18, 2022 20:38

jbowens force-pushed the jackson/pebble-master-e567fec84c6e branch from 52d1182 to ad606d0 Compare May 18, 2022 20:41

nicktrav approved these changes May 18, 2022

View reviewed changes

jbowens force-pushed the jackson/pebble-master-e567fec84c6e branch 2 times, most recently from 0f23bec to aec0ab0 Compare May 19, 2022 15:26

nicktrav reviewed May 19, 2022

View reviewed changes

jbowens force-pushed the jackson/pebble-master-e567fec84c6e branch from aec0ab0 to 76b37d0 Compare May 19, 2022 16:39

jbowens force-pushed the jackson/pebble-master-e567fec84c6e branch from 76b37d0 to 1c696b8 Compare May 19, 2022 19:04

jbowens requested a review from a team as a code owner May 19, 2022 19:04

jbowens requested review from stevendanna and removed request for a team May 19, 2022 19:04

jbowens force-pushed the jackson/pebble-master-e567fec84c6e branch 3 times, most recently from c62e455 to 31fa14c Compare May 19, 2022 20:39

jbowens force-pushed the jackson/pebble-master-e567fec84c6e branch from 31fa14c to 1cc2596 Compare May 19, 2022 22:56

jbowens commented May 20, 2022

View reviewed changes

craig bot merged commit 85d99af into cockroachdb:master May 20, 2022

jbowens deleted the jackson/pebble-master-e567fec84c6e branch May 20, 2022 18:26

erikgrinaker mentioned this pull request May 23, 2022

vendor: bump Pebble to b8c9a560bed5 #81300

Closed

nicktrav mentioned this pull request Jul 14, 2022

sql/logictest: use of a closed engine's FS can result in a goroutine leak; TestLogic flaky #84437

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

vendor: bump Pebble to e567fec84c6e #81389

vendor: bump Pebble to e567fec84c6e #81389

jbowens commented May 17, 2022 •

edited

Loading

cockroach-teamcity commented May 17, 2022

nicktrav left a comment

jbowens left a comment •

edited

Loading

nicktrav left a comment

nicktrav left a comment

yuzefovich commented May 18, 2022

jbowens commented May 18, 2022

jbowens commented May 18, 2022

nicktrav left a comment

yuzefovich commented May 19, 2022

nicktrav left a comment

jbowens commented May 19, 2022

nicktrav commented May 19, 2022

jbowens commented May 19, 2022

jbowens left a comment

jbowens commented May 20, 2022

craig bot commented May 20, 2022

vendor: bump Pebble to e567fec84c6e #81389

vendor: bump Pebble to e567fec84c6e #81389

Conversation

jbowens commented May 17, 2022 • edited Loading

cockroach-teamcity commented May 17, 2022

nicktrav left a comment

Choose a reason for hiding this comment

jbowens left a comment • edited Loading

Choose a reason for hiding this comment

nicktrav left a comment

Choose a reason for hiding this comment

nicktrav left a comment

Choose a reason for hiding this comment

yuzefovich commented May 18, 2022

jbowens commented May 18, 2022

jbowens commented May 18, 2022

nicktrav left a comment

Choose a reason for hiding this comment

yuzefovich commented May 19, 2022

nicktrav left a comment

Choose a reason for hiding this comment

jbowens commented May 19, 2022

nicktrav commented May 19, 2022

jbowens commented May 19, 2022

jbowens left a comment

Choose a reason for hiding this comment

jbowens commented May 20, 2022

craig bot commented May 20, 2022

jbowens commented May 17, 2022 •

edited

Loading

jbowens left a comment •

edited

Loading