Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
…db#88718 cockroachdb#88724 cockroachdb#88730

88582: vendor: bump Pebble to 829c25fa5db3 r=nicktrav a=jbowens

```
829c25fa db: fix RangeKeyChanged and -WithLimit interaction
0f464e9c sstable: fix interaction between bpf and monotonic bounds optimization
46a6a539 sstable: remove commented code
9a552270 db: fix TestRangeKeyMaskingRandomized
8418ebc2 build: bump min go version to 1.17
5f8eb821 *: remove references to `ioutil`
05a4b29c db: expand iter_histories test coverage
2855ba7c db: refactor TestRangeKeys into TestIterHistories
```

Release note: None

88615: bench: add a benchmark for update with assigment casts r=yuzefovich a=yuzefovich

Informs: cockroachdb#88525.

Release note: None

88673: stats: fix missing autostats on cluster startup r=msirek a=msirek

Fixes cockroachdb#87247

Previously, automatic statistics may fail to be collected on tables
with no stats at cluster startup when the
`sql.stats.automatic_collection.enabled` cluster setting is false
but table storage parameter `sql_stats_automatic_collection_enabled`
is true.

Method `ensureAllTables` does not put entries into the settingOverrides
map, and since storage parameter info is not normally looked up in the
table header in auto stats processing, we fall back on the cluster
setting to determine if auto stats should be collected.

To address this, this patch modifies auto stats to flag whether the
current batch of tables saved in the mutationCounts map comes from
cluster startup processing, and if so, ensures that storage
parameters controlling auto stats are always looked up in the table
header during that processing.

Release note (bug fix): This patch fixes missing automatic statistics
collection at cluster startup when the
`sql.stats.automatic_collection.enabled` cluster setting is false,
but there are tables with storage parameter
`sql_stats_automatic_collection_enabled` set to true.


88718: storage: add log scoping to MVCC benchmarks r=erikgrinaker a=jbowens

This prevents logs from being interleaved within benchmark output when using `--stream-output` with `dev bench`. Since the Pebble logs are chatty, this might have a non-neglible impact on the benchmark performance. I'll backport this to prior releases so that comparisons between the HEADs of the various release branches are comparing apples to apples.

Release note: None

88724: ui: fix ui on dbconsole table detail page with very long index name r=amyyq2 a=amyyq2

Previously, when there was a very long index name, it would run off the edge of the summary card and cause the Index Stats table to scroll for a long time. This change wraps the index name text so the user can see the entirety of the name on the summary card and in the table. This change also fixed the width of the Index Stats table to be aligned with the other summary cards on the page.

<img width="1203" alt="cluster-ui" src="https://user-images.githubusercontent.com/54999459/192319099-71c7734d-3ed1-4ee0-b6ee-568bd46f0dba.png">

Partially Fixes cockroachdb#86559

Release justification: Category 2: Bug fixes and
low-risk updates to new functionality

Release note: None

88730: cli: deprecate `debug unsafe-remove-dead-replicas` r=aliher1911 a=erikgrinaker

This patch marks `unsafe-remove-dead-replicas` as deprecated, and refers users to the new `recover` commands. It will be removed in v23.1.

Resolves cockroachdb#86543.

Release note (cli change): The `debug unsafe-remove-dead-replicas` CLI command has been deprecated, and will be removed in v23.1. Users should use the new `debug recover` set of commands instead.

Co-authored-by: Jackson Owens <jackson@cockroachlabs.com>
Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>
Co-authored-by: Mark Sirek <sirek@cockroachlabs.com>
Co-authored-by: Amy Qian <amy.qian@cockroachlabs.com>
Co-authored-by: Erik Grinaker <grinaker@cockroachlabs.com>
  • Loading branch information
6 people committed Sep 26, 2022
7 parents 18b15cf + c2ab3a6 + 6dab55f + b2e0f43 + fd42024 + e565d16 + e71924e commit 089d9c0
Show file tree
Hide file tree
Showing 11 changed files with 152 additions and 15 deletions.
6 changes: 3 additions & 3 deletions DEPS.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -1425,10 +1425,10 @@ def go_deps():
patches = [
"@com_github_cockroachdb_cockroach//build/patches:com_github_cockroachdb_pebble.patch",
],
sha256 = "d6484d702e7a30ce907ff29a472f69ede077a2bde3bdca10cfd6edac6889aec2",
strip_prefix = "github.com/cockroachdb/pebble@v0.0.0-20220921152447-63300403d537",
sha256 = "0bbd362c80a97b0c3b37df1729a524dd4c977621a37e0f22f1533d842b5767dc",
strip_prefix = "github.com/cockroachdb/pebble@v0.0.0-20220923162449-829c25fa5db3",
urls = [
"https://storage.googleapis.com/cockroach-godeps/gomod/github.com/cockroachdb/pebble/com_github_cockroachdb_pebble-v0.0.0-20220921152447-63300403d537.zip",
"https://storage.googleapis.com/cockroach-godeps/gomod/github.com/cockroachdb/pebble/com_github_cockroachdb_pebble-v0.0.0-20220923162449-829c25fa5db3.zip",
],
)
go_repository(
Expand Down
2 changes: 1 addition & 1 deletion build/bazelutil/distdir_files.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ DISTDIR_FILES = {
"https://storage.googleapis.com/cockroach-godeps/gomod/github.com/cockroachdb/go-test-teamcity/com_github_cockroachdb_go_test_teamcity-v0.0.0-20191211140407-cff980ad0a55.zip": "bac30148e525b79d004da84d16453ddd2d5cd20528e9187f1d7dac708335674b",
"https://storage.googleapis.com/cockroach-godeps/gomod/github.com/cockroachdb/gostdlib/com_github_cockroachdb_gostdlib-v1.13.0.zip": "b3d43d8f95edf65f73a5348f29e1159823cac64b148f8d3bb48340bf55d70872",
"https://storage.googleapis.com/cockroach-godeps/gomod/github.com/cockroachdb/logtags/com_github_cockroachdb_logtags-v0.0.0-20211118104740-dabe8e521a4f.zip": "1972c3f171f118add3fd9e64bcea6cbb9959a3b7fa0ada308e8a7310813fea74",
"https://storage.googleapis.com/cockroach-godeps/gomod/github.com/cockroachdb/pebble/com_github_cockroachdb_pebble-v0.0.0-20220921152447-63300403d537.zip": "d6484d702e7a30ce907ff29a472f69ede077a2bde3bdca10cfd6edac6889aec2",
"https://storage.googleapis.com/cockroach-godeps/gomod/github.com/cockroachdb/pebble/com_github_cockroachdb_pebble-v0.0.0-20220923162449-829c25fa5db3.zip": "0bbd362c80a97b0c3b37df1729a524dd4c977621a37e0f22f1533d842b5767dc",
"https://storage.googleapis.com/cockroach-godeps/gomod/github.com/cockroachdb/redact/com_github_cockroachdb_redact-v1.1.3.zip": "7778b1e4485e4f17f35e5e592d87eb99c29e173ac9507801d000ad76dd0c261e",
"https://storage.googleapis.com/cockroach-godeps/gomod/github.com/cockroachdb/returncheck/com_github_cockroachdb_returncheck-v0.0.0-20200612231554-92cdbca611dd.zip": "ce92ba4352deec995b1f2eecf16eba7f5d51f5aa245a1c362dfe24c83d31f82b",
"https://storage.googleapis.com/cockroach-godeps/gomod/github.com/cockroachdb/sentry-go/com_github_cockroachdb_sentry_go-v0.6.1-cockroachdb.2.zip": "fbb2207d02aecfdd411b1357efe1192dbb827959e36b7cab7491731ac55935c9",
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ require (
github.com/cockroachdb/go-test-teamcity v0.0.0-20191211140407-cff980ad0a55
github.com/cockroachdb/gostdlib v1.13.0
github.com/cockroachdb/logtags v0.0.0-20211118104740-dabe8e521a4f
github.com/cockroachdb/pebble v0.0.0-20220921152447-63300403d537
github.com/cockroachdb/pebble v0.0.0-20220923162449-829c25fa5db3
github.com/cockroachdb/redact v1.1.3
github.com/cockroachdb/returncheck v0.0.0-20200612231554-92cdbca611dd
github.com/cockroachdb/stress v0.0.0-20220803192808-1806698b1b7b
Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -470,8 +470,8 @@ github.com/cockroachdb/gostdlib v1.13.0/go.mod h1:eXX95p9QDrYwJfJ6AgeN9QnRa/lqqi
github.com/cockroachdb/logtags v0.0.0-20190617123548-eb05cc24525f/go.mod h1:i/u985jwjWRlyHXQbwatDASoW0RMlZ/3i9yJHE2xLkI=
github.com/cockroachdb/logtags v0.0.0-20211118104740-dabe8e521a4f h1:6jduT9Hfc0njg5jJ1DdKCFPdMBrp/mdZfCpa5h+WM74=
github.com/cockroachdb/logtags v0.0.0-20211118104740-dabe8e521a4f/go.mod h1:Vz9DsVWQQhf3vs21MhPMZpMGSht7O/2vFW2xusFUVOs=
github.com/cockroachdb/pebble v0.0.0-20220921152447-63300403d537 h1:XsUYWzuCkaLfXwH14+p2UrdIZ4nw+sncFw4ZyKr1aNc=
github.com/cockroachdb/pebble v0.0.0-20220921152447-63300403d537/go.mod h1:890yq1fUb9b6dGNwssgeUO5vQV9qfXnCPxAJhBQfXw0=
github.com/cockroachdb/pebble v0.0.0-20220923162449-829c25fa5db3 h1:NVo2LilFFcw4mkeeAkJyxXzc7cxIU3iAD8mVkUtbbUc=
github.com/cockroachdb/pebble v0.0.0-20220923162449-829c25fa5db3/go.mod h1:kLR1opoVvCCE4czHe/oymuca481JU4p2sFMlGr2+xq4=
github.com/cockroachdb/redact v1.0.8/go.mod h1:BVNblN9mBWFyMyqK1k3AAiSxhvhfK2oOZZ2lK+dpvRg=
github.com/cockroachdb/redact v1.1.3 h1:AKZds10rFSIj7qADf0g46UixK8NNLwWTNdCIGS5wfSQ=
github.com/cockroachdb/redact v1.1.3/go.mod h1:BVNblN9mBWFyMyqK1k3AAiSxhvhfK2oOZZ2lK+dpvRg=
Expand Down
39 changes: 39 additions & 0 deletions pkg/bench/bench_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@ func BenchmarkSQL(b *testing.B) {
runBenchmarkInsertSecondaryIndex,
runBenchmarkTrackChoices,
runBenchmarkUpdate,
runBenchmarkUpdateWithAssignmentCast,
runBenchmarkUpsert,
} {
fnName := runtime.FuncForPC(reflect.ValueOf(runFn).Pointer()).Name()
Expand Down Expand Up @@ -572,6 +573,44 @@ func runBenchmarkUpdate(b *testing.B, db *sqlutils.SQLRunner, count int) {
b.StopTimer()
}

// runBenchmarkUpdateWithAssignmentCast benchmarks updating count random rows in
// a table where we need to perform an assigment cast to get the updated values.
func runBenchmarkUpdateWithAssignmentCast(b *testing.B, db *sqlutils.SQLRunner, count int) {
defer func() {
db.Exec(b, `DROP TABLE IF EXISTS bench.update`)
}()

const rows = 10000
db.Exec(b, `CREATE TABLE bench.update (k INT PRIMARY KEY, v INT)`)

var buf bytes.Buffer
buf.WriteString(`INSERT INTO bench.update VALUES `)
for i := 0; i < rows; i++ {
if i > 0 {
buf.WriteString(", ")
}
fmt.Fprintf(&buf, "(%d, %d)", i, i)
}
db.Exec(b, buf.String())

s := rand.New(rand.NewSource(5432))

b.ResetTimer()
for i := 0; i < b.N; i++ {
buf.Reset()
buf.WriteString(`UPDATE bench.update SET v = (v + 1.0)::FLOAT WHERE k IN (`)
for j := 0; j < count; j++ {
if j > 0 {
buf.WriteString(", ")
}
fmt.Fprintf(&buf, `%d`, s.Intn(rows))
}
buf.WriteString(`)`)
db.Exec(b, buf.String())
}
b.StopTimer()
}

// runBenchmarkUpsert benchmarks upserting count rows in a table.
func runBenchmarkUpsert(b *testing.B, db *sqlutils.SQLRunner, count int) {
defer func() {
Expand Down
4 changes: 3 additions & 1 deletion pkg/cli/debug.go
Original file line number Diff line number Diff line change
Expand Up @@ -1104,8 +1104,10 @@ func runDebugSyncBench(cmd *cobra.Command, args []string) error {

var debugUnsafeRemoveDeadReplicasCmd = &cobra.Command{
Use: "unsafe-remove-dead-replicas --dead-store-ids=[store ID,...] [path]",
Short: "Unsafely attempt to recover a range that has lost quorum",
Short: "Unsafely attempt to recover a range that has lost quorum (deprecated)",
Long: `
DEPRECATED: use 'debug recover' instead. unsafe-remove-dead-replicas will be
removed in CockroachDB v23.1.
This command is UNSAFE and should only be used with the supervision of
a Cockroach Labs engineer. It is a last-resort option to recover data
Expand Down
8 changes: 7 additions & 1 deletion pkg/sql/stats/automatic_stats.go
Original file line number Diff line number Diff line change
Expand Up @@ -388,14 +388,20 @@ func (r *Refresher) Start(
// once on startup.
const initialTableCollectionDelay = time.Second
initialTableCollection := time.After(initialTableCollectionDelay)
var ensuringAllTables bool

for {
select {
case <-initialTableCollection:
r.ensureAllTables(ctx, &r.st.SV, initialTableCollectionDelay)
if len(r.mutationCounts) > 0 {
ensuringAllTables = true
}

case <-timer.C:
mutationCounts := r.mutationCounts
refreshingAllTables := ensuringAllTables
ensuringAllTables = false

var settingOverrides map[descpb.ID]catpb.AutoStatsSettings
// For each mutation count, look up auto stats setting overrides using
Expand Down Expand Up @@ -436,7 +442,7 @@ func (r *Refresher) Start(
// processing the current table longer than the refresh
// interval, look up the table descriptor to ensure we don't
// have stale table settings.
if elapsed > DefaultRefreshInterval {
if elapsed > DefaultRefreshInterval || refreshingAllTables {
desc = r.getTableDescriptor(ctx, tableID)
if desc != nil {
if !r.autoStatsEnabled(desc) {
Expand Down
59 changes: 59 additions & 0 deletions pkg/sql/stats/automatic_stats_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -582,6 +582,65 @@ func TestAutoStatsReadOnlyTables(t *testing.T) {
})
}

func TestAutoStatsOnStartupClusterSettingOff(t *testing.T) {
defer leaktest.AfterTest(t)()
defer log.Scope(t).Close(t)
ctx := context.Background()

s, sqlDB, kvDB := serverutils.StartServer(t, base.TestServerArgs{})
defer sqlDB.Close()
defer s.Stopper().Stop(ctx)

st := cluster.MakeTestingClusterSettings()
AutomaticStatisticsClusterMode.Override(ctx, &st.SV, false)
evalCtx := eval.NewTestingEvalContext(st)
defer evalCtx.Stop(ctx)

sqlRun := sqlutils.MakeSQLRunner(sqlDB)
sqlRun.Exec(t,
`CREATE DATABASE t;
CREATE TABLE t.a (k INT PRIMARY KEY);
ALTER TABLE t.a SET (sql_stats_automatic_collection_enabled = true);
CREATE TABLE t.b (k INT PRIMARY KEY);
ALTER TABLE t.b SET (sql_stats_automatic_collection_enabled = false);
CREATE TABLE t.c (k INT PRIMARY KEY);`)

executor := s.InternalExecutor().(sqlutil.InternalExecutor)
cache := NewTableStatisticsCache(
10, /* cacheSize */
kvDB,
executor,
s.ClusterSettings(),
s.CollectionFactory().(*descs.CollectionFactory),
)
require.NoError(t, cache.Start(ctx, keys.SystemSQLCodec, s.RangeFeedFactory().(*rangefeed.Factory)))
refresher := MakeRefresher(s.AmbientCtx(), st, executor, cache, time.Microsecond /* asOfTime */)

// Refresher start should trigger stats collection on t.a.
if err := refresher.Start(
ctx, s.Stopper(), time.Millisecond, /* refreshInterval */
); err != nil {
t.Fatal(err)
}

// There should be one stat for table t.a.
sqlRun.CheckQueryResultsRetry(t,
`SELECT statistics_name, column_names, row_count FROM [SHOW STATISTICS FOR TABLE t.a]`,
[][]string{
{"__auto__", "{k}", "0"},
})

// There should be no stats for table t.b.
sqlRun.CheckQueryResultsRetry(t,
`SELECT statistics_name, column_names, row_count FROM [SHOW STATISTICS FOR TABLE t.b]`,
[][]string{})

// There should be no stats for table t.c.
sqlRun.CheckQueryResultsRetry(t,
`SELECT statistics_name, column_names, row_count FROM [SHOW STATISTICS FOR TABLE t.c]`,
[][]string{})
}

func TestNoRetryOnFailure(t *testing.T) {
defer leaktest.AfterTest(t)()
defer log.Scope(t).Close(t)
Expand Down
Loading

0 comments on commit 089d9c0

Please sign in to comment.