Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Insights page added to Cloud Dedicated #87693

Closed
maryliag opened this issue Sep 9, 2022 · 2 comments · Fixed by #88419
Closed

Insights page added to Cloud Dedicated #87693

maryliag opened this issue Sep 9, 2022 · 2 comments · Fixed by #88419
Assignees
@ericharmeling
Labels
A-sql-observability Related to observability of the SQL layer C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Comments

@maryliag
Copy link
Contributor

maryliag commented Sep 9, 2022 •
edited by cockroach-jira-scripts
Loading

Once SQL over http is available on Cloud, the Insights page should be added to Cloud Dedicated

Jira issue: CRDB-19482
@maryliag maryliag added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-sql-observability Related to observability of the SQL layer T-sql-observability labels Sep 9, 2022
@ericharmeling ericharmeling mentioned this issue Sep 21, 2022
Merged
ericharmeling added a commit to ericharmeling/cockroach that referenced this issue Sep 26, 2022
@ericharmeling
ui: add connected components for insights
b05415f 
This commit adds connected components for the workload and schema insights pages, for use
in the CC Console.

Fixes cockroachdb#87693.

Release note: None
craig bot pushed a commit that referenced this issue Sep 27, 2022
@ericharmeling @xinhaoz
@nvanbenschoten @pav-kv
Merge #88419 #88510 #88594 #88596 #88800
03b8afb 
88419: ui: add connected components for insights r=ericharmeling a=ericharmeling

This commit adds connected components for the workload and schema insights pages, for use in the CC Console.

Fixes #87693.

https://www.loom.com/share/08163a7c125948119ca71ac097099a29

Release note: None

88510: ui: move `latestQuery`, `latestFormattedQuery` from redux to local state r=xinhaoz a=xinhaoz

Previously, the fields `latestQuery` and `latestFormattedQuery` representing the latest non-empty query string for a statement viewed from the detaisl page was being stored in redux. The purpose of these fields was to preserve the query when changing tabs in the stmt details page. Saving this in the redux store was unnecessary and so this commit moves these fields to the stmt details local state.

Release note: None


https://www.loom.com/share/a28d412fb83a429391210935982404de

88594: cloud: add version gate for auth via assume role in AWS and GCP stora… r=rhu713 a=rhu713

…ge and KMS

Add a version gate for auth via assume role in AWS and GCP storage and KMS to prevent this type of auth until all nodes in the cluster has been upgraded to 22.2. The gate prevents a class of job failures where sometimes a job can succeed with assume role auth if its processors happen to all be on 22.2 nodes, but fail at times when one of its processor nodes don't support assume role. This version gate preempts the issue by preventing this type of auth until the cluster has been finalized on 22.2 and gives a better error message of why the auth cannot be used.

It's important to note that this gate does not prevent a user from creating a BACKUP job that uses assume role auth, e.g. via the DETACHED option, because the destination storage is not accessed during planning. This is inline with existing behavior for other types of auth errors, e.g. if the user enters incorrect credentials. The BACKUP job will still fail with the version gate error when it eventually executes.

Release note: None

88596: kv: use RaftAppliedIndexTerm to generate SnapshotMetadata, don't scan log r=nvanbenschoten a=nvanbenschoten

This commit replaces the call to `Term(raftAppliedIndex)` with direct use of the new `RaftAppliedIndexTerm` field (added in c3bc064) when generating a `SnapshotMetadata` in service of the `raft.Storage.Snapshot` interface. As of v22.2, this field has been fully migrated in.

First and foremost, this is a code simplification. However, it also helps with projects like #87050, where async Raft log writes make it possible for a Raft leader to apply an entry before it has been appended to the leader's own log. Such flexibility[^1] would help smooth out tail latency in any single replica's local log writes, even if that replica is the leader itself. This is an important characteristic of quorum systems that we fail to provide because of the tight coupling between the Raft leader's own log writes and the Raft leader's acknowledgment of committed proposals.

Release justification: None. Don't backport to release-22.2.

Release note: None.

[^1]: if safe, I haven't convinced myself that it is in all cases. It certainly is not for operations like non-loosely coupled log truncation.

88800: kvserver: use type-safe atomics in raftSendQueue r=nvanbenschoten a=pavelkalinnikov

Go 1.19 introduced atomic types that enforce atomic access to variables, which in many situation is less error-prone. This commit resolves a TODO to take advantage of these types.

Release note: None

Co-authored-by: Eric Harmeling <eric.harmeling@cockroachlabs.com>
Co-authored-by: Xin Hao Zhang <xzhang@cockroachlabs.com>
Co-authored-by: Rui Hu <rui@cockroachlabs.com>
Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
Co-authored-by: Pavel Kalinnikov <pavel@cockroachlabs.com>
@craig craig bot closed this as completed in 15e8355 Sep 27, 2022
@maryliag
Copy link
Contributor Author

Re-opening since it still needs to be enabled on Cloud side, after cluster-ui gets updated

@maryliag maryliag reopened this Sep 27, 2022
ericharmeling added a commit to ericharmeling/cockroach that referenced this issue Oct 18, 2022
@ericharmeling
ui: add connected components for insights
7c5c21b 
This commit adds connected components for the workload and schema insights pages, for use
in the CC Console.

Fixes cockroachdb#87693.

Release note: None
ericharmeling added a commit to ericharmeling/cockroach that referenced this issue Oct 28, 2022
@ericharmeling
ui: add connected components for insights
8f31a2c 
This commit adds connected components for the workload and schema insights pages, for use
in the CC Console.

Fixes cockroachdb#87693.

Release note: None
ericharmeling added a commit to ericharmeling/cockroach that referenced this issue Oct 28, 2022
@ericharmeling
ui: add connected components for insights
3efa4be 
This commit adds connected components for the workload and schema insights pages, for use
in the CC Console.

Fixes cockroachdb#87693.

Release note: None
@ericharmeling
Copy link
Contributor

Closed by https://github.com/cockroachlabs/managed-service/pull/10097

@ericharmeling ericharmeling mentioned this issue Dec 2, 2022
Merged
ericharmeling added a commit to ericharmeling/cockroach that referenced this issue Dec 2, 2022
@ericharmeling
ui: added common id URL parameter constant for jobs, schedules, insights
feddb8f 
This commit adds a common URL parameter constant and parameter selector
for the jobs, schedules, and insights details pages.

Part of cockroachdb#87693.

Background thread: https://cockroachlabs.slack.com/archives/G01Q9D01NTU/p1670001751479989

Release note: None
craig bot pushed a commit that referenced this issue Dec 2, 2022
@andreimatei @michae2
@ericharmeling @yuzefovich @biradarganesh25
Merge #92614 #92706 #92707 #92932 #92937 #92948
9c21578 
92614: upgrades: remove some old upgrades r=andreimatei a=andreimatei

These upgrades were needed for upgrades to old versions; they are not needed for cluster upgrading to 23.1.

Release note: None
Epic: None

92706: tracing/collector: make cluster_inflight_traces work with tenants  r=andreimatei a=andreimatei

Prior to this patch, crdb_internal.cluster_inflight_traces could not be
queried in a tenant. This patch makes it work by abstracting the trace
trace collector's dependency on node liveness for enumerating node IDs
and using instance IDs for tenants.

The patch also adds a test that queries this table; there didn't appear
to be any.

Release note: None
Epic: https://cockroachlabs.atlassian.net/browse/CRDB-14524

92707: sql/stats: guard against nil HistogramData.ColumnType when forecasting r=yuzefovich,rytaft a=michae2

Sometimes HistogramData.ColumnType can be nil. We check for this in a few places, but were not in statistics forecasting. If we run into these histograms we will consider them "missing" histograms (as if `sql.stats.histogram_collection.enabled` were false) and skip over them when predicting histograms.

Fixes: #92489

Release note (bug fix): Fix a rare panic only present in v22.2.0 that
occurs when using particular forms of old statistics in table statistics
forecasting.

This panic can also be mitigated by deleting old statistics, or by
disabling forecasting with either:

```
SET CLUSTER SETTING sql.stats.forecasts.enabled = false;
```

or, if the specific table with the problematic statistics is known:

```
ALTER TABLE t SET (sql_stats_forecasts_enabled = false);
```

92932: ui: added common id URL parameter constant for jobs, schedules, insights r=ericharmeling a=ericharmeling

This commit adds a common URL parameter constant and parameter selector for the jobs, schedules, and insights details pages.

Part of #87693.

Background thread: https://cockroachlabs.slack.com/archives/G01Q9D01NTU/p1670001751479989

Release note: None

92937: logictest: add couple more virtual lookup join tests r=yuzefovich a=yuzefovich

This commit adds a couple more queries that use left semi and left anti virtual lookup joins as well as the execbuilder ones. Additionally, it replaces tabs with spaces in the tests added in the previous commit.

Epic: None

Release note: None

92948: backupccl: cast wrapped Array datums properly when exporting to Parquet r=biradarganesh25 a=biradarganesh25

If we're encoding a DOidWrapper, then we want to cast the wrapped datum. But this is not being done when encoding Array datums when exporting to parquet.

This bug was found from the TestRandomParquetExports test. Since it randomly generates tables, reproducing this bug would be hard. So I am pasting a snippet of code that can be used to reproduced it (the table schema was copied from the test log output. Also, this snippet is incomplete, it is merely intended to document the schema which caused the test to fail. Most of the code is copied from
exportparquet_test.go):

```
		sqlDB.Exec(t, `
CREATE TABLE table1 (
        	col1_0 NAME[] NOT NULL,
        	col1_1 STRING NOT NULL,
        	col1_2 CHAR NULL,
        	col1_3 INT8 NOT NULL,
        	col1_4 STRING NOT NULL AS (lower(col1_2)) VIRTUAL,
        	col1_5 STRING NULL AS (lower(col1_2)) VIRTUAL,
        	col1_6 STRING NOT NULL AS (lower(col1_2)) STORED,
        	        	CONSTRAINT table1_pkey PRIMARY KEY (col1_4 ASC, col1_1 ASC, col1_3 ASC, col1_6 ASC, col1_0 ASC),
        	INDEX table1_col1_0_col1_3_col1_5_col1_1_col1_2_idx (col1_0 ASC, col1_3 ASC, col1_5 DESC, col1_1 ASC, col1_2 DESC) WHERE (((col1_1 = 'X':::STRING) AND (col1_5 <= e'\x00':::STRING)) OR (col1_2 > e'\'':::STRING)) OR (col1_3 > 32767:::INT8) NOT VISIBLE,
        	INVERTED INDEX table1_col1_4_col1_3_col1_0_col1_2_col1_6_col1_1_col1_5_idx (col1_4 DESC, col1_3 DESC, col1_0 ASC, col1_2 DESC, col1_6 ASC, col1_1 ASC, col1_5 gin_trgm_ops),
        	UNIQUE INDEX table1_col1_0_col1_2_col1_3_col1_1_col1_6_col1_5_key (col1_0 ASC, col1_2 ASC, col1_3 DESC, col1_1 ASC, col1_6 ASC, col1_5 DESC),
        	INDEX table1_col1_6_col1_2_col1_1_col1_4_col1_3_idx (col1_6 ASC, col1_2 ASC, col1_1 ASC, col1_4 ASC, col1_3 ASC),
        	INDEX table1_expr_col1_5_idx (lower(col1_2) DESC, col1_5 ASC),
        	FAMILY fam_0_col1_2_col1_6_col1_1 (col1_2, col1_6, col1_1),
        	FAMILY fam_1_col1_0 (col1_0),
        	FAMILY fam_2_col1_3 (col1_3)
        	);
`)
numRows, err := randgen.PopulateTableWithRandData(rng, db, "table1", 20)
// Ensure the table only contains columns supported by EXPORT Parquet. If an
// unsupported column cannot be dropped, try populating another table
tableName := "table1"
if err := func() error {
    _, cols, err := ie.QueryRowExWithCols(
        ctx,
        "",
        nil,
        sessiondata.InternalExecutorOverride{
            User:     username.RootUserName(),
            Database: dbName},
        fmt.Sprintf("SELECT * FROM %s LIMIT 1", tableName))
    require.NoError(t, err)

    for _, col := range cols {
        _, err := importer.NewParquetColumn(col.Typ, "", false)
        if err != nil {
            _, err = sqlDB.DB.ExecContext(ctx, fmt.Sprintf(`ALTER TABLE %s DROP COLUMN %s`, tableName, col.Name))
            if err != nil {
                return err
            }
        }
    }
    return nil
}(); err != nil {
    continue
}
success = true
break
}

With the table created and the data populated, if we run 'EXPORT INTO
PARQUET 'nodelocal://0/table1' FROM SELECT * FROM table1', it should throw
an error.
```

To fix this, we will cast the wrapped datum for Array datums also.

Resolves: #92199

Release note (bug fix): Fixed a bug where encoding of Array type to Parquet format would fail in some cases during `EXPORT` command.

Co-authored-by: Andrei Matei <andrei@cockroachlabs.com>
Co-authored-by: Michael Erickson <michae2@cockroachlabs.com>
Co-authored-by: Eric Harmeling <eric.harmeling@cockroachlabs.com>
Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>
Co-authored-by: Ganeshprasad Rajashekhar Biradar <ganeshprasad.biradar@cockroachlabs.com>
@ericharmeling ericharmeling mentioned this issue Jan 9, 2023
Merged
ericharmeling added a commit to ericharmeling/cockroach that referenced this issue Jan 9, 2023
@ericharmeling
ui: added common id URL parameter constant for jobs, schedules, insights
43734e1 
This commit adds a common URL parameter constant and parameter selector
for the jobs, schedules, and insights details pages.

Part of cockroachdb#87693.

Background thread: https://cockroachlabs.slack.com/archives/G01Q9D01NTU/p1670001751479989

Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ericharmeling ericharmeling

Labels
A-sql-observability Related to observability of the SQL layer C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ui: add connected components for insights ericharmeling/cockroach
2 participants
@maryliag @ericharmeling