Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
54041: bulkio: Count corrupt schedules. r=miretskiy a=miretskiy

Differentiate between "bad schedules" -- schedules
we could not execute, and corrupt ones (schedules
we couldn't even parse).

Release Notes: None
Release Justification: No impact metrics change.

54639: rowexec: paired joiners to accomplish left joins r=sumeerbhola a=sumeerbhola

The paired joiners are used to accomplish left {outer,semi,anti}
joins when the first joiner will produce false positives (as
known to the optimizer).
Currently, only the invertedJoiner can function as this first
joiner but there is wording in the spec describing how this
could also be useful for join expressions that can't be fully
evaluated on one non-inverted index.

The first joiner outputs an additional bool column representing
a continuation value. This is used to demarcate groups of
consecutive rows output by the first joiner that represent
the same original left row. The first joiner needs to preserve
input row order (the invertedJoiner always does this). The
second joiner is a lookup join and handles these groups in
a special manner. The second join does not need to be order
preserving.

Informs #53576

Prior to this, the way to do:
- a left outer join with an inverted join is to
  do an inner join with the same ON condition, which is a pair
  of inverted join and lookup join, and then wrap the expression
  in another left join with the original left side.
- a left anti join with an inverted join is to map it to a left
  outer join (previous bullet).
- a left semi join is to map it to an inner join with the same
  ON condition, which is a pair of inverted join and lookup
  join, and then project, sort (or make the lookup join order
  preserving) and dedup.

We expect that the alternative outlined in this PR (it excludes
the optimizer changes) will be more efficient since it is
simply a pairing of inverted joiner and lookup join (and the
latter does not need to be order preserving).

Release note: None

54655: backupccl: add opt-in last-backup-time metric r=dt a=dt

This adds a metric for the time captured by the last completed backup run by a schedule
designated as updating this metric. This can be used to specify a schedule as maintaining
your RPO (say you hourly cluster backup) and thus have a timeseries metric tracking that RPO
which can then be monitored.

Note: if an operator wants granular tracking of separate schedules to monitor the RPO of
one database or another, they should use SHOW against those specific schedules or some other
means, as we do not want to add an unbounded number of user-defined timeseries to our metrics.
Instead, this adds only one metric, and the operator can decide, which, if any, schedules
completing will update that metric.

Fixes #54412.

Release note (sql change): Add an option to scheduled backups to maintain a timeseries metric for last backed up timestamp.

54760: docs: Update BACKUP diagram r=lnhsingh a=lnhsingh

Excluding BACKUP INTO from the syntax diagram for now.

Release justification: low-risk update to SQL diagram for docs
Release notes: none

54774: roachprod: Add support for io2 SSD for AWS r=miretskiy a=miretskiy

Support io2 SSDs for aws.
Support aws-image-ami override.

Release Notes: None
Release Justification: No user changes; N/A

54853: cli: fix default value in help text of max-disk-temp-storage flag r=knz a=DuskEagle

Previously, in the output of `cockroach start --help`, the
`--max-disk-temp-storage` flag section reported that the default value
for the flag was "0 B".

```
$ cockroach start --help
...
      --max-disk-temp-storage bytes
...
          (default 0 B)
```

This is inaccurate, as the actual default value is dependent on whether
the store is a disk store or an in-memory store.

To fix this, this patch changes the BytesValue.String() method when the
underlying value is nil. Previously, it would return "0 B", which caused
our flags library to think the value was a non-zero value. By changing
the method to return `"<nil>"`, our flags library recognizes the value as
a zero value, and does not print an incorrect default value for the flag
in our help text.

Release note (cli change): The --help text for --max-disk-temp-storage
now properly reports the default value.

54952: backupccl: update default backup telemetry source to ".manual" r=dt a=pbardea

This commit updates the telemetry for backup to always include the
source (either ".manual" or ".scheduled"). This allows for folks to
search by a prefix to get the telemetry for all backups, and the
specific source can be appened to filter the telemetry between manual
and scheduled backups.

Release note: None

Co-authored-by: Yevgeniy Miretskiy <yevgeniy@cockroachlabs.com>
Co-authored-by: sumeerbhola <sumeer@cockroachlabs.com>
Co-authored-by: David Taylor <tinystatemachine@gmail.com>
Co-authored-by: Lauren <lauren@cockroachlabs.com>
Co-authored-by: Joel Kenny <joel@cockroachlabs.com>
Co-authored-by: Paul Bardea <pbardea@gmail.com>
  • Loading branch information
7 people committed Sep 29, 2020
8 parents b245a96 + 182a9c8 + 74f83be + 11f8a84 + 5f034d2 + 1b5fd57 + 92cc140 + e3f714b commit cfcc947
Show file tree
Hide file tree
Showing 30 changed files with 1,382 additions and 567 deletions.
20 changes: 1 addition & 19 deletions docs/generated/sql/bnf/backup.bnf
Original file line number Diff line number Diff line change
@@ -1,23 +1,5 @@
backup_stmt ::=
'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' subdirectory 'IN' destination 'AS OF SYSTEM TIME' timestamp 'WITH' backup_options_list
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' subdirectory 'IN' destination 'AS OF SYSTEM TIME' timestamp 'WITH' 'OPTIONS' '(' backup_options_list ')'
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' subdirectory 'IN' destination 'AS OF SYSTEM TIME' timestamp
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' subdirectory 'IN' destination 'WITH' backup_options_list
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' subdirectory 'IN' destination 'WITH' 'OPTIONS' '(' backup_options_list ')'
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' subdirectory 'IN' destination
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' destination 'AS OF SYSTEM TIME' timestamp 'WITH' backup_options_list
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' destination 'AS OF SYSTEM TIME' timestamp 'WITH' 'OPTIONS' '(' backup_options_list ')'
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' destination 'AS OF SYSTEM TIME' timestamp
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' destination 'WITH' backup_options_list
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' destination 'WITH' 'OPTIONS' '(' backup_options_list ')'
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' destination
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' 'LATEST' 'IN' destination 'AS OF SYSTEM TIME' timestamp 'WITH' backup_options_list
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' 'LATEST' 'IN' destination 'AS OF SYSTEM TIME' timestamp 'WITH' 'OPTIONS' '(' backup_options_list ')'
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' 'LATEST' 'IN' destination 'AS OF SYSTEM TIME' timestamp
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' 'LATEST' 'IN' destination 'WITH' backup_options_list
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' 'LATEST' 'IN' destination 'WITH' 'OPTIONS' '(' backup_options_list ')'
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'INTO' 'LATEST' 'IN' destination
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'TO' destination 'AS OF SYSTEM TIME' timestamp 'INCREMENTAL FROM' full_backup_location ( | ',' incremental_backup_location ( ',' incremental_backup_location )* ) 'WITH' backup_options_list
'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'TO' destination 'AS OF SYSTEM TIME' timestamp 'INCREMENTAL FROM' full_backup_location ( | ',' incremental_backup_location ( ',' incremental_backup_location )* ) 'WITH' backup_options_list
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'TO' destination 'AS OF SYSTEM TIME' timestamp 'INCREMENTAL FROM' full_backup_location ( | ',' incremental_backup_location ( ',' incremental_backup_location )* ) 'WITH' 'OPTIONS' '(' backup_options_list ')'
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'TO' destination 'AS OF SYSTEM TIME' timestamp 'INCREMENTAL FROM' full_backup_location ( | ',' incremental_backup_location ( ',' incremental_backup_location )* )
| 'BACKUP' ( ( 'TABLE' | ) table_pattern ( ( ',' table_pattern ) )* | 'DATABASE' database_name ( ( ',' database_name ) )* ) 'TO' destination 'AS OF SYSTEM TIME' timestamp 'WITH' backup_options_list
Expand Down
Loading

0 comments on commit cfcc947

Please sign in to comment.