Mz clusters add introspection columns #27803

jubrad · 2024-06-21T17:37:19Z

Motivation

We should show these as they may cause replicas to be redeployed on alter cluster if changed.

Tips for reviewer

Checklist

This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
This PR includes the following user-facing behavior changes:

jubrad · 2024-06-21T17:38:59Z

src/adapter/src/catalog/builtin_table_updates.rs

+                    Some(config.logging.log_logging),
+                    config.logging.interval.map(|d| {
+                        Interval::from_duration(&d)
+                            .expect("planning ensured this convertible back to interval")


I'm not sure about this, I stole it from the cluster_schedules table...

Yeah, this is fine. We create the Duration from an Interval in planning, so converting it back should always succeed.

jkosh44

LGTM, can you also update the docs:

materialize/doc/user/content/sql/system-catalog/mz_catalog.md

Lines 105 to 120 in 72a363f

    
           ### `mz_clusters` 
        
           The `mz_clusters` table contains a row for each cluster in the system. 
        
           <!-- RELATION_SPEC mz_catalog.mz_clusters --> 
        
           | Field                | Type                 | Meaning                                                                                                                                  | 
        
           |----------------------|----------------------|------------------------------------------------------------------------------------------------------------------------------------------| 
        
           | `id`                 | [`text`]             | Materialize's unique ID for the cluster.                                                                                                 | 
        
           | `name`               | [`text`]             | The name of the cluster.                                                                                                                 | 
        
           | `owner_id`           | [`text`]             | The role ID of the owner of the cluster. Corresponds to [`mz_roles.id`](/sql/system-catalog/mz_catalog/#mz_roles).                       | 
        
           | `privileges`         | [`mz_aclitem array`] | The privileges belonging to the cluster.                                                                                                 | 
        
           | `managed`            | [`boolean`]          | Whether the cluster is a [managed cluster](/sql/create-cluster/) with automatically managed replicas.                   | 
        
           | `size`               | [`text`]             | If the cluster is managed, the desired size of the cluster's replicas. `NULL` for unmanaged clusters.                                    | 
        
           | `replication_factor` | [`uint4`]            | If the cluster is managed, the desired number of replicas of the cluster. `NULL` for unmanaged clusters.                                 | 
        
           | `disk`               | [`boolean`]          | **Unstable** If the cluster is managed, `true` if the replicas have the `DISK` option . `NULL` for unmanaged clusters.                   | 
        
           | `availability_zones` | [`text list`]        | **Unstable** If the cluster is managed, the list of availability zones specified in `AVAILABILITY ZONES`. `NULL` for unmanaged clusters. |

.

You'll also need to wait for SQL council approval before merging.

src/catalog/src/builtin.rs

jkosh44 · 2024-06-21T17:42:54Z

src/adapter/src/catalog/builtin_table_updates.rs

+                    Some(config.logging.log_logging),
+                    config.logging.interval.map(|d| {
+                        Interval::from_duration(&d)
+                            .expect("planning ensured this convertible back to interval")


Yeah, this is fine. We create the Duration from an Interval in planning, so converting it back should always succeed.

benesch

One minor tweak on behalf of the SQL council. Thanks for doing this!

benesch · 2024-06-22T22:31:14Z

doc/user/content/sql/system-catalog/mz_catalog.md

+| `replication_factor`      | [`uint4`]            | If the cluster is managed, the desired number of replicas of the cluster. `NULL` for unmanaged clusters.                                 |
+| `disk`                    | [`boolean`]          | **Unstable** If the cluster is managed, `true` if the replicas have the `DISK` option . `NULL` for unmanaged clusters.                   |
+| `availability_zones`      | [`text list`]        | **Unstable** If the cluster is managed, the list of availability zones specified in `AVAILABILITY ZONES`. `NULL` for unmanaged clusters. |
+| `introspection_logging`   | [`boolean`]          | Whether introspection data gather is logged.                                                                                             |


We should match the public CREATE CLUSTER wording here (INTROSPECTION DEBUGGING), rather than the internal wording in the code.

doc/user/content/sql/system-catalog/mz_catalog.md

benesch

LGTM from a SQL council perspective. 🎉 Thanks again for knocking this one out!

doc/user/content/sql/system-catalog/mz_catalog.md

jubrad requested a review from jkosh44 June 21, 2024 17:37

jubrad requested review from a team as code owners June 21, 2024 17:37

jubrad force-pushed the mz_clusters-add-introspeection-columns branch from 6d95778 to 47a8c39 Compare June 21, 2024 17:37

jubrad commented Jun 21, 2024

View reviewed changes

jkosh44 approved these changes Jun 21, 2024

View reviewed changes

jubrad force-pushed the mz_clusters-add-introspeection-columns branch from 47a8c39 to 0efe6e1 Compare June 21, 2024 17:58

jubrad requested a review from morsapaes as a code owner June 21, 2024 17:58

jubrad force-pushed the mz_clusters-add-introspeection-columns branch from 2b36022 to 73e5f1d Compare June 21, 2024 19:25

benesch requested changes Jun 22, 2024

View reviewed changes

jubrad changed the title ~~Mz clusters add introspeection columns~~ Mz clusters add introspection columns Jun 24, 2024

jubrad force-pushed the mz_clusters-add-introspeection-columns branch from 73e5f1d to fb27998 Compare June 24, 2024 01:26

benesch reviewed Jun 24, 2024

View reviewed changes

doc/user/content/sql/system-catalog/mz_catalog.md Outdated Show resolved Hide resolved

jubrad force-pushed the mz_clusters-add-introspeection-columns branch from fb27998 to 5098f2f Compare June 24, 2024 01:58

benesch approved these changes Jun 24, 2024

View reviewed changes

doc/user/content/sql/system-catalog/mz_catalog.md Outdated Show resolved Hide resolved

jubrad force-pushed the mz_clusters-add-introspeection-columns branch from 5098f2f to 6315b82 Compare June 24, 2024 14:50

add logging and introspection fields to mz_clusters

3a8ad39

jubrad force-pushed the mz_clusters-add-introspeection-columns branch from 6315b82 to 3a8ad39 Compare June 24, 2024 14:51

jubrad merged commit 5a7ca9c into MaterializeInc:main Jun 24, 2024
80 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mz clusters add introspection columns #27803

Mz clusters add introspection columns #27803

jubrad commented Jun 21, 2024

jubrad Jun 21, 2024

jkosh44 Jun 21, 2024

jkosh44 left a comment

jkosh44 Jun 21, 2024

benesch left a comment

benesch Jun 22, 2024 •

edited

Loading

benesch left a comment

	### `mz_clusters`

	The `mz_clusters` table contains a row for each cluster in the system.

	<!-- RELATION_SPEC mz_catalog.mz_clusters -->
	\| Field \| Type \| Meaning \|
	\|----------------------\|----------------------\|------------------------------------------------------------------------------------------------------------------------------------------\|
	\| `id` \| [`text`] \| Materialize's unique ID for the cluster. \|
	\| `name` \| [`text`] \| The name of the cluster. \|
	\| `owner_id` \| [`text`] \| The role ID of the owner of the cluster. Corresponds to [`mz_roles.id`](/sql/system-catalog/mz_catalog/#mz_roles). \|
	\| `privileges` \| [`mz_aclitem array`] \| The privileges belonging to the cluster. \|
	\| `managed` \| [`boolean`] \| Whether the cluster is a [managed cluster](/sql/create-cluster/) with automatically managed replicas. \|
	\| `size` \| [`text`] \| If the cluster is managed, the desired size of the cluster's replicas. `NULL` for unmanaged clusters. \|
	\| `replication_factor` \| [`uint4`] \| If the cluster is managed, the desired number of replicas of the cluster. `NULL` for unmanaged clusters. \|
	\| `disk` \| [`boolean`] \| Unstable If the cluster is managed, `true` if the replicas have the `DISK` option . `NULL` for unmanaged clusters. \|
	\| `availability_zones` \| [`text list`] \| Unstable If the cluster is managed, the list of availability zones specified in `AVAILABILITY ZONES`. `NULL` for unmanaged clusters. \|

Mz clusters add introspection columns #27803

Mz clusters add introspection columns #27803

Conversation

jubrad commented Jun 21, 2024

Motivation

Tips for reviewer

Checklist

jubrad Jun 21, 2024

Choose a reason for hiding this comment

jkosh44 Jun 21, 2024

Choose a reason for hiding this comment

jkosh44 left a comment

Choose a reason for hiding this comment

jkosh44 Jun 21, 2024

Choose a reason for hiding this comment

benesch left a comment

Choose a reason for hiding this comment

benesch Jun 22, 2024 • edited Loading

Choose a reason for hiding this comment

benesch left a comment

Choose a reason for hiding this comment

benesch Jun 22, 2024 •

edited

Loading