Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: add catalog.Column interface, replace catalog.TableDescriptor methods for columns #59789

Merged
merged 7 commits into from
Feb 5, 2021

Conversation

postamar
Copy link
Contributor

@postamar postamar commented Feb 4, 2021

In an effort to not directly use descpb.ColumnDescriptor protos everywhere, this multi-commit PR:

  1. introduces a new catalog.Column interface to encapsulate them, and a bunch of new methods in catalog.TableDescriptor to replace those which have descpb.ColumnDescriptor in their signatures, this is done in the first commit;
  2. replaces calls to the old methods with calls to the new methods throughout the codebase in the subsequent commits.

This breakdown into multiple commits is done to ease review, most of the substantial changes are in the first commit, the others follow through with the change and so are quite noisy.

Fixes #59805.

See also the parent issue #56306, as well as the related issue #57465 which tracks the same work I've already done for descpb.IndexDescriptor. In this PR I reused many of the same patterns I introduced in the corresponding PR #58678.

Subsequent work should consist in propagating this descpb.ColumnDescriptor -> catalog.Column change further throughout the codebase.

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@postamar postamar force-pushed the use-column-interface branch 2 times, most recently from 6c88b2e to 52705f2 Compare February 4, 2021 15:45
@postamar postamar changed the title [WIP] virtualize descpb.ColumnDescriptor in TableDescriptor sql: add catalog.Column interface, replace catalog.TableDescriptor methods for columns Feb 4, 2021
@postamar postamar marked this pull request as ready for review February 4, 2021 16:32
@postamar postamar requested a review from a team February 4, 2021 16:32
@postamar postamar requested a review from a team as a code owner February 4, 2021 16:32
@postamar postamar requested review from adityamaru and a team and removed request for a team February 4, 2021 16:32
Copy link
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm_strong: just tiny comment things.

Reviewed 10 of 10 files at r1, 10 of 10 files at r2, 36 of 36 files at r3, 22 of 22 files at r4, 25 of 25 files at r5, 48 of 48 files at r6, 70 of 70 files at r7.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @adityamaru and @postamar)


pkg/sql/opt_catalog.go, line 638 at r4 (raw file):

	}

	ot.columns = make([]cat.Column, len(cols), numCols)

this cat.Column thing is something to pay attention to.


pkg/sql/opt_exec_factory.go, line 1956 at r6 (raw file):

}

func makePublicColIdxToRetIdx(

The name of this function doesn't clearly describe to me its behavior; a comment would be helpful.


pkg/sql/catalog/descriptor.go, line 404 at r1 (raw file):

	// The ordinal of a column in a `tableDesc descpb.TableDescriptor` is
	// defined as follows:
	// - [0:len(tableDesc.Columns)] is the range of public columns,

Should we comment this in terms of methods rather than fields?


pkg/sql/catalog/descriptor.go, line 411 at r1 (raw file):

Quoted 15 lines of code…
	// Public returns true iff the column is active, i.e. readable, in the table
	// descriptor.
	Public() bool
	// WriteAndDeleteOnly returns true iff the column is a mutation in the
	// delete-and-write-only state in the table descriptor.
	WriteAndDeleteOnly() bool
	// DeleteOnly returns true iff the column is a mutation in the delete-only
	// state in the table descriptor.
	DeleteOnly() bool
	// Adding returns true iff the column is an add mutation in the table
	// descriptor.
	Adding() bool
	// Dropped returns true iff the column is a drop mutation in the table
	// descriptor.
	Dropped() bool

Are we happy with these as booleans? I know we did this on indexes but I have mixed feelings about it vs. an enum.


After having reviewed this a bit, I think these works out pretty well, 🤷


pkg/sql/catalog/tabledesc/table.go, line 466 at r1 (raw file):

// like FindPublicColumnWithName applied repeatedly to the names in the
// provided list.
func FindPublicColumnsWithNames(

Note that an error will be returned if any of the names do not exist.

Marius Posta added 7 commits February 5, 2021 09:03
Previously, the catalog.TableDescriptor interface and its implementing
types would liberally return descpb.ColumnDescriptor values, pointers and
slices. In an effort to stop manipulating such protos directly, this
patch introduces a catalog.Column interface to encapsulate it. In order
to enventually propagate this change throughout the code base, this
patch marks some existing catalog.TableDescriptor methods as deprecated
and introduces new ones to eventually replace them.

This work is very similar to that which I already performed for cockroachdb#57465.

Fixes cockroachdb#59805.

Release note: None
Previously, these methods would be called on a table descriptor interface
to apply a function on *descpb.ColumnDescriptor for some subset of columns.

This patch removes these calls, along with the method definitions, in favour
of new methods which use the catalog.Column interface type instead.

Fixes cockroachdb#59805.

Release note: None
Previously, this method would be called on a table descriptor interface to
return a target *descpb.ColumDescriptor.

This patch removes these calls, along with the method definitions, in
favour of new methods and functions which use the catalog.Column interface
type instead.

Fixes cockroachdb#59805.

Release note: None
Previously, these methods would be called on a table descriptor interface
to obtain []descpb.ColumnDescriptor values.

This patch removes these calls, along with the method definitions, in
favour of new methods which use the catalog.Column interface type instead.

GetPublicColumns() is treated in a separate commit owing to its large
number of call sites.

Fixes cockroachdb#59805.

Release note: None
Previously, the ColumnIdxMap* methods and the GetColumnAtIdx method would be
called on a table descriptor interface to return catalog.TableColMap and
corresponding descpb.ColumnDescriptor values, respectively.

This patch removes these calls, along with the method definitions, in
favour of new methods and functions which use the catalog.Column interface
type instead.

Fixes cockroachdb#59805.

Release note: None
Previously, this method would be called on a table descriptor interface
to return a []descpb.ColumnDescriptor value containing all public
columns.

This patch removes these calls, along with the method definition, in
favour of new methods which use the catalog.Column interface type instead.

Fixes cockroachdb#59805.

Release note: None
This patch renames the catalog.Column-slice-returning methods which I
recently added to catalog.TableDescriptor. I gave them provisional names
to differentiate them from the existing and very similar deprecated methods
that they replaced. Now that all deprecated methods have been removed we
can give them their definitive names.

Fixes cockroachdb#59805.

Release note: None
Copy link
Contributor Author

@postamar postamar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reviewing and for the quick turnaround. I'm going to hurry up and merge this before more conflicts arise. I already had to rebase once.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @adityamaru and @ajwerner)


pkg/sql/opt_catalog.go, line 638 at r4 (raw file):

Previously, ajwerner wrote…

this cat.Column thing is something to pay attention to.

Yes, definitely! Unifying catalog.Column and cat.Column is a logical next step. I'll update the parent issue accordingly.


pkg/sql/opt_exec_factory.go, line 1956 at r6 (raw file):

Previously, ajwerner wrote…

The name of this function doesn't clearly describe to me its behavior; a comment would be helpful.

Done.


pkg/sql/catalog/descriptor.go, line 404 at r1 (raw file):

Previously, ajwerner wrote…

Should we comment this in terms of methods rather than fields?

We do, it's at the bottom of the comment block, which follows the same structure as in catalog.Index.Ordinal. I might revise all of them once we're done introducing these new interfaces.


pkg/sql/catalog/descriptor.go, line 411 at r1 (raw file):

Previously, ajwerner wrote…
	// Public returns true iff the column is active, i.e. readable, in the table
	// descriptor.
	Public() bool
	// WriteAndDeleteOnly returns true iff the column is a mutation in the
	// delete-and-write-only state in the table descriptor.
	WriteAndDeleteOnly() bool
	// DeleteOnly returns true iff the column is a mutation in the delete-only
	// state in the table descriptor.
	DeleteOnly() bool
	// Adding returns true iff the column is an add mutation in the table
	// descriptor.
	Adding() bool
	// Dropped returns true iff the column is a drop mutation in the table
	// descriptor.
	Dropped() bool

Are we happy with these as booleans? I know we did this on indexes but I have mixed feelings about it vs. an enum.


After having reviewed this a bit, I think these works out pretty well, 🤷

Thanks but I just copied the prevailing pattern. I usually like boolean method names to begin with Is or Has anyway.


pkg/sql/catalog/tabledesc/table.go, line 466 at r1 (raw file):

Previously, ajwerner wrote…

Note that an error will be returned if any of the names do not exist.

Good point. I updated the comment.

@postamar
Copy link
Contributor Author

postamar commented Feb 5, 2021

bors r+

@craig
Copy link
Contributor

craig bot commented Feb 5, 2021

Build succeeded:

@craig craig bot merged commit 4516f63 into cockroachdb:master Feb 5, 2021
@postamar postamar deleted the use-column-interface branch February 5, 2021 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sql: stop returning raw protobufs for ColumnDescriptor API getters
3 participants