Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql/parser: Define overload returnType interface and cleanup builtins #13209

Conversation

nvanbenschoten
Copy link
Member

@nvanbenschoten nvanbenschoten commented Jan 28, 2017

Supersedes #12644 and addresses all comments.

Fixes #12207.

This change introduces the notion of a returnTyper type-level function for
overload implementations. The purpose of this is to more clearly express cases
where overloads return types are based on their input argument types.
This was previously possible but was messy and inextensible.

The change also adjusts aggregate constructors take their parameter types.
This is useful for when aggregate behavior needs to change depending on
the input parameters. For instance, the same constructor needs to be
used for array_agg(string) and array_agg(name), but their runtime
behavior needs to be different.


This change is Reviewable

@knz
Copy link
Contributor

knz commented Jan 28, 2017

Reviewed 14 of 14 files at r1, 9 of 9 files at r2.
Review status: 15 of 17 files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.


pkg/sql/parser/overload.go, line 203 at r1 (raw file):

}

// returnType defines the method in which a functions' return type is determined.

Welcome to the world of higher-kinded types. What you're defining here is a family type-level functions (functions from types to types).

What you call a "method" is simply a function which, given the types of the arguments, returns a type for the return value.

Your "fixed" thing is a constant function -- regardless of the argument types it returns the same type.
Your "identity" thing is a projection -- a function which returns one of its arguments unchanged
your "identityarray" thing is the composition of a projection with a decorator

I'd suggest:

  • drop this interface entirely.
  • define type returnTyper func(args []TypedExpr) Type (for the name I'd probably prefer "returnTypeSelectionFunction" but it's perhaps a tad too long; for the prototype ideally it should be returnTyper(argTypes []Type) Type, but I get that it's silly to extract the []Type array where in most cases only one of them is needed)
  • define in the contract of this function type that it should return some special constant (e.g. InvalidType) when it doesn't know what to do
  • replace all occurrences of fixedReturnType{T} by fixedReturnType(T), where fixedReturnType is defined as func fixedReturnType(t Type) returnTyper { return func ... { return t } }
  • replace all occurences of identityReturnType{N} by func ... { return len(args) == 0 ? InvalidType : args[N].ResolvedType() }
  • then you have a nice and clean solution for unnest

pkg/sql/parser/overload.go, line 437 at r1 (raw file):

			// For now, we only filter on the return type for overloads with
			// fixed return types. This could be improved.
			if t, ok := o.returnType().(fixedReturnType); ok {

if o.returnTyper([]TypedExpr(nil)) != InvalidType


Comments from Reviewable

@cuongdo
Copy link
Contributor

cuongdo commented Jan 31, 2017

I just ran a Sequelize app against this branch and got:

Executing (default): SELECT i.relname AS name, ix.indisprimary AS primary, ix.indisunique AS unique, ix.indkey AS indkey, array_agg(a.attnum) as column_indexes, array_agg(a.attname) AS column_names, pg_get_indexdef(ix.indexrelid) AS definition FROM pg_class t, pg_class i, pg_index ix, pg_attribute a WHERE t.oid = ix.indrelid AND i.oid = ix.indexrelid AND a.attrelid = t.oid AND t.relkind = 'r' and t.relname = 'user' GROUP BY i.relname, ix.indexrelid, ix.indisprimary, ix.indisunique, ix.indkey ORDER BY i.relname;
Unhandled rejection TypeError: result.indkey.split is not a function
    at /Users/cdo/Dropbox/sequelize/node_modules/sequelize/lib/dialects/postgres/query.js:150:39
    at Array.forEach (native)
    at /Users/cdo/Dropbox/sequelize/node_modules/sequelize/lib/dialects/postgres/query.js:135:15
    at tryCatcher (/Users/cdo/Dropbox/sequelize/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/Users/cdo/Dropbox/sequelize/node_modules/bluebird/js/release/promise.js:507:35)
    at Promise._settlePromise (/Users/cdo/Dropbox/sequelize/node_modules/bluebird/js/release/promise.js:567:18)
    at Promise._settlePromise0 (/Users/cdo/Dropbox/sequelize/node_modules/bluebird/js/release/promise.js:612:10)
    at Promise._settlePromises (/Users/cdo/Dropbox/sequelize/node_modules/bluebird/js/release/promise.js:691:18)
    at Promise._fulfill (/Users/cdo/Dropbox/sequelize/node_modules/bluebird/js/release/promise.js:636:18)
    at PromiseArray._resolve (/Users/cdo/Dropbox/sequelize/node_modules/bluebird/js/release/promise_array.js:125:19)
    at PromiseArray._promiseFulfilled (/Users/cdo/Dropbox/sequelize/node_modules/bluebird/js/release/promise_array.js:143:14)
    at PromiseArray._iterate (/Users/cdo/Dropbox/sequelize/node_modules/bluebird/js/release/promise_array.js:113:31)
    at PromiseArray.init [as _init] (/Users/cdo/Dropbox/sequelize/node_modules/bluebird/js/release/promise_array.js:77:10)
    at Promise._settlePromise (/Users/cdo/Dropbox/sequelize/node_modules/bluebird/js/release/promise.js:564:21)
    at Promise._settlePromise0 (/Users/cdo/Dropbox/sequelize/node_modules/bluebird/js/release/promise.js:612:10)
    at Promise._settlePromises (/Users/cdo/Dropbox/sequelize/node_modules/bluebird/js/release/promise.js:691:18)

It looks like pg_index.indkey is still being returned as an int[] instead of a int2vector, and these have different encodings. The former is an array, while the latter is a space-delimited string.

This change introduces the notion of a `returnType` enum for overload
implementations. The purpose of this is to more clearly express cases
where overloads return types are based on their input argument types.
This was previously possible, but was messy and inextensible.

> returnType defines the method in which a functions' return type is determined.
> The vast majority of functions have a hard-coded return type, denoted as "fixed".
> However, some functions' return types are not static, but are instead based on
> the types of arguments provided to them.
Fixes cockroachdb#12207.

This change adjusts aggregate constructors take their parameter types.
This is useful for when aggregate behavior needs to change depending on
the input parameters. For instance, the same constructor needs to be
used for `array_agg(string)` and `array_agg(name)`, but their runtime
behavior needs to be different.
@nvanbenschoten nvanbenschoten force-pushed the nvanbenschoten/returnStyleAlt branch from 15a5e71 to 977e6d5 Compare February 6, 2017 10:22
This change unifies the builtins for `unnest`. It does so by creating a
`returnTyper` type, which is a function that replaces the `returnType`
interface. This function defines the type-level function in which a
builtin function's return type is determined. It return an out-of-band
constant when the arguments provided are not sufficient to determine a
return type, which is used during overload resolution.

In introducing `dynamicReturnType`, we can remove the previous
`fixedReturnType`, `identityReturnType` and `identityArrayReturnType`
implementations, which were holdovers from before a more general approach
to type-level function was introduced.
@nvanbenschoten nvanbenschoten force-pushed the nvanbenschoten/returnStyleAlt branch from 977e6d5 to 5132cd6 Compare February 6, 2017 11:01
@nvanbenschoten
Copy link
Member Author

@knz thanks for all the input! I've gone ahead and removed the unnecessary specializations of returnType as you suggested. I went back and forth a few times on whether I preferred leaving fixedReturnType and the interface or removing them but then requiring all other returnTypers to deal with invalid return types (unknownReturnType). In the end, I think I did exactly what you suggested. PTAL.

@cuongdo Thanks for testing with this branch. The error you got looks unrelated and implies that this change addressed the current issue. Unfortunately, it also implies that we may need to introduce an int2vector type. Some of the recent work we've done with OIDWrappers may be able to help, but I'm wondering if we'll want to do something more general for all of these "vector" types. Do you mind opening an issue for us to inspect further?

@jordanlewis
Copy link
Member

I forgot about the int2vector issue - that's addressed in the original issue #12207. We should either spin off a new issue from that or remove the Fixes annotation from this PR because there is further work to do to fully resolve it.

@knz
Copy link
Contributor

knz commented Feb 6, 2017

Awesome. The type computation stuff looks great. LGTM


Reviewed 3 of 14 files at r3, 6 of 9 files at r4, 5 of 8 files at r5.
Review status: 14 of 17 files reviewed at latest revision, 2 unresolved discussions, all commit checks successful.


Comments from Reviewable

@knz
Copy link
Contributor

knz commented Feb 6, 2017

Reviewed 1 of 8 files at r5.
Review status: 15 of 17 files reviewed at latest revision, 2 unresolved discussions, all commit checks successful.


Comments from Reviewable

@nvanbenschoten
Copy link
Member Author

TFTR!

@jordanlewis I opened #13434 per your request to continue tracking the int2vector issue.

@nvanbenschoten nvanbenschoten merged commit 715e223 into cockroachdb:master Feb 6, 2017
@nvanbenschoten nvanbenschoten deleted the nvanbenschoten/returnStyleAlt branch February 6, 2017 19:31
@jordanlewis
Copy link
Member

Thanks @nvanbenschoten!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants