Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gen4: subquery refactor #8974

Merged
merged 19 commits into from
Oct 13, 2021
Merged

Conversation

frouioui
Copy link
Member

@frouioui frouioui commented Oct 11, 2021

Description

This pull request contains a big refactor of the subqueries system in Gen4. We have introduced a new AST struct ExtractedSubquery that replaces entire subquery expression and its parents if it is a comparison or an exists (i.e: (select 1), a = (select 1), exists (select 1), a in (select 1), etc). Now, instead of directly rewriting the subquery with its arguments (i.e: :__sq1, :__sq_has_values1, etc), we rewrite it with an ExtractedSubquery. If in the later planning phase we need to rewrite the subquery to its original form we can directly use ExtractedSubquery's Original field, and if we wish to keep the subquery as an argument, ExtractedSubquery's format function will do it.

Related Issue(s)

Checklist

  • Should this PR be backported?
  • Tests were added or are not required
  • Documentation was added or is not required

@frouioui frouioui added Type: Enhancement Logical improvement (somewhere between a bug and feature) Component: Query Serving release notes none labels Oct 11, 2021
@frouioui frouioui force-pushed the gen4-subquery-refactor branch from a91c309 to 2a7114d Compare October 11, 2021 05:59
@systay systay force-pushed the gen4-subquery-refactor branch 2 times, most recently from c96d2bc to 7a6f71d Compare October 11, 2021 09:12
@frouioui frouioui force-pushed the gen4-subquery-refactor branch 2 times, most recently from 7f1f398 to 4937db9 Compare October 11, 2021 12:03
frouioui and others added 12 commits October 12, 2021 09:20
…ning and its understanding

Co-authored-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
…None

Co-authored-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
We should be able to handle calculating dependencies even if
the expression is a slice and not valid as a map key.

Co-authored-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
@frouioui frouioui force-pushed the gen4-subquery-refactor branch from 395edb6 to dc15f61 Compare October 12, 2021 07:20
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
…es done on SelStmt

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
@frouioui frouioui marked this pull request as ready for review October 12, 2021 09:41
@frouioui frouioui requested a review from systay as a code owner October 12, 2021 09:41
@@ -1720,3 +1720,33 @@ func (node *RenameTable) Format(buf *TrackedBuffer) {
prefix = ", "
}
}

func (node *ExtractedSubquery) Format(buf *TrackedBuffer) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing comment, and we should not create the new expressions here, we should create them when we create this struct, I think

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as the new struct is also an SQLNode, we can keep the Format function here.

Original Expr // original expression that was replaced by this ExtractedSubquery
ArgName string
HasValuesArg string
OpCode int // this should really be engine.PulloutOpCode, but we cannot depend on engine :(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we move the opcode here then? or some other common package?

go/vt/sqlparser/ast.go Show resolved Hide resolved
go/vt/vtgate/planbuilder/abstract/operator.go Show resolved Hide resolved
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
@@ -1720,3 +1720,33 @@ func (node *RenameTable) Format(buf *TrackedBuffer) {
prefix = ", "
}
}

func (node *ExtractedSubquery) Format(buf *TrackedBuffer) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as the new struct is also an SQLNode, we can keep the Format function here.

@systay systay merged commit 90c2edb into vitessio:main Oct 13, 2021
@systay systay deleted the gen4-subquery-refactor branch October 13, 2021 05:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Query Serving Type: Enhancement Logical improvement (somewhere between a bug and feature)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants