Add more tests for operator-executed table functions #16057
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kasiafi
force-pushed
the
455TFOperatorTestWithOptimizations
branch
from
4d056c9 to
4db6d64
Compare
TableFunctionProcessorNode has a mapping from source symbols to marker symbols. When the plan is optimized with UnaliasSymbolReferences, some source symbols, and their corresponding marker symbols, might be recognized as semantically identical, and rewritten to the same symbols. Before this change, building the symbol-to-marker map failed in such case, because the used map collector could not resolve conflicts. After this change, duplicate entries are accepted provided that equal keys have eqal values (markers) assigned.
TableFunctionProcessorNode specifies pass-through symbols for all sources. When the plan is optimized with UnaliasSymbolReferences, some pass-through symbols from one or more sources might be recognized as semantically identical, and rewritten to the same symbol. Before this change, all the rewritten symbols remained in the pass-through lists. If duplicates occurred, it caused failure in the LocalExecutionPlanner while computing the output mapping for the TableFunctionProcessorNode.
Adds support for table functions that have table inputs. Execution by operator for table functions without sources is not yet implemented. If the input is empty, and it does not have the KEEP WHEN EMPTY property, the function is not executed, and empty result is returned. If the input is empty, and it has the KEEP WHEN EMPTY property, the function is executed upon the EmptyTableFunctionPartition It is ensured that the function with empty input is executed once per node, and that there is single node distribution in case of the KEEP WHEN EMPTY property.
Rename the existing TableFunctionProcessor to TableFunctionDataProcessor, and introduce another interface for processing splits.
This change adds the LeafTableFunctionOperator for processing table functions that do not take input (without table arguments). Splits are supported.
Add more detailed check for TableFunctionProcessorNode.passThroughSpecifications in TableFunctionProcessorMatcher.
Add check for TableFunctionProcessorNode.requiredSymbols in TableFunctionProcessorMatcher.
Add check for TableFunctionProcessorNode.hashSymbol in TableFunctionProcessorMatcher.
Adds an optimizer rule to remove TableFunctionProcessorNode with source being an empty relation, based on the "prune when empty" property.
kasiafi
force-pushed
the
455TFOperatorTestWithOptimizations
branch
from
4db6d64 to
07796f3
Compare
|
These changes were moved to #16012. I'm closing this PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Based on #15575 (operator) and #16012 (optimizations).
Adds more tests involving the execution by operator of table functions optimized by the column pruning rules.
No docs or release notes needed.