Enable querying policy-enabled table in MSQ, and use RestrictedDataSource as a base in DataSourceAnalysis. #17666
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR enables querying policy-enabled table in MSQ.
Key changed/added classes in this PR
DataSourceAnalysis
,getBaseTableDataSource
can now return the base ofRestrictedDataSource
. This is a more robust solution than using the underlying table as base.DruidQuery
, can also be created bywithPolicy
, which just applies policy restriction to the originalquery
.MSQTaskQueryMaker
would apply restrictions onDruidQuery
, instead of throw permission error.DataSourcePlan
can handleRestrictedDataSource
.RestrictedInputNumberDataSource
, which basically wraps a NumberDataSource with a policy, and itsSegmentMapFn
can be used to create aRestrictedSegment
.RunWorkOrder
, try to make a few refactors to make the code clear, no behavior change.ShufflePipelineBuilder.build()
, it was not clear before that the channel future should only be returned when the resultFuture is ready. Also, the sanity check is moved toOutputChannels
.BaseLeafFrameProcessorFactory.makeProcessors()
. Previously,makeSegmentMapFnProcessor
can return null. After this PR, it cannot. The difference is that now the SegmentMapFn from query.dataSource is handled inChainedProcessorManager
for all query (was just join query before).This PR has: