Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-48307][SQL][FOLLOWUP] not-inlined CTE references sibling shoul…
…d not fail ### What changes were proposed in this pull request? This is a follow-up of apache#46617 to fix a bug. When we re-construct the `WithCTE` node, we should use the new CTE definitions that have been applied `inlineCTE`. ### Why are the changes needed? bug fix, otherwise we may hit errors such as ``` java.util.NoSuchElementException: key not found: 0 at scala.collection.MapOps.default(Map.scala:289) at scala.collection.MapOps.default$(Map.scala:288) at scala.collection.AbstractMap.default(Map.scala:420) at scala.collection.mutable.HashMap.apply(HashMap.scala:440) at org.apache.spark.sql.catalyst.optimizer.PushdownPredicatesAndPruneColumnsForCTEDef$.gatherPredicatesAndAttributes(PushdownPredicatesAndPruneColumnsForCTEDef.scala:74) at org.apache.spark.sql.catalyst.optimizer.PushdownPredicatesAndPruneColumnsForCTEDef$.$anonfun$gatherPredicatesAndAttributes$1(PushdownPredicatesAndPruneColumnsForCTEDef.scala:68) at scala.collection.immutable.Vector.foreach(Vector.scala:2124) at org.apache.spark.sql.catalyst.optimizer.PushdownPredicatesAndPruneColumnsForCTEDef$.gatherPredicatesAndAttributes(PushdownPredicatesAndPruneColumnsForCTEDef.scala:67) ``` ### Does this PR introduce _any_ user-facing change? no, the bug is not released yet ### How was this patch tested? new test ### Was this patch authored or co-authored using generative AI tooling? no Closes apache#47141 from cloud-fan/fix. Lead-authored-by: Wenchen Fan <wenchen@databricks.com> Co-authored-by: Wenchen Fan <cloud0fan@gmail.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com>
- Loading branch information