Tracking: support select ... from @<stage>

[youngsofun]

Summary

Features

• basic select from parquet
• select ... from @<stage> ( <select_stage_options> )
  • feat: select from stage with files/pattern. #9877
• distributed query
  • list file in read_partitions refactor: ParquetTable list file in read_partition. #9871
• Feature: table function infer_schema #9908
• Feature: select from <uri> #9974
• doc docs(stage): stage table function and user defined file format. #10208
• options
  • file_format

Bugs

• bug: explain estimated rows is zero when select * from stage #9884

Related

• create format clause, so we can use format name in <select_stage_options>

Optional

• error tolerant (not necessary if not used in copy)
• select from json(low priority)
• select from CSV / TSV(low priority)

[youngsofun]

@BohuTANG @sundy-li @Xuanwo
do we need a desc @stage/path or infer_schema @stage/path like desc table ?
not know which columns can be used.

[youngsofun]

is select ... from '<uri>' necessary?
need to design a syntax to pass connection = (...)

[BohuTANG]

@BohuTANG @sundy-li @Xuanwo do we need a desc @stage/path or infer_schema @stage/path like desc table ? not know which columns can be used.

Better to have it, we can use it as:

1. For the cloud, we can infer the parquet schema
2. For debugging the schema

[BohuTANG]

is select ... from '<uri>' necessary? need to design a syntax to pass connection = (...)

Good idea, it is still better to have, but this task can be considered as low priority.

[BohuTANG]

@youngsofun Add INFER_SCHEMA(..) to this tracking, really need it.

[youngsofun]

since select from stage looks like a table function ( with @stage_name as function name) .
should we put the doc under table-functions too?

[BohuTANG]

since select from stage looks like a table function ( with @stage_name as function name) . should we put the doc under table-functions too?

Ok