Consider refactory catalog trait

[BohuTANG]

Summary

catalog trait is not clear, for example in catalog trait it has a build_table to return a dyn Table by table info and used by many places:

fn build_table(&self, table_info: &TableInfo) -> Result<Arc<dyn Table>>;

In database trait, it has get_table to return dyn Table by db_name and table_name and used by many places other:

async fn get_table(&self, db_name: &str, table_name: &str) -> Result<Arc<dyn Table>>;

Goal:

1. Provide a unitfied trait api
2. User can easily implement their own datasource under the api

[Veeupup]

agree with you ... they have a similar meaning and I get confused when first encountering these two trait methods

[BohuTANG]

Some initial thoughts:

1. Lift the database trait to catalog trait, it something as: https://github.com/datafuselabs/databend/blob/2fd5b9c175ae6212d12c5115ab189333616d3271/query/src/catalogs/catalog.rs
   How should we define the functions of catalog?

2. Rename datasources to storages, the directory layout looks like:

storages
├── csv
├── memory
├── null
├── parquet
├── github
├── system
└── fuse

In storages dir, the table engine defines:
How and where data is stored, where to write it and where read from.

[drmingdrmer]

To my understanding, there two major pieces of information for data:

• Where the data is,
• and what the format of the data is.

Take csv as an example, it actually defines both of these two.

Maybe we need to clarify these two roles from bottom up, e.g.:
CsvTable should only care about the data format. where the data is should be provided by something like a DAL.

[BohuTANG]

To my understanding, there two major pieces of information for data:

• Where the data is,
• and what the format of the data is.

Take csv as an example, it actually defines both of these two.

Maybe we need to clarify these two roles from bottom up, e.g.: CsvTable should only care about the data format. where the data is should be provided by something like a DAL.

I am trying on it:#3136
In storages, each table(CSV/Parquet/Memory etc.) is an storage engine, there is no concept of a database engine.
It looks a bit clearer now, but still needs some work, I guess it will be finished soon.