Refactor all protocol related code to frontend

[v0y4g3r]

What type of enhancement is this?

Tech debt reduction

What does the enhancement do?

We've been expecting that all protocol-related logic remains in frontend, including protocol parsing and logical planning. For some reason now datanode also keeps a parser and can also accepts SQL queries. Also frontend delegates SQL parsing to datanode in standalone mode:

greptimedb/src/frontend/src/instance.rs

Line 182 in de0b8aa

sql_handler: StandaloneSqlQueryHandler::arc(dn_instance.clone()),

This compromise brings a lot of tech debt and maintenance overhead, for example, introducing a new protocol involves both frontend and datanode. Now it's time to get things right.

Steps

• remove the SQL handler and MySQL server implementation from datanode refactor: remove the SQL execution interfaces in Datanode #1135
• move the tests mod in Datanode to Frontend. They are largely sql and promql tests.
  • make "instance_test.rs" also ran with distributed instance
• move promql exec to frontend
  • also move "promql_test.rs" to frontend
• check sqlness test cases, some may move to "common"

Implementation challenges

• Move all protocol parsing stuff from datanode to frontend
  • Datanode instance should not implement SqlQueryHandler trait, it should only keep a GrpcQueryHandler implementation.
    

    greptimedb/src/datanode/src/instance/sql.rs

    Line 234 in de0b8aa

    impl SqlQueryHandler for Instance {

  • The sql_handler field of datanode::instance::Instance should be moved to frontend instance. SqlHandler also has some field like table_engine and catalog_manager, which in standalone mode, datanode should share these components to frontend so that frontend can build a SqlHandler.
    

    greptimedb/src/datanode/src/instance.rs

    Line 71 in 75b8afe

    pub(crate) sql_handler: SqlHandler,

  • Move thePromqlHandler implementation from datanode to frontend. In standalone mode it's also delegated to datanode.
    

    greptimedb/src/frontend/src/instance.rs

    Line 528 in de0b8aa

    promql_handler.do_query(query).await

• Improve our GrpcQueryHandler to accommodate more plans and data types.
  After the refactoring, datanode accepts only grpc requests with logical plans encoded as substrait (for inserts, gRPC protocol also has an Insert variant). Thus we need to support all the possible logical plans and data types in substrait.
  • Logical plan conversions, especially LogicalPlan::Extension.
    

    greptimedb/src/common/substrait/src/df_logical.rs

    Line 305 in 8ffc078

    fn logical_plan_to_rel(

  • Data type conversions:
    

    greptimedb/src/common/substrait/src/types.rs

    Line 57 in d1730a9

    pub fn to_concrete_type(ty: &SType) -> Result<(ConcreteDataType, bool)> {

[evenyag]

@MichaelScofield I create a task list to track the tasks you mentioned in #1135