Fix creation of replicated tables when using legacy strategy and unique_key

[StevenReitsma]

Problem

Table creation fails in the situation where:

• you are not using distributed materializations
• you are using ReplicatedMergeTree table engines
• you are using cluster_mode: true
• you have defined a unique_key for a model
• you are using legacy incremental model strategy

Cause

The cause is twofold:

1. create table [table1] as [table2] on cluster [cluster] is invalid SQL for ClickHouse (at least for OSS). The on cluster [cluster] clause is not supported for create table .. as statements.
2. When removing the on_cluster_clause, table creation fails on Code: 36. DB::Exception: Received from localhost:9000. DB::Exception: Macro 'uuid' and empty arguments of ReplicatedMergeTree are supported only for ON CLUSTER queries with Atomic database engine. (BAD_ARGUMENTS).

So in this case, on cluster is both required and not supported at the same time.

Solution

This PR changes create table [table1] as [table2] to create table [table1] empty as ([sql for table1]). This lets dbt supply ClickHouse with the SQL that was used to create [table1] for [table2], instead of letting ClickHouse look at [table2] itself to copy the schema.

Tests

All tests pass locally.

[CLAassistant]

All committers have signed the CLA.

[genzgd]

Thanks for the PR. I have mixed feelings about this because ON CLUSTER is actually supported, it's just the order is wrong in that macro. CREATE TABLE <table> ON CLUSTER <cluster> AS is the right order, and I'd be inclined to let ClickHouse do this instead of a Jinja macro.

[StevenReitsma]

Ah seems like you're right, it's just the ON CLUSTER being in the wrong place. Strange that there is no mention of this in the official documentation.

However, moving the ON CLUSTER clause doesn't solve the problem yet, as we get the following error:

DB::Exception: Replica /clickhouse/tables/0/db_foo/tbl_bar/e5f54f75-b9fc-436c-a5a9-59c7f781ee40/replicas/service-dwh-0-0 already exists.

The engine for this model is set to ReplicatedMergeTree('/clickhouse/tables/{shard}/{database}/{table}/{uuid}', '{replica}'). Shouldn't ClickHouse generate a new UUID when running a CTAS instead of copying the one from the source?

[genzgd]

What you're seeing might be a result of the behavior described here: ClickHouse/ClickHouse#12135. If you are running the model repeatedly within 8 minutes the old table path might still be in zookeeper. I'm fairly certain the UUID for "CREATE TABLE AS" would be based on either a timestamp, the "different" table name, or both, so I don't think it's reusing the source table one.

It's likely we should be applying the SYNC modifier to dropping any Replicated tables.

Unfortunately this sort of behavior is one of the reasons that dbt is problematic on ClickHouse clusters.

[StevenReitsma]

You're absolutely right, not sure why I didn't catch that before. The UUID does change when doing a CTAS, but somehow the UUID was hardcoded for some of my tables. I set up a fresh instance and using CTAS with ON CLUSTER works just fine there. Of course, the clause is still in the wrong place in the query, so I've changed this PR to just move the clause to the correct position.