[Feature][Clickhouse-Connector-V2] Add Clickhouse e2e testcase #2954
Conversation
|
Please solve the problem of CI. please look at https://github.com/apache/incubator-seatunnel/blob/dev/docs/en/contribution/setup.md |
| </parent> | ||
| <modelVersion>4.0.0</modelVersion> | ||
|
|
||
| <artifactId>connector-clickhouse-spark-e2e</artifactId> |
There was a problem hiding this comment.
You can use it test spark/flink with once code. Not only spark.
There was a problem hiding this comment.
@FWLamb seatunnel-{enigne}-connector-v2-e2e modules are old implementation, which requires e2e testing for each engine (spark 3.3 and seatunnel engine will be added later), there will be a lot of duplicate code, which is boring and tiring.
seatunnel-connector-v2-e2e unifies the e2e of the connector-v2 and can support the e2e test of multiple engines at the same time.
The status quo is a legacy issue, I will upgrade to a new implementation as soon as possible.
| String initializeSourceTableSql = "CREATE TABLE default.source_table (" + | ||
| " `name` Nullable(String)," + | ||
| " `age` Nullable(Int32)" + | ||
| ")ENGINE = Memory"; | ||
| String initializeSinkTableSql = "CREATE TABLE default.sink_table (" + | ||
| " `name` Nullable(String)," + | ||
| " `age` Nullable(Int32)" + | ||
| ")ENGINE = Memory"; |
There was a problem hiding this comment.
| sql { | ||
| sql = "select name,age from source_table" | ||
| } |
There was a problem hiding this comment.
remove?
test all supported fields
| @Test | ||
| public void testFakeSourceToClickhouseSink() throws IOException, InterruptedException { | ||
| Container.ExecResult execResult = executeSeaTunnelSparkJob("/clickhouse/fake_to_clickhouse.conf"); | ||
| Assertions.assertEquals(0, execResult.getExitCode()); | ||
| } | ||
|
|
||
| @Test | ||
| public void testClickhouseSourceToClickhouseSink() throws IOException, InterruptedException { | ||
| Container.ExecResult execResult = executeSeaTunnelSparkJob("/clickhouse/clickhouse_to_clickhouse.conf"); | ||
| Assertions.assertEquals(0, execResult.getExitCode()); | ||
| Assertions.assertIterableEquals(generateTestDataset(), queryResult()); | ||
| } |
There was a problem hiding this comment.
| @Test | |
| public void testFakeSourceToClickhouseSink() throws IOException, InterruptedException { | |
| Container.ExecResult execResult = executeSeaTunnelSparkJob("/clickhouse/fake_to_clickhouse.conf"); | |
| Assertions.assertEquals(0, execResult.getExitCode()); | |
| } | |
| @Test | |
| public void testClickhouseSourceToClickhouseSink() throws IOException, InterruptedException { | |
| Container.ExecResult execResult = executeSeaTunnelSparkJob("/clickhouse/clickhouse_to_clickhouse.conf"); | |
| Assertions.assertEquals(0, execResult.getExitCode()); | |
| Assertions.assertIterableEquals(generateTestDataset(), queryResult()); | |
| } | |
| @Test | |
| public void testClickhouseSourceToClickhouseSink() throws IOException, InterruptedException { | |
| Container.ExecResult execResult = executeSeaTunnelSparkJob("/clickhouse/fake_to_clickhouse.conf"); | |
| Assertions.assertEquals(0, execResult.getExitCode()); | |
| Container.ExecResult execResult = executeSeaTunnelSparkJob("/clickhouse/clickhouse_to_clickhouse.conf"); | |
| Assertions.assertEquals(0, execResult.getExitCode()); | |
| Assertions.assertIterableEquals(generateTestDataset(), queryResult()); | |
| } |
merge together
| fields { | ||
| name = "string" | ||
| age = "int" | ||
| } |
| </parent> | ||
| <modelVersion>4.0.0</modelVersion> | ||
|
|
||
| <artifactId>connector-clickhouse-spark-e2e</artifactId> |
There was a problem hiding this comment.
@FWLamb seatunnel-{enigne}-connector-v2-e2e modules are old implementation, which requires e2e testing for each engine (spark 3.3 and seatunnel engine will be added later), there will be a lot of duplicate code, which is boring and tiring.
seatunnel-connector-v2-e2e unifies the e2e of the connector-v2 and can support the e2e test of multiple engines at the same time.
The status quo is a legacy issue, I will upgrade to a new implementation as soon as possible.
|
|
||
|
|
||
| @BeforeEach | ||
| @SuppressWarnings("checkstyle:MagicNumber") |
There was a problem hiding this comment.
| @SuppressWarnings("checkstyle:MagicNumber") |
For classes suffixed with IT or Test, it is automatically suppressed.
Purpose of this pull request
Check list
New License Guide