You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Gave up most changes in #743 because 1) mixed APIs are confusing; 2) risky to upgrade.
This pull request is to add a brand new JDBC driver com.clickhouse.jdbc.ClickHouseDriver with minimum changes to legacy code, for better backward compatibility and visibility of what's coming next.
TODOs:
primitive array support
new connection string
support both jdbc:clickhouse: and jdbc:ch: as prefix
protocol defaults to http, so jdbc:ch://localhost is same as jdbc:ch:http://localhost
format: RowBinaryWithNamesAndTypes
new implementation under com.clickhouse.jdbc
move SQL parser to com.clickhouse.jdbc.parser
implement clickhouse-http-client and make it default provider for the new driver
new and enhanced JDBC driver built on top of clickhouse-client:
enhance parser to support standard synchronous UPDATE and DELETE statements
update a set c1=1 where c2=2 -> alter table a update c1=1 where c2=2 settings mutations_sync=1
delete from a -> truncate table a
delete from a where c1=1 -> alter table a delete where c1=1 settings mutations_sync=1
support timestamp with timezone
support query with custom format(e.g. select * from system.query_log limit 5 format JSONEachRow) - only works for text format
improve prepared statement
* ternary operator support - select 1 ? 'a' : 'b', 2 ? (select 1) : 2, ? only contains one parameter(the last ?)
* type inferring from input function and maybe magic comments?
* parameter templating based on first value?
shared test cases for clickhouse-grpc-client and clickhouse-http-client
better support of AggregateFunction
optimize Bitmap support by implementing ClickHouseValue interface and use InputStream/OutputStream instead of in-memory byte array
make Gson optional
Update:
performance test on my laptop
50 concurrent users, each issue query select * from numbers(10000) for 10,000 times; connection pool size is 50, select 1 is used to validate connection.
Driver
Samples
Average
Median
90% Line
95% Line
99% Line
Min
Max
Error %
Throughput
Received KB/sec
0.3.2-SNAPSHOT(1st time)
500000
32
26
63
79
129
3
1151
0
1515.15152
72349.96449
clickhouse4j 1.4.4
500000
45
37
91
104
149
4
726
0
1067.12653
50956.33378
0.3.1-patch
500000
45
37
89
101
167
5
854
0
1067.42269
50970.47569
Native JDBC 2.5.6
500000
46
40
75
97
163
4
1494
0
1050.2238
50149.21219
0.3.2-SNAPSHOT(2nd time)
500000
33
27
66
84
135
3
2143
0
1469.60706
70175.17211
Note: JMeter was used for the testing(close and re-open for each run).
Key features of the new JDBC driver are ready and being tested in local for weeks. It's time to get the pull request merged into develop for creating test builds.
I'll create a separate PR to complete below features:
basic JDBC escape syntax support
enhance prepared statement - 1) multiple values for batch insertion; 2) leverage JDBC escape syntax and external table for non-insert query(in a similar way as input function based approach)
shared test cases for all type of clients
better support of AggregateFunction(and further optimization to bitmap)
implement unwrap methods to expose Java client API for JDBC users
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gave up most changes in #743 because 1) mixed APIs are confusing; 2) risky to upgrade.
This pull request is to add a brand new JDBC driver
com.clickhouse.jdbc.ClickHouseDriver
with minimum changes to legacy code, for better backward compatibility and visibility of what's coming next.TODOs:
jdbc:clickhouse:
andjdbc:ch:
as prefixhttp
, sojdbc:ch://localhost
is same asjdbc:ch:http://localhost
format
: RowBinaryWithNamesAndTypescom.clickhouse.jdbc
com.clickhouse.jdbc.parser
clickhouse-http-client
and make it default provider for the new driverclickhouse-client
:clickhouse-grpc-client
) for JDBC driverDatabaseMetaData.getIndexInfo
DatabaseMetaData.getSchemas
log_comment
(in system.query_log)custom_http_headers
: "User-Agent=New Client, X-Forwarded-For=1\,2"custom_http_params
: "max_execution_time=30, mutations_sync=2"jdbc_compliant
option to enable:update a set c1=1 where c2=2
->alter table a update c1=1 where c2=2 settings mutations_sync=1
delete from a
->truncate table a
delete from a where c1=1
->alter table a delete where c1=1 settings mutations_sync=1
select * from system.query_log limit 5 format JSONEachRow
) - only works for text format* ternary operator support -
select 1 ? 'a' : 'b', 2 ? (select 1) : 2, ?
only contains one parameter(the last?
)* type inferring from input function and maybe magic comments?
* parameter templating based on first value?
clickhouse-grpc-client
andclickhouse-http-client
AggregateFunction
ClickHouseValue
interface and use InputStream/OutputStream instead of in-memory byte arrayUpdate:
performance test on my laptop
50 concurrent users, each issue query
select * from numbers(10000)
for 10,000 times; connection pool size is 50,select 1
is used to validate connection.Note: JMeter was used for the testing(close and re-open for each run).