For more information, resources, documentation of Kunlun distributed RDBMS, please visit www.kunlunbase.com Download KunlunBase and use it for free at http://downloads.kunlunbase.com/ This document isn't maintained anymore, for latest release notes, see http://doc.kunlunbase.com/Release_notes.html
Completed adaption and development of essential distrbuted query optimization and execution of advanced SQL features against multiple tables and/or partitions scattered on multiple storage shards, including aggregation, multi-table join, sub-queries, window functions, stored procedures, etc. Also, fixed over 50 bugs for features such as distributed query processing, sequences, distributed DDL execution&replication, etc.
Starting from this version, Kunlun can reliably process all supported SQL queries including all supported DDL and DML, can do distributed transaction processing and distributed query optimization and execution. So it's great time start trying it out and do some POC testing.
We built our Kunlun Test Suite(KTS) using PostgreSQL's original test suite, removing test cases around features that are not supported in Kunlun, such as foreign keys, triggers, check constraints, object database features(table/type inheritance, creating table from composite types, etc). And we also added many extra SQL tests into KTS to test all newly added features exhaustively.
Now Kunlun database has passed most of KTS test cases. There are about 50 bugs that are not fixed yet, which either belong to trivial/corner features or features scheduled for development in future versions. They will be fixed in the near future. All bug reports can be found in bugs.zettadb.com
10 pg_dump: 0 is outside the valid range for parameter
13 drop schema cascade fails when the schema has tables
14 Alter a cloumn type fails when the column has an index on it
16 Can not create index on text column
17 The command does not return the number of affected records in psql
21 Crashcase: select attrelid::regclass, attname, attnum from pg_attribute where attname = 'a' and (attrelid = 'mlparted'::regclass or attrelid = 'mlparted1'::regclass);
25 Create table with 'CONCURRENTLY' fails
28 Explain update on view fails
29 No target columns found in update statement UPDATE
30 Parser Bug: syntax error in select from view
31 ERROR: attribute number 2 exceeds number of columns 1
32 Bad output for table join results
33 Crash when releasing a savepoint
34 Primary index for partitioned table is not found
36 Warnings for relcache reference leak
39 Problem in alloc set ExecutorState: detected write past chunk end
41 Bad results for aggregates with subquery
42 rollback a readonly transaction make the progress crash
43 Lateral expression crash in table join queries
48 Bugs for processing the query using current_user
51 Errors for insert into select from
53 Unable to serialize where clause expression in statement UPDATE
54 The features should not be supported, but there is no error when creating/using it.
55 Misleading info when select postgres system column from a table
56 Record is not forbidden to insert into the wrong partitioned subtable
57 Fail to create temp table inside an explicit transaction
60 errors when explain select from a view
61 alter table change schema errors when it is current schema
64 Crash when alter table with 'SET GENERATED BY DEFAULT'
65 Bad results for column with 'GENERATED'
66 error in partition join query
67 Crash in partition join when SET enable_partitionwise_join to true
71 Add timestamp column fails when using now() as the default value
72 ERROR: can't specify non default null ordering for a column of a remote relation's index
74 Copy command is not supported yet
78 Better message when the type is not supported
81 Bad processing for join query
83 Crash when insert into using values of a function call
84 INTERSECT query result error
85 cluster_mgr mysql header dependence
90 Unexpected statements exist in ddl log
92 An implicit sequence may not be stored where its owner table is stored
110 Select from a complex view fails
111 Misleading info when executing 'create table of type'
112 create view with local check option does not produce error/warning output, but it does not really work
114 Update crash when involving CURRENT_USER and SESSION_USER
116 Using numeric without precision/scale will drop the digits after point
121 A constraint can not be added again after it is deleted
126 Update/delete statements sent to storage node is not correct when using returning clause
127 Some DDL statements are not logged into ddl_ops_log_clust1
Improved overall stability and performance of kunlun computing node.
Query processing performance improvement.
In this release we completed the following features:
Made PostgreSQL native sequence feature work with Kunlun architecture. A sequence metadata is stored in storage shard meta tables(kunlun_sysdb.sequences), and no matter it's created in which computing nodes, all computing nodes will have the sequence locally and be able to use it efficiently.
All PostgreSQL sequence features are supported including explicit create and use of sequences as well as implicit measures such as serial type, the 'generated as identity' option in 'create table' statement. Also implemented sequence grammar of Oracle and autoincrement grammar of MySQL.
Now we support all DDLs that we plan to support in Kunlun RDBMS, including "ALTER TABLE/INDEX/SEQUENCE statements", among others. Unsupported grammers will return error gracefully. So now DDLs are complete in Kunlun.
This is crucial for security and query processing performance.
The functionality is working now, but it needs extra optimization work for premium performance in certain cases, will be done in the 0.8 milestone.
The functionality is working now, but it needs extra optimization work for premium performance in certain cases, will be done in the 0.8 milestone.
This is very important if you need to get all fields of modified rows atomically.
All Kunlun modules are released in docker images now and containers will continue to be one form of release for Kunlun in future.
In this release we completed a group of very important features involving crash safety, auto failover and more.
Kunlun-storage development, in which all known XA bugs in official MySQL-8.0 are fixed. And also we enhanced its performance so that in standard sysbench performance tests, QPS improved by at most %50 and latency drops by at most 50%. See this report for details.
All other supporting features required for collaboration with computing nodes of a Kunlun DDC.
Added cluster_manager program which maintains correct MGR status of storage shards and metadata shard of Kunlun DDCs that are registered in the same meta-data shard, so that all nodes of an MGR shard keeps running online and XA transaction branches left over by node crashes are correctly handled.
Global deadlocks that couldn't be handled by MySQL Innodb storage engine could now be correctly detected and resolved, so that Kunlun DDC could execute transactions in optimal performance.
When a storage shard/metadata shard's primary node changes for any reason, computing nodes can automatically discover the incident and adapt to it.
Crash safety for computing nodes, storage shards and meta-data shard. In all now a Kunlun DDC could survive network partition, and/or node failures caused by any accidents.
All standard SQL data types supported by PostgreSQL, and some PostgreSQL extended data types, are supported as below.
1.1 All integer types and numeric types: bool, smallint, int, bigint, numeric(P,S), money, float/real, double;
1.2 All text types: char(N), varchar(N), text, blob, user defined enum types, and bit(N)/varbit(N))
1.3 Most date and/or time types, including date, time, timetz, timestamp, timestamptz. interval is not yet supported.
1.4 Some PostgreSQL private types are supported: Name, Oid, CID, TID, XID, LSN, macaddr, macaddr8, cidr, uuid.
1.5 Some PostgreSQL's private extensions are NOT supported, including arrays, ranges, vectors, composite types, row types, table inheritance, table/relation types, etc.
1.6 Domains and user defined types that derive from basic numeric or text types are not supported now but they may be supported in future; All other user defined domains/types except enum types will not be supported.
All standard SQL operators and functions, and most postgreSQL specific functions&operators are supported. Although MySQL doesn't support non-SQL-standard operators and functions of PostgreSQL, Kunlun can handle such queries correctly, some by converting to MySQL supported ones, some by computing locally inside computing nodes and only push down the operators and functions that MySQL can handle to storage shards.
Partitions of a table partition may be stored in one or more storage shards, decided by the computing nodes. They are read/written as necessary, also decided by computing nodes.
Create/drop table/index/database/schema are well supported.
These DDLs are well supported as being 'automatic' --- when a client sends such a DDL to one computing node execution, the database object(table, schema, database) will be automatically created on all computing nodes and target storage shards.
create table NAME (like table-template); is supported; UNLOGGED tables are created as MyISAM tables in storage nodes. A TEMPORARY table is stored in the computing node since it's only used throughout the session where it's created.
Unsupported clauses of these DDLs are listed below.
For all other DDLs, users have to execute the same DDL on each computing node instance for now, and such DDLs don't need storage shard actions. This may be made partly automatic in future, so that users need only to send the same such DDLs only once to Kunlun system and it will take effect on all computing nodes of the cluster.
Insert/delete/update/select stmts are well supported, with below exceptions. update/delete don't support multi-table statements or 'returning' clause yet, will be supported in future. insert/update don't support 'ON CONFLICT' clauses yet, will be supported in future.
This is on-going effort, the ultimate goal is to support all of them in highly efficient manner.
This is transparently done inside Kunlun system, users don't need to do anything special, they simply issue transaction commands the same way as using PostgreSQL. In terms of transaction processing, users can simply take Kunlun as if they were using a PostgreSQL database. With mysql community server as storage shard node, global deadlock detector can't work, so there can be global deadlocks which can't be resolved by innodb. One needs specialized version which will be released soon.
Some of these will be supported in the future, others will never be supported, as detailed below. For those that 'may be supported in future' or 'never supported', using it on current version causes undefined behavior and undefined result to your data and cluster, so never attempt on a production deployment. This is denoted as 'behavior undefined' below.
For local catalog tables, everything works just as in original PostgreSQL, all the stuff here is about user tables whose payload data is actually stored in storage shards.
In all we can roughly conclude that most of PostgreSQL's 'advanced' features and 'private extensions' are not supported now or never in future. We only expose and make use of the most commonly used features of PostgreSQL as Kunlun's functionality set.
1.1 Alter table stmts
Some of them will be supported in future. For now, behavior undefined.
1.2 create table ... select from
never supported, per restriction by MySQL. For now, behavior undefined.
1.3 foreign keys
never supported, for performance reasons. Specifying them produces an error.
1.4 'generated as identity' in 'create table' stmt
may be supported in future. For now, behavior undefined.
1.5 db/table/index storage settings
These include 'with oids' table option, the tablespace settings and table storage parameters for dbs, tables and indexes. Since user tables are not 'stored' in computing nodes, no such storage related settings are supported. Specifying 'WITH OIDS' setting and any storage parameter of tables other than the newly added 'shard' parameter, produces an error. Storage parameters for indexes and attributes are simply ignored; All tablespace settings are ignored.
1.7. forbidden index settings/clauses in 'create index'
Exclude, include, COLLATE, and partial indexing. Specifying them produces an error, they'll be never supported as restricted by mysql. Expressions as index key parts is not supported for now, may support it in future. 'concurrent' clause is ignored.
1.8 DEFAULT partition of a partitioned table
may be supported in future.
1.9 triggers, stored procedures
may be supported in future.
1.10 TABLESPACES
CREATE/ALTER TABLESPACE, and tablespace settings in anywhere including CREATE/ALTER database/table/index.(never supported) Users can still create/alter/remove tablespaces, but these tablespaces will never be effectively used.
1.11 ALTER DATABASE
Rename database will never be supported, doing so produces an error. Other features of 'alter database' may be supported in future.
1.12 Cursors
Cursors will never be supported, per the nature of table sharding. So DECLARE, FETCH, MOVE stmts, and the use of cursors in UPDATE/DELETE stmts, will never be supported, the result of using them is undefined.
1.13 unsupported table options
ON COMMIT clause in 'CREATE TABLE' is not supported. create table using table inheritance is NOT supported. Using both of these stmts is behavior undefined.
2.1 multi-table join
This is on-going effort, some already working. It will be much better improved in future versions.
2.2 insert into ... select from
may be supported in future
2.3 access to system columns
System columns including OID, TableOid, xmin, xmax, txnid, etc are not available, since user data is not stores in PostgreSQL heap relations.
2.4 update/delete stmts
We don't support multi-table statements or 'returning' clause yet. It will be supported in future.