[YSQL] Support multiple colocation groups in one database #4525
Comments
vvkgopalan
added a commit
to vvkgopalan/yugabyte-db
that referenced
this issue
Jul 7, 2020
Summary: Initial grammar changes - CREATE TABLEGROUP, DROP TABLEGROUP, CREATE TABLE ... TABLEGROUP ..., and CREATE INDEX ... TABLEGROUP ... For questions on design see design doc here: https://docs.google.com/document/d/1HmSYbjIahMO_Svhw0CN0-ggSDEQSW52ZZSh8RKU4V5I/edit?usp=sharing This is the first diff to support colocated tablegroups. The way that I am breaking this project up is the following: 1. Grammar changes - support CREATE & DROP TABLEGROUP as a beta feature. Extend CREATE TABLE & INDEX to support TABLEGROUP. 2. PG metadata changes - create the system table. Will need to look into genbki perl script that generates bki files to initialize template database. Will need to create a pg_tablegroup header file in include/catalog for schema info. Will also need to create tablegroup.c in include/commands that will be responsible for oid/name lookup from the system table as well as creating / dropping entries. Will also have changes in pg_class. 3. CREATE TABLEGROUP flow and CREATE TABLE flow end-to-end, including catalog manager changes. 4. DROP TABLEGROUP flow and DROP TABLE / TRUNCATE TABLE end-to-end. 5. Do upgrade testing on dbs with COLOCATED=TRUE, ensure that new system table is created and populated. 6. ysql_dump / backup & restore support. Test Plan: Just ensure proper errors are thrown for incorrect syntax. These statements do not do anything as of now. Reviewers: neha, nicolas, jason Subscribers: yql Differential Revision: https://phabricator.dev.yugabyte.com/D8747
vvkgopalan
added a commit
to vvkgopalan/yugabyte-db
that referenced
this issue
Jul 14, 2020
Summary: Depends on D8747 This diff has functionality related to tablegroups - primarily passing information about tablegroup DDL from the Postgres layer down to the catalog manager. Catalog manager changes (i.e. storing tablegroup info / tablegroup-to-tablet mappings) will be included in another diff. In ybccmds there are two new tablegroup DDL functions: - YBCCreateTablegroup - YBCDropTablegroup Which are called by tablegroup.c (implemented in D8747) and pass tablegroup oid and name to ybc pggate. This information (along with the name of the connected database) is eventually passed down to the yb client. There are two new RPCs: - CreateTablegroup which takes CreateTablegroupRequestPB and returns CreateTablegroupResponsePB - DeleteTablegroup which takes DeleteTablegroupRequestPB and returns DeleteTablegroupResponsePB Test Plan: TODO Reviewers: jason, nicolas Subscribers: bogdan, neha, ybase Differential Revision: https://phabricator.dev.yugabyte.com/D8830
vvkgopalan
added a commit
to vvkgopalan/yugabyte-db
that referenced
this issue
Jul 14, 2020
Summary: Depends on D8747 This diff has functionality related to tablegroups - primarily passing information about tablegroup DDL from the Postgres layer down to the catalog manager. Catalog manager changes (i.e. storing tablegroup info / tablegroup-to-tablet mappings) will be included in another diff. In ybccmds there are two new tablegroup DDL functions: - YBCCreateTablegroup - YBCDropTablegroup Which are called by tablegroup.c (implemented in D8747) and pass tablegroup oid and name to ybc pggate. This information (along with the name of the connected database) is eventually passed down to the yb client. There are two new RPCs: - CreateTablegroup which takes CreateTablegroupRequestPB and returns CreateTablegroupResponsePB - DeleteTablegroup which takes DeleteTablegroupRequestPB and returns DeleteTablegroupResponsePB Test Plan: TODO Reviewers: jason, nicolas Subscribers: bogdan, neha, ybase Differential Revision: https://phabricator.dev.yugabyte.com/D8830
vvkgopalan
added a commit
that referenced
this issue
Jul 29, 2020
Summary:
The motivation behind having multiple groups of co-located tables within a database is similar to that of specifying a database as co-located. Currently, in specifying a database as co-located, we are able to generate a single tablet that all future tables (that do not opt out of co-location) are added to. This allows a user to co-locate closely related tables together and avoid paying the cost of additional round trips when making joins etc. amongst the related data or paying the storage/compute overhead of creating a tablet for every relation, improving performance.
The first step in supporting more flexible colocation is creating the concept of a "tablegroup." Colocating tables together within a tablegroup allows for related tables to be placed on one tablet (without restrictions on their primary keys). This will be supported in YSQL as follows:
- CREATE TABLEGROUP tablegroup_name grpoptions;
This will create a tablegroup within the current database to allow for future tables to be attached to and colocated together. Similar to our implementation of a colocated database, on the docdb side we will create a single tablet to store the data of all tables that belong to the tablegroup. grpoptions will be how we support Copartitioning / Interleaving in the future.
- DROP TABLEGROUP tablegroup_name;
This will remove a tablegroup of the corresponding name (if it exists).
- CREATE TABLE ... TABLEGROUP tablegroup_name;
This will add a table to a tablegroup. We will not support the joint inclusion of SPLIT and TABLEGROUP in any DDL as tablegroups (for now) are single tablets. CREATE INDEX follows the tablegroup of the indexed table.
This also involves some changes to the postgres system catalog in order to maintain tablegroup relationships on backup and restore as well as catch DDL errors on the PG side. The following changes will be included in this diff:
- The inclusion of the catalog `pg_tablegroup` which will include a tablegroup's oid, name, owner, acl, and colocation options.
- The inclusion of a `tablegroup` option in pg_class `reloptions` that contains the Oid of a relation's tablegroup (if any, otherwise the text option is null and the `tablegroup` field in the rd_options struct is unallocated).
```
Table "pg_catalog.pg_tablegroup"
Column | Type | Collation | Nullable | Default
------------+-----------+-----------+----------+---------
grpname | name | | not null |
grpowner | oid | | not null |
grpacl | aclitem[] | | |
grpoptions | text[] | | |
Indexes:
"pg_tablegroup_oid_index" PRIMARY KEY, lsm (oid)
```
Additionally, to avoid confusion, we will not mix the previous colocation functionality and tablegroups. That means that within a colocated database, no tablegroups will be able to be created. The prior implementation of colocation will eventually be deprecated and running clusters with colocated databases will be migrated to use tablegroups.
This diff also includes functionality related to privileges and tablegroup ddl. In order to CREATE TABLEGROUP, the user must either have CREATE privs within the database or be a superuser. In order to DROP TABLEGROUP, the user must be the owner of the tablegroup or a superuser.
The default OIDs I have picked are as followed:
- pg_class OID for pg_tablegroup (TableGroupRelationId): 8000
- pg_class OID for pg_tablegroup_oid_index (TablegroupOidIndexId): 8001
- pg_type rowtype OID for pg_tablegroup (TablegroupRelation_Rowtype_Id): 8002
For all future changes to the catalog that introduce a new default OID, please change `YB_MIN_UNUSED_OID` in `include/catalog/catalog.h` which is currently set to 8003.
Test Plan:
Added the following Java test:
```
org/yb/pgsql/TestPgRegressTablegroup.java
```
As well as the following test schedule with corresponding sql test and expected output:
```
regress/yb_tablegroups_schedule
regress/sql/yb_create_tablegroup.sql
regress/expected/yb_create_tablegroup.out
regress/sql/yb_create_tablegroup_permissions.sql
regress/expected/yb_create_tablegroup_permissions.out
```
The test can be run as follows (reinitdb necessary for pg metadata changes):
```
./yb_build.sh reinitdb --clean-postgres --java-test org.yb.pgsql.TestPgRegressTablegroup
```
To test upgrade did not break things I did the following:
```
git checkout master
ybd && ybd reinitdb
./bin/yb-ctl destroy && ./bin/yb-ctl create
git checkout metadata
ybd
./bin/yb-ctl restart && ./bin/ysqlsh
```
Then I tested various DDL/DML operations to ensure that pg_class tuples were correctly inserted / operations did not break.
I also tested that any statements involving tablegroups produced the following:
```
ERROR: Tablegroup system catalog does not exist.
```
Reviewers: neha, jason, nicolas, mihnea
Reviewed By: jason, nicolas, mihnea
Subscribers: yql
Differential Revision: https://phabricator.dev.yugabyte.com/D8747
vvkgopalan
added a commit
that referenced
this issue
Jul 30, 2020
Summary: Depends on D8747 This diff has functionality related to tablegroups - primarily passing information about tablegroup DDL from the Postgres layer down to the catalog manager. Catalog manager changes (i.e. storing tablegroup info / tablegroup-to-tablet mappings) will be included in another diff. In ybccmds there are two new tablegroup DDL functions: - YBCCreateTablegroup - YBCDropTablegroup Which are called by tablegroup.c (implemented in D8747) and pass tablegroup oid and name to ybc pggate. This information (along with the name of the connected database) is eventually passed down to the yb client. There are two new RPCs: - CreateTablegroup which takes CreateTablegroupRequestPB and returns CreateTablegroupResponsePB - DeleteTablegroup which takes DeleteTablegroupRequestPB and returns DeleteTablegroupResponsePB Test Plan: Ensure that this test does not break: ``` ./yb_build.sh --java-test org.yb.pgsql.TestPgRegressTablegroup ``` Reviewers: jason, nicolas Reviewed By: nicolas Subscribers: jenkins-bot, bogdan, neha, ybase Differential Revision: https://phabricator.dev.yugabyte.com/D8830
vvkgopalan
added a commit
that referenced
this issue
Aug 8, 2020
Summary: Depends on D8830. This diff implements the docdb side of tablegroup creation and deletion. The flow is as follows: For CREATE TABLEGROUP, a parent table is created for the tablegroup in the same manner that a parent table is created for a database with colocated=true. This table will have a single tablet that all future tables for the tablegroup will be added to. Catalog manager will store information related to tablegroups and tablets within two maps: # `tablegroup_tablet_ids_map_` is a map of `<NamespaceId, TablegroupTabletMap>` ## `TablegroupTabletMap` is a map of `<TablegroupId, TabletInfo>` - which will become `<TablegroupId, vector<TabletInfo>>` once copartitioning/interleaving is implemented. # `tablegroup_ids_map_` is a map of `<TablegroupId, TablegroupInfo>` Tablegroup uniqueness/existance is checked on the Postgres side. For DROP TABLEGROUP, the parent table for the tablegroup is deleted. The corresponding map entries for the TablegroupId are erased. Test Plan: This test is for DDL, permissions, and DML operations: ``` ./yb_build.sh --java-test org.yb.pgsql.TestPgRegressTablegroup ``` This tests that tables are properly placed on the parent tablets / verifies that parent tablets are being created for tablegroups: ``` ybd --cxx-test pgwrapper_pg_libpq-test --gtest_filter PgLibPqTest.ColocatedTablegroups ``` This tests remote bootstrapping with tablegroups (should be no different to traditional colocation): ``` ybd --cxx-test create-table-itest --gtest_filter CreateTableITest.TablegroupRemoteBootstrapTest ``` This tests restarts: ``` ybd --cxx-test master-test --gtest_filter MasterTest.TestTablegroups ``` Reviewers: jason, nicolas Reviewed By: nicolas Subscribers: neha, bogdan, ybase Differential Revision: https://phabricator.dev.yugabyte.com/D8844
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently for colocated tables, we have one colocation group per database (associated with the database).
In some cases though, several mostly-independent groups of tables (e.g. in different schemas) can exist in the same database and benefit from colocation within the group but adding all to the same group limits scalability (since they are all on the same tablet).
In such cases having more than one colocation group in a database (e.g. one per schema) can help find the best tradeoff between latency and throughput/scalability for the use-case.
The text was updated successfully, but these errors were encountered: