Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[docdb] Support savepoints in YSQL #9219

Closed
robertsami opened this issue Jul 8, 2021 · 1 comment
Closed

[docdb] Support savepoints in YSQL #9219

robertsami opened this issue Jul 8, 2021 · 1 comment
Assignees
@robertsami
Labels
area/docdb YugabyteDB core features area/ysql Yugabyte SQL (YSQL)
Projects
YBase features YSQL

Comments

@robertsami
Copy link
Contributor

robertsami commented Jul 8, 2021 •
edited

Add support the following commands:

  • SAVEPOINT {name}
  • RELEASE SAVEPOINT {name}
  • RELEASE {name}
  • ROLLBACK TO SAVEPOINT {name}
  • ROLLBACK TO {name}

Breaking these commands out of the original issue: #1125

Breaking out work items here:

Status Feature GitHub Issue
✅ Add client-side support for managing subtransaction state
✅ Persist subtransaction ids with intent values
✅ Ignore intents of aborted subtransactions during reads in same transaction
✅ Ignore intents of aborted subtransactions during transaction apply #9586
✅ Refactor client-side tracking of subtransaction state #9593
✅ Ensure read restarts work as intended with savepoints #9799
✅ Synchronously read aborted subtransaction state of unapplied but committed intents #9855
✅ Enable savepoints by default #10141
✅ Support exception handling using savepoints #2397

Other potential work items:
@robertsami robertsami mentioned this issue Jul 8, 2021
Open
robertsami added a commit that referenced this issue Jul 20, 2021
@robertsami
[#9219] ysql - Add superficial client-side support for SAVEPOINT and …
197d4c9 
…RELEASE commands

Summary:
This revision adds superficial support for savepoint creation and release at the ysql shell. It enables postgres-side management of subtransaction state and plumbs that metadata through the client to the tserver in all session Read/Write requests. It only includes such metadata if a savepoint has been created in this session at least once AND a transaction has been initialized -- otherwise, the requests are unchanged.

This revision also takes the conservative stance that no transactions with active savepoints at any time should be opaquely retried on e.g. a read restart or other failure.

The current implementation on the postgres side uses SubTransactionId=1 as the starting SubTransactionId. We plan to adopt that convention and fully accept the SubTransactionId accounting done by postgres.

Follow-up client-side work will partially enable "ROLLBACK" commands by adding some client-side state to track aborted subtransactions and send this along in every read, write, and commit request to tservers.

Test Plan:
bin/yugabyted start --tserver_flags "vmodule=tablet_service=2,pg_savepoints_enabled=true"

Then, tail the tserver logs file and issue the following commands:

```
yugabyte=# begin;
BEGIN
yugabyte=# insert into foo values(3);
INSERT 0 1
yugabyte=# savepoint a;
SAVEPOINT
yugabyte=# insert into foo values(4);
INSERT 0 1
yugabyte=# release a;
RELEASE
yugabyte=# insert into foo values(5);
INSERT 0 1
yugabyte=# savepoint b;
SAVEPOINT
yugabyte=# insert into foo values(6);
INSERT 0 1
yugabyte=# release b;
RELEASE
yugabyte=# insert into foo values(7);
INSERT 0 1
```

And each insert should show the following TransactionMetadata:
```
write_batch {
  transaction {
    transaction_id: "P\353\202}\3332K\233\203\374\343\034\326se("
    isolation: SNAPSHOT_ISOLATION
    status_tablet: "3c99c9f0bd044961b164597de13fa282"
    priority: 17077319265436291138
    start_hybrid_time: 6659154577935114240
  }
  DEPRECATED_may_have_metadata: true
}

write_batch {
  transaction {
    transaction_id: "P\353\202}\3332K\233\203\374\343\034\326se("
  }
  DEPRECATED_may_have_metadata: true
  subtransaction {
    subtransaction_id: 2
  }
}

write_batch {
  transaction {
    transaction_id: "P\353\202}\3332K\233\203\374\343\034\326se("
  }
  DEPRECATED_may_have_metadata: true
  subtransaction {
    subtransaction_id: 1
  }
}

write_batch {
  transaction {
    transaction_id: "P\353\202}\3332K\233\203\374\343\034\326se("
  }
  DEPRECATED_may_have_metadata: true
  subtransaction {
    subtransaction_id: 3
  }
}

write_batch {
  transaction {
    transaction_id: "P\353\202}\3332K\233\203\374\343\034\326se("
  }
  DEPRECATED_may_have_metadata: true
  subtransaction {
    subtransaction_id: 1
  }
}
```

Reviewers: pjain, mihnea, mbautin, dmitry

Reviewed By: mbautin, dmitry

Subscribers: sergei, bogdan, yql

Differential Revision: https://phabricator.dev.yugabyte.com/D12209
robertsami added a commit that referenced this issue Jul 21, 2021
@robertsami
[#9219] docdb - Persist SubTransactionId with intent value
1415b1a 
Summary: This diff introduces a new flag "yb_enable_savepoints" as well as codepaths which will persist SubTransactionId in the intent value prefix if the flag is enabled and subtransaction metadata is present in the write request being processed.

Test Plan:
`bin/yugabyted start --tserver_flags "vmodule=intent_aware_iterator=4,pg_savepoints_enabled=true,yb_savepoints_enabled=true"`

Issue the following series of commands:
```
yugabyte=# begin;
BEGIN
yugabyte=# insert into foo values(12);
INSERT 0 1
yugabyte=# savepoint a;
SAVEPOINT
yugabyte=# insert into foo values(13);
INSERT 0 1
yugabyte=# select * from foo;
```

On the second `select * ...`, confirm from logs that SubTransactionId 2 was decoded from two intent values. Additionally confirm that commit works as expected.

Reviewers: mbautin, sergei

Reviewed By: sergei

Subscribers: ybase, pjain, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D12259
@bmatican bmatican added area/docdb YugabyteDB core features area/ysql Yugabyte SQL (YSQL) labels Jul 22, 2021
@bmatican bmatican added this to Backlog in YBase features via automation Jul 22, 2021
@bmatican bmatican added this to Backlog in YSQL via automation Jul 22, 2021
robertsami added a commit that referenced this issue Aug 2, 2021
@robertsami
[#9219] docdb - Ignore intents from aborted subtransactions during re…
c8eb974 
…ads and writes of the same transaction

Summary:
This revision add support for client-side tracking of aborted subtransactions, transmission of this
metadata to docdb, and consideration of this metadata when processing intents for reads and writes.
This revision *does not* ensure that aborted intents are not applied after transaction commit, so
"ROLLBACK TO SAVEPOINT" cannot yet be safely used and remains guarded by default.

This revision also implements some basic java test coverage of some of the savepoint functionality
implemented so far, but is by no means exhaustive. These tests were meant to merely demonstrate
basic functionality that has been implemented in previous revisions newly in this one.

Test Plan:
ybd --java-test 'org.yb.pgsql.TestPgSavepoints'
ybd --cxx-test util_uint_set-test

Reviewers: mbautin, sergei

Reviewed By: sergei

Subscribers: ybase, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D12331
robertsami added a commit that referenced this issue Aug 27, 2021
@robertsami
[#9219] ysql - Add test illustrating duplicate name savepoint behavior
c980b90 
Summary:
This revision adds a test which verifies and illustrates the behavior of savepoints which are
identically named and active simultaneously.

Test Plan: ybd --java-test 'org.yb.pgsql.TestPgSavepoints#testStackedSavepoints'

Reviewers: pjain

Reviewed By: pjain

Subscribers: mbautin, bogdan, ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D12753
robertsami added a commit that referenced this issue Sep 1, 2021
@robertsami
[BACKPORT TO 2.9][#9219] ysql - Add test illustrating duplicate name …
5002384 
…savepoint behavior

Summary:
This revision adds a test which verifies and illustrates the behavior of savepoints which are
identically named and active simultaneously.

Original commit: D12753 / c980b90

Test Plan:
ybd --java-test 'org.yb.pgsql.TestPgSavepoints#testStackedSavepoints'

Jenkins: rebase: 2.9.0

Reviewers: pjain

Reviewed By: pjain

Differential Revision: https://phabricator.dev.yugabyte.com/D12830
robertsami added a commit to robertsami/yugabyte-db that referenced this issue Sep 22, 2021
@robertsami
[yugabyte#9219] Add API reference for SAVEPOINT commands
51f9cbe
robertsami added a commit that referenced this issue Sep 23, 2021
@robertsami
[#9219] Add API reference for SAVEPOINT commands (#10050)
13faf5c 
Co-authored-by: Alex Ball <aball@yugabyte.com>
@xvaara xvaara mentioned this issue Oct 5, 2021
Closed
@robertsami
Copy link
Contributor Author

Closing with a note that a few items were never addressed. We do not currently plan to complete them. If there is a need for them, please leave a comment and re-open the specific issue so we may address it!

YBase features automation moved this from Backlog to Done Mar 1, 2022
YSQL automation moved this from Backlog to Done Mar 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@robertsami robertsami

Labels
area/docdb YugabyteDB core features area/ysql Yugabyte SQL (YSQL)
Projects
YBase features
  
Done
YQL-beta
Status: Done
YSQL
  
Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bmatican @robertsami