-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Delta Lake: Improve error message related to storage layers which allow write collisions #14904
Delta Lake: Improve error message related to storage layers which allow write collisions #14904
Conversation
This seems like something that should go in the docs, not in an error message. An individual user doesn't have the ability to turn this on, like they do with session properties. It should also come along with a warning that writes are safe as long as you only have one cluster doing them. |
@alexjo2144 what about using this error message:
? From a usability point of view, we should give somehow the hint in the message which can be used by sysadmin to adapt the connector configuration if needed. |
According to my understanding when competing writes are happening, the order of execution is random and therefore also the result, even when only one cluster is used. So enabling should probably come with a warning in any case. I personally think it is user-friendly to also mention the property to set. We do the same in trino/plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/IcebergMetadata.java Lines 693 to 696 in ea02559
|
That's how transaction failures work on most systems with any isolation level higher than "repeatable read". If multiple queries hold a table lock only one of them will win but which one isn't guaranteed. The failed one has to be retried. You don't notice it as often with other databases since the default isolation is generally never "repeatable read". (I do agree we should explain this to users - just wanted to add context to which users of other systems may relate). I like how Postgres docs explain it https://www.postgresql.org/docs/current/transaction-iso.html (search for |
This is better, to me. |
plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMetadata.java
Outdated
Show resolved
Hide resolved
b380ae0
to
bdd966a
Compare
Description
Fixes #14464
Non-technical explanation
Release notes
(x) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text: