-
Notifications
You must be signed in to change notification settings - Fork 143
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Delayed/Failure to read your writes, lost writes are common #1254
Comments
Hi, To help interpret cycles and anomalies, an HTML table is now created showing all transactions interacting with any of the [k v] in the cycle. Here's an example that shows data loss: {:cyclic-versions ({:sources #{:writes-follow-reads-vg
:read-prefix-vg
:monotonic-reads-vg
:monotonic-writes-vg
:init-nil-vg},
:sccs (#{[10 126] [10 127] [10 128]}
;; plus 9 more different cycles
)})} Looking at the transactions that interacted with the cyclic versions:
Here's an example where node n4 writes multiple values with different keys, and then only reads one update from each key:
The lost writes occur in multiple different patterns and are common. The same test using an un-electrified SQLite3 db shows no anomalies |
In an attempt to shrink the test environment down to the simplest form that shows anomalous behavior:
Here's an example using the generated client API:
Observations:
Although not conclusive, it does seem to indicate, and is consistent with the idea that electrification of a SQLite3 table changes its behavior in fundamental ways regardless of the API used for access. IOW, it's likely the full complement of anomalies are always present and using a constrained API only affects the nature and frequency of their occurrence? |
Thanks for such a detailed report. Cc @balegas |
👋 we've been working the last month on a rebuild of the Electric server over at a temporary repo https://github.com/electric-sql/electric-next/ You can read more about why we made the decision at https://next.electric-sql.com/about We're really excited about all the new possibilities the new server brings and we hope you'll check it out soon and give us your feedback. We're now moving the temporary repo back here. As part of that migration we're closing all the old issues and PRs. We really appreciate you taking the time to investigate and report this issue! |
Hi,
This issue is a follow-on to #1245.
It's another example of how electrification changes the behavior of SQLite3 in unexpected ways.
Delayed/Failure to read your writes is common.
Please see #1245 for basic test architecture and runtime information:
Here's an example:
Looking at the transaction history for interactions with [8 1] and [8 3] we find a series of transactions for process 1:
Or more formally:
A GitHub action,
ElectricSQL: Read-Your-Writes
, https://github.com/nurturenature/jepsen-causal-consistency/actions/, has been created to reproduce the results.In the results,
:G-single-item-process
and:cyclic-versions
are the anomalies most likely to be failures to read your writes.The
causal
directory will have a directory for each process and anyG-single-item-process
explanations and graphs.There are diverse ways this anomaly presents itself.
And in general, it can take detective work and going through the history by hand.
If you have any ideas for post-analysis tools and how to present the information in understandable/actionable ways they would be appreciated.
Thanks.
The text was updated successfully, but these errors were encountered: