Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document correct usage of a ResultIterator #85

Closed
wants to merge 1 commit into from

Conversation

dnephin
Copy link
Contributor

@dnephin dnephin commented Jan 21, 2021

Last week we had a panic in Consul that was caused by incorrect usage of a ResultIterator. A minimal reproduction of this panic can be seen here.

Many of us learned that there are undocumented rules for using a ResultIterator correctly with a write transaction.

This commit attempts to document that safe use of a ResultIterator.

We receicently had [a panic in Consul](hashicorp/consul#9566)
that was caused by incorrect usage of a ResultIterator. A minimal
reproduction of this panic can be seen [here](https://github.com/hashicorp/go-memdb/compare/panic-delete-inside-iter).

Many of us learned that there are undocumented rules for using a
ResultIterator correct with a write transaction.

This commit attempts to document that safe use of a ResultIterator.
@dnephin
Copy link
Contributor Author

dnephin commented Jan 21, 2021

The panic happens because ResultIterator gets a copy of the edges slice from a Node. When an edge is removed the backing array of that slice is modified and the last element is zeroed out so that the GC can free any allocated memory that is no longer referenced by a slice (but is still in the backing array).

When the next call to ResultIterator.Next happens, the copy of the slice in the iterator still has the old length, so it gets an edge with a nil node pointer, which panics when it tries to read a field on that nil.

Even without a panic, modifications to the tree can cause Next() to return duplicate results (or maybe skip results as well?).

dnephin added a commit that referenced this pull request Jan 21, 2021
Previously this could panic anyway (see #85),
but only when the nodse in an index line up in a very specify way. This
meant that tests may not exercise the bug, and the panic would get into
a release.

By adding an explicit panic here we ensure that any basic test coverage
of an iteration would fail.

In the case of inserts it would not panic, but could return incorrect
results (most likely duplicate). This panic ensures we fail instead of
returning incorrect results.

It would have been nice to error instead of panic, but there's no place
in the current interface to return an error.
@dnephin dnephin closed this in #87 Jan 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant