Skip to content
This repository has been archived by the owner on Jan 28, 2021. It is now read-only.

sql: use checksums to determine if indexes are outdated #583

Merged
merged 1 commit into from
Jan 14, 2019

Conversation

erizocosmico
Copy link
Contributor

@erizocosmico erizocosmico commented Jan 10, 2019

First part of src-d/gitbase#661

After a few attempts, I've settled with this way of handling this, since this is backwards compatible, does not introduce any breaking changes and uses existing mechanisms.

Checksum is sent to the driver using the index configuration that we were not using for anything yet. Instead of changing the API of the drivers to accept a checksum, since this is more of a temporal solution until updates are supported, I preferred to use this way.

Indexes hold the checksum value inside so they can return it. An sql.Index may or may not implement sql.Checksumable, which makes it really up to the driver to use checksums or not.

Tables may also implement sql.Checksumable. If a table is not checksumable, the driver will do nothing related to checksums. If it is, when the driver is loaded it will compare both checksums and mark the index as outdated if they don't match.

A new IndexOutdated status has been added for indexes. The index is loaded, so it can be dropped, but it can not be used in any query or overwritten before deleting it. Not loading it if it was outdated would have made it possible for multiple identical indexes to exist at the same time for different versions, which did not make much sense.
When the index is marked as outdated a warning is printed.

@erizocosmico erizocosmico requested a review from a team January 10, 2019 14:47
Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
Copy link
Contributor

@kuba-- kuba-- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how we gonna calculate the checksum, because it should be per expression. In other words, if you change non-indexed column in the table there is no need to invalidate index.

@erizocosmico
Copy link
Contributor Author

@kuba-- since this is temporary until we support updates I went the fast path: if the repo changes, the index is outdated. Checking if something has changed for an expression would essentially be recreating the index and comparing, which is not feasible.

@kuba--
Copy link
Contributor

kuba-- commented Jan 11, 2019

Sometimes I have a feeling, that maybe we should trigger two go routines to find values (one with index and the second one will ignore the index). Generally the first one wins, but in case of error, we can wait for the second one).

@ajnavarro ajnavarro merged commit 8a4cd29 into src-d:master Jan 14, 2019
@erizocosmico erizocosmico deleted the feature/index-checksum branch January 14, 2019 14:46
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants