A few more minor changes #2
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
drawks
commented
Feb 26, 2015
drawks
commented
- updated .gitignore to not track _vendor/pkg
- export GOPATH in Makefile
bom-d-van
added a commit
that referenced
this pull request
Jan 12, 2022
There are two bugs fixed in this commit:
1. Empty directories not properly renewed during indexing (and could be
trimmed at every two file list scans).
During insert, trie.insert failed to bump up trie nodes properly if it's
inserting directory. This means that if the directory being inserted is
empty and already exists in the trie tree, when concurrent index is
enabled, the directory nodes might be pruned and then re-inserted in the
next file list scan. And then on and on. This issue it self is not a
serious concern if bug #2 below doesn't exist.
2. Query logics do not handle well for empty directories.
The symptoms are that if the trie index tree contains empty directories,
and if a query happens to matching it, it would causes the query state
stack (matchers) jump to the wrong index, and lead to incorrect matches of
the metrics.
Context:
On our Graphite production, for cleanup purpose, we have daily cron jobs
removing stale graphite metric files and empty directories in the whisper tree.
The cronjob that removes files is run at 5AM and empty directory removal at 7AM.
This means the above bugs have a time window of 1-2 hours being triggered. And
because empty directories are not handled properly due to bug #1, the issue is
triggered every 2 file list scans. Essentially, if the query matched an empty
directory node and there are other metrics listed after the empty directory path,
trie query can't return proper result. In our case, it's returning missing
results, in theory, it could also return incorrect results.
In theory, these fixes are not needed because we stop indexing empty directory
nodes in commit 67446d3. But it's nice to figure
out the root cause and resolve the issue properly!
Took me almost a week to figure it out! But I'm happy. Tears in rain.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.