add support for creating an index & using it when searching for easyconfigs #3210
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…everage this in search_file function
…ry, load_index return None if there is no index & dump_index return path to index file
…load_index check whether index is still valid based on age
… creating/loading index
houndci-bot
reviewed
Feb 21, 2020
houndci-bot
reviewed
Feb 21, 2020
|
@boegel is there an easy way to remove the index as well? |
You mean Or do you want a |
|
I was thinking about how this is going to be used in practice. |
|
@smoors Having an |
smoors
reviewed
Mar 2, 2020
easybuild/tools/filetools.py
Outdated
| # do not consider (certain) hidden directories | ||
| # note: we still need to consider e.g., .local ! | ||
| # replace list elements using [:], so os.walk doesn't process deleted directories | ||
| # see http://stackoverflow.com/questions/13454164/os-walk-without-hidden-folders |
boegel
force-pushed
the
easyconfigs_index
branch
from
ed0129b to
b5c5b38
Compare
boegel
force-pushed
the
easyconfigs_index
branch
from
b5c5b38 to
6bb2131
Compare
smoors
reviewed
Mar 15, 2020
smoors
reviewed
Mar 15, 2020
|
Going in, thanks @boegel! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
With
--create-indexyou can create an index file that significantly speeds up searching for (easyconfig) files (& patches) on slow filesystems. You can control how long the index is valid via--index-max-age(in seconds), where 1 week (604800 seconds) is the default, and0indicates that the index remains valid indefinitely.If an index is available, it's used both for
--searchand--robot.