Refactor and test glob sets.

This commit goes a long way toward refactoring glob sets so that the code is easier to maintain going forward. In particular, it makes the literal optimizations that glob sets used a lot more structured and much easier to extend. Tests have also been modified to include glob sets. There's still a bit of polish work left to do before a release. This also fixes the immediate issue where large gitignore files were causing ripgrep to slow way down. While we don't technically fix it for good, we're a lot better about reducing the number of regexes we compile. In particular, if a gitignore file contains thousands of patterns that can't be matched more simply using literals, then ripgrep will slow down again. We could fix this for good by avoiding RegexSet if the number of regexes grows too large. Fixes #134.
BurntSushi · Oct 5, 2016 · 175406d · 175406d
1 parent 89811d4
commit 175406d
Show file tree

Hide file tree

Showing 8 changed files with 1,884 additions and 921 deletions.
diff --git a/Cargo.lock b/Cargo.lock
diff --git a/globset/Cargo.toml b/globset/Cargo.toml
@@ -4,7 +4,9 @@ version = "0.1.0"
 authors = ["Andrew Gallant <jamslam@gmail.com>"]
 
 [dependencies]
+aho-corasick = "0.5.3"
 fnv = "1.0"
 lazy_static = "0.2"
+log = "0.3"
 memchr = "0.1"
 regex = "0.1.77"