Multi path fast filters #22574
Multi path fast filters #22574
Conversation
| // mechanism does not create a Set unless there's a Node for it | ||
| return [...nodesByKeyValue] | ||
| // Put smallest last (we'll pop it) | ||
| caches.sort((a, b) => b.length - a.length) |
There was a problem hiding this comment.
Should get buckets for filters return pre-sorted caches? Maybe not.
There was a problem hiding this comment.
It's possible but these lists shouldn't be big. Generally, the number of filters per query is not that big so this sort should be super-duper fast
| // Put smallest last (we'll pop it) | ||
| caches.sort((a, b) => b.length - a.length) | ||
| // Iterate on the set with the fewest elements and create the intersection | ||
| const needles = caches.pop() |
|
Going to run a benchmark to get some numbers and then I'll merge. The Cloud tests failed on some unrelated trunk breakage and I don't expect them to be broken by this. |
|
Ran some benchmarks;
This seems to a consistent small regression, except for the site I was actually trying to improve. So I guess that's good :) |
|
Merging since it seems to have the desired effect at scale, even if it's a small regression for non-scale. This will unblock further improvements in this area. |
|
Published in gatsby@2.20.9 |
This will support the fast filter cache for filters with multiple paths (all leading to
eq).Before this commit, the fast filter would only apply to "flat" filters leading to eq. That means
filter: {a: {b: {c: {eq: 5}}}}would be fast butfilter: {a: {b: {c: {eq: 5}, d: {eq: 6}}}}(and all varieties) would be slow, take the sift path.After this commit, any number of filters should take the fast(er) path. It'll not be as fast as a flat path because it needs to do an intersection of all filter conditions (of course), but should still be faster than going through sift.