This repository has been archived by the owner on Apr 4, 2023. It is now read-only.
Only flatten the required objects #494
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
9 tasks
Kerollmops
suggested changes
Apr 12, 2022
irevoire
force-pushed
the
flatten-what-is-needed
branch
from
1c1fd8c
to
52d5044
Compare
Kerollmops
previously approved these changes
Apr 13, 2022
bors bot
added a commit
that referenced
this pull request
Apr 13, 2022
494: Only flatten the required objects r=Kerollmops a=irevoire Instead of flattening every object to write them in the flatenned sorter we now check if we needs to flatten the object or if we can insert it as-is. ``` group indexing_flatten-what-is-needed_a6031f9a indexing_main_6b073738 ----- ---------------------------------------- ---------------------- indexing/Indexing geo_point 1.00 25.1±0.20s ? ?/sec 1.00 25.2±0.20s ? ?/sec indexing/Indexing movies in three batches 1.01 18.3±0.12s ? ?/sec 1.00 18.2±0.10s ? ?/sec indexing/Indexing movies with default settings 1.00 17.6±0.11s ? ?/sec 1.00 17.5±0.09s ? ?/sec indexing/Indexing songs in three batches with default settings 1.00 66.4±0.46s ? ?/sec 1.03 68.3±1.01s ? ?/sec indexing/Indexing songs with default settings 1.00 55.7±1.15s ? ?/sec 1.14 63.2±0.78s ? ?/sec indexing/Indexing songs without any facets 1.00 51.6±1.04s ? ?/sec 1.16 59.6±1.00s ? ?/sec indexing/Indexing songs without faceted numbers 1.00 55.3±1.09s ? ?/sec 1.13 62.8±0.38s ? ?/sec indexing/Indexing wiki 1.00 1006.6±26.89s ? ?/sec 1.00 1009.2±25.25s ? ?/sec indexing/Indexing wiki in three batches 1.00 1140.5±11.97s ? ?/sec 1.00 1142.0±9.97s ? ?/sec ``` We now have performance similar to what we had before for the non nested datasets 🎉 Co-authored-by: Tamo <tamo@meilisearch.com>
|
This PR was included in a batch that successfully built, but then failed to merge into main. It will not be retried. Additional information: {"message":"Required status check \"Specify breaking\" is expected.","documentation_url":"https://docs.github.com/articles/about-protected-branches"} |
|
Could you rebase? This PR does not have the latest changes regarding the bors settings |
irevoire
force-pushed
the
flatten-what-is-needed
branch
from
52d5044
to
cff986e
Compare
|
I rebased on main |
bors bot
added a commit
that referenced
this pull request
Apr 13, 2022
494: Only flatten the required objects r=Kerollmops a=irevoire Instead of flattening every object to write them in the flatenned sorter we now check if we needs to flatten the object or if we can insert it as-is. ``` group indexing_flatten-what-is-needed_a6031f9a indexing_main_6b073738 ----- ---------------------------------------- ---------------------- indexing/Indexing geo_point 1.00 25.1±0.20s ? ?/sec 1.00 25.2±0.20s ? ?/sec indexing/Indexing movies in three batches 1.01 18.3±0.12s ? ?/sec 1.00 18.2±0.10s ? ?/sec indexing/Indexing movies with default settings 1.00 17.6±0.11s ? ?/sec 1.00 17.5±0.09s ? ?/sec indexing/Indexing songs in three batches with default settings 1.00 66.4±0.46s ? ?/sec 1.03 68.3±1.01s ? ?/sec indexing/Indexing songs with default settings 1.00 55.7±1.15s ? ?/sec 1.14 63.2±0.78s ? ?/sec indexing/Indexing songs without any facets 1.00 51.6±1.04s ? ?/sec 1.16 59.6±1.00s ? ?/sec indexing/Indexing songs without faceted numbers 1.00 55.3±1.09s ? ?/sec 1.13 62.8±0.38s ? ?/sec indexing/Indexing wiki 1.00 1006.6±26.89s ? ?/sec 1.00 1009.2±25.25s ? ?/sec indexing/Indexing wiki in three batches 1.00 1140.5±11.97s ? ?/sec 1.00 1142.0±9.97s ? ?/sec ``` We now have performance similar to what we had before for the non nested datasets 🎉 Co-authored-by: Tamo <tamo@meilisearch.com>
|
This PR was included in a batch that successfully built, but then failed to merge into main. It will not be retried. Additional information: {"message":"Required status check \"Specify breaking\" is expected.","documentation_url":"https://docs.github.com/articles/about-protected-branches"} |
|
bors try |
|
bors merge |
bors bot
added a commit
that referenced
this pull request
Apr 14, 2022
494: Only flatten the required objects r=irevoire a=irevoire Instead of flattening every object to write them in the flatenned sorter we now check if we needs to flatten the object or if we can insert it as-is. ``` group indexing_flatten-what-is-needed_a6031f9a indexing_main_6b073738 ----- ---------------------------------------- ---------------------- indexing/Indexing geo_point 1.00 25.1±0.20s ? ?/sec 1.00 25.2±0.20s ? ?/sec indexing/Indexing movies in three batches 1.01 18.3±0.12s ? ?/sec 1.00 18.2±0.10s ? ?/sec indexing/Indexing movies with default settings 1.00 17.6±0.11s ? ?/sec 1.00 17.5±0.09s ? ?/sec indexing/Indexing songs in three batches with default settings 1.00 66.4±0.46s ? ?/sec 1.03 68.3±1.01s ? ?/sec indexing/Indexing songs with default settings 1.00 55.7±1.15s ? ?/sec 1.14 63.2±0.78s ? ?/sec indexing/Indexing songs without any facets 1.00 51.6±1.04s ? ?/sec 1.16 59.6±1.00s ? ?/sec indexing/Indexing songs without faceted numbers 1.00 55.3±1.09s ? ?/sec 1.13 62.8±0.38s ? ?/sec indexing/Indexing wiki 1.00 1006.6±26.89s ? ?/sec 1.00 1009.2±25.25s ? ?/sec indexing/Indexing wiki in three batches 1.00 1140.5±11.97s ? ?/sec 1.00 1142.0±9.97s ? ?/sec ``` We now have performance similar to what we had before for the non nested datasets 🎉 Co-authored-by: Tamo <tamo@meilisearch.com>
|
This PR was included in a batch that successfully built, but then failed to merge into main. It will not be retried. Additional information: {"message":"Required status check \"Specify breaking\" is expected.","documentation_url":"https://docs.github.com/articles/about-protected-branches"} |
Co-authored-by: Clément Renault <clement@meilisearch.com>
irevoire
force-pushed
the
flatten-what-is-needed
branch
from
cff986e
to
00f78d6
Compare
curquiza
approved these changes
Apr 14, 2022
This was referenced Apr 14, 2022
irevoire
added
the
performance
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Instead of flattening every object to write them in the flatenned sorter we now check if we needs to flatten the object or if we can insert it as-is.
We now have performance similar to what we had before for the non nested datasets 🎉