This repository has been archived by the owner on Apr 4, 2023. It is now read-only.
Implement documents API #324
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MarinPostma
force-pushed
the
obkv-documents
branch
from
642008e
to
65df5fb
Compare
MarinPostma
force-pushed
the
obkv-documents
branch
4 times, most recently
from
bc7b676
to
a1ce3cd
Compare
Kerollmops
suggested changes
Sep 7, 2021
MarinPostma
force-pushed
the
obkv-documents
branch
2 times, most recently
from
16b641c
to
46b80b8
Compare
Kerollmops
suggested changes
Sep 11, 2021
MarinPostma
force-pushed
the
obkv-documents
branch
3 times, most recently
from
a99dc21
to
6de1b41
Compare
ManyTheFish
suggested changes
Sep 20, 2021
ManyTheFish
suggested changes
Sep 20, 2021
MarinPostma
force-pushed
the
obkv-documents
branch
from
b3d61e0
to
842c817
Compare
ManyTheFish
previously approved these changes
Sep 21, 2021
MarinPostma
force-pushed
the
obkv-documents
branch
from
842c817
to
6b3c260
Compare
|
I have rebased and squashed this pr |
MarinPostma
force-pushed
the
obkv-documents
branch
from
6b3c260
to
0f6c053
Compare
Kerollmops
suggested changes
Sep 21, 2021
document reader transform remove update format support document sequences fix document transform clean transform improve error handling add documents! macro fix transform bug fix tests remove csv dependency Add comments on the transform process replace search cli fmt review edits fix http ui fix clippy warnings Revert "fix clippy warnings" This reverts commit a1ce3cd. fix review comments remove smallvec in transform loop review edits
MarinPostma
force-pushed
the
obkv-documents
branch
from
0f6c053
to
aa6c5df
Compare
Kerollmops
approved these changes
Sep 21, 2021
Merged
bors bot
added a commit
that referenced
this pull request
Sep 22, 2021
364: Fix all the benchmarks r=Kerollmops a=irevoire #324 broke all benchmarks. I fixed everything and noticed that `cargo check --all` was insufficient to check the bench in multiple workspaces, so I also updated the CI to use `cargo check --workspace --all-targets`. Co-authored-by: Tamo <tamo@meilisearch.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pr implement the intermediary document representation for milli. The JSON, JSONL and CSV formats are replaced with the format instead, to push the serialization duty on the client side.
The
documentsmodule contains the interface to the new document format:DocumentsBuilderallows the creation of a writer backed document addition, when documents are added either one by one, or as arrays of depth 1. This is made possible by the fact that the seriliazer used by theadd_documentsmethods only accepts[Object]andObject. The related serialization logic is located in theserde.rsfile.DocumentsReaderallows to to iterate over the documents created by aDocumentsBuilder. A call tonext_document_with_indexreturns the next obkv reader in the document addition, along with a reference to the index used to map the field ids in the obkv reader to the field namesAll references to json, jsonl or csv in the tests have been replaced with the
documents!macro, works exaclty like theserde_json::jsonmacro, as a convenient way to create aDocumentsReader.Rewrote the search cli, to the
clicrate, to also allow index manipulation. This only offers basic functionalities for now, but is meant to be easier to extend than http uiblocked by #308