Nested fields

[irevoire]

For the following document:

{
  "id": 1,
  "person": {
    "name": "tamo",
    "age": 25,
  }
}

Suppose the user sets person as a filterable attribute. We need to store person in the filterable obviously. But we also need to keep track of person.name and person.age somewhere.
That’s where I changed a little bit the logic of the engine.

Currently, we have a function called faceted_field that returns the union of the filterable and sortable.
I renamed this function in user_defined_faceted_field. And now, when we finish indexing documents, we look at all the fields and see if they « match » a user_defined_faceted_field.
So in our case:

• does id match person: 🔴
• does person.name match person: 🟢
• does person.age match person: 🟢

And thus, we insert in the database the following faceted fields: person, person.name, person.age.

The good thing about that solution is that we generate everything during the indexing phase, and then during the search, we can access our field without recomputing too much globbing.

---

Now the bad thing is that I had to create a new db.

And if that was only one db, that would be ok, but actually, I need to do the same for the:

• Displayed attributes
• Attributes to retrieve
• Attributes to highlight
• Attribute to crop

@Kerollmops
Do you think there is a better way to do it?
Apart from all the code, can we have a problem because we have too many dbs?

[Kerollmops]

Could you fix the benchmarks, please?
https://github.com/meilisearch/milli/runs/5676613548?check_suite_focus=true#step:8:248

[Kerollmops]

I think that declaring these new databases isn't an issue, I find that this is a good way to do that. I haven't finished my review, I need to continue on the transform.rs file but I prefer publishing that than nothing.

[Kerollmops]

Could you please publish the benchmarks you ran recently here? And remove all of the unwraps (or at least the problematic ones), this is probably something that you forgot to change 😃

[Kerollmops]

It could probably be an issue if we no more call this callback, don't you think? It is even easier to use it now that we have only have one loop, no?

[irevoire]

I put the call back in the loop that compute the field distribution. Not sure it's the best but it's something.

[irevoire]

Maybe we should remove this callback entirely actually? Wdyt?

[Kerollmops]

Yeah, remove it if you don't use it, it would be better :)

[Kerollmops]

I approve this PR, it look good to me! Thanks for having added the flatten-serde-json crate in the repo along with the fuzzer! Now, we will see what people think about it in the RC0+ of Meilisearch 😃

bors merge

[bors]

Build succeeded:

• Run Rustfmt
• Tests on macos-latest with stable
• Tests on ubuntu-18.04 with stable
• Tests on windows-latest with stable