Properties related to non-food products should not be returned for food products (and vice-versa)

[CharlesNepote]

There are now more than 75 properties created by users (2023-01).

Some of them are dedicated to food products.

• packaging:nutriscore:multiple
• ingredients:garlic
• nutrition_facts:multiple
• [...]

Some of them could be used for many, or even nearly any, kind of products:

• has_funny_barcode
• packaging:has_character
• same_product
• multipack
• recalled_product
• [...]

Some of them could be used for different kind of products including food:

• conservation:temperature
• color_of_the_cap
• [...]

And some of them are clearly not relevant for food products and even sometimes exclusively dedicated to some categories:

• interfaces
• french_reparability_index
• memory_size
• main_camera_resolution_in_million_pixels

These properties are visible at three places:

• In the page listing all the properties: https://world.openfoodfacts.org/properties
• While typing chars in the property field of the folksonomy engine form: an autocomplete UI display possible properties while you type
• On the Open Food Facts wiki: https://wiki.openfoodfacts.org/Folksonomy/Property

I think that deploying Folksonomy Engine to sister sites (see #62) will quickly lead to anarchy if we don't address this topic.

[CharlesNepote]

Solution 1

The first solution is to use prefixes to identify which things the property applies to. There are no changes in the API. When suggesting properties, the UI can filter the relevant ones.

Case 1: the property is dedicated to food products. Eg. packaging:nutriscore:multiple.

• Entered in Open Food Facts: food:packaging:nutriscore:multiple
• Stored in Folksonomy Engine DB: food:packaging:nutriscore:multiple
• The property is not suggested for other kind of products, in Open Beauty Facts or Open Products Facts, because there is the food prefix.

Case 2: the property is dedicated to general products and beauty products, but not food products. Eg. flammable.

• Either:
  • Two properties are created Open Products Facts or Open Beauty Facts as general:flammable and beauty:flammable.
  • One flammable property is created but there is no way to filter it for food products (except hard-coding it in the UI)

PROs:

• no coding required in the backend

CONs:

• for the user creating a new property, it can be difficult to know if he/she has to add a prefix or not
• the case where a property is suited for two product's types but not all, is not managed

Solution 2

Create another DB which lists all properties and their targeted products.

property	food	beauty	general
packaging:nutriscore:multiple	true
flammable		true	true
conservation:temperature	true	true	true

PROs:

• short property names
• can evolve by the time

CONs:

• we need to either change Folksonomy Engine backend, either create another one
• needs either another DB with its own interfaces, either multiple API call

Solution 3

Create a taxonomy for the properties. A taxonomy allows complex cases. Eg.:

<food_properties
en:packaging:nutriscore:multiple
fr:emballage:nutriscore:multiple
en:description:There are more than one Nutri-Score displayed on the packaging. The values should be the letters separated by commas.
en:example:a,b
xx:regexp_control:[a-e],[a-e](,[a-e])*

<beauty_properties
<general_properties
en:flammable

PROs:

• allows much more usages: descriptions, translation, etc.
• manage the case where a category can fit to some products' types but not all

CONs:

• It's harder to specify properties usages: it is necessary to add it to the taxonomy.
• New properties, before entering the taxonomy, will be displayed by default for every products' types (food, beauty, etc.) (we could create an entry automatically based on the website where the new property has been entered)
• We have to modify the UI code to filter properties based on the taxonomy.

[benbenben2]

Do you think that we could have option to add a column in the database for the sources (Open Food Facts, Open Beauty Facts, etc.)?

Like this, flammable entered in both Open Food Facts and Open Beauty Facts would lead to 2 rows in the database.

Then, we could either look at flammable for Open Food Facts or for Open Beauty Facts or for both.

[alexgarel]

I like the idea of @benbenben2 because it does not require more work. It just separates usages. Until a property is added to a product type, it won't be suggested.

And API call can decide to filter or not on database type.

[CharlesNepote]

@benbenben2 @alexgarel yes this solution is quite simple, it's a bit like solution 1, but with an extra field.

it does not require more work

It does:

• agree with the value: off, obf, opf?
• create the field
• manage it in API calls
• populate it (I guess we could default to "OFF" as there are very few products related to OBF, OPF, etc.)
• modify the front-end
• manage the documentation of the properties: in this solution two properties with the same name could have a slightly different meanings, and the properties documentation won't make the difference. This is the most difficult part IMHO.

Solution 1 seems more simpler: we only need to modify the front-end (+ perhaps move few wiki pages). But it has the drawback to use long names, which can be different in the database and in the front-end...

[benbenben2]

Maybe I do not get the point of the folksonomy engine.
Isn't it a tool literally for everyone?
If we impose rules and conventions based on a documentation, isn't it a risk to make it not for everyone?

[alexgarel]

@benbenben2 this is the way openStreetMaps works !

(as we democratize Folksonomy Engine, we may also add some widgets for common properties)

[CharlesNepote]

@benbenben2 like in OpenStreetMap project, everyone can create his/her own properties, but the properties are much efficient if they are also used by other people. Documenting the properties is not mandatory, but the undocumented properties get less chance to be reused.

[teolemon]

I believe this might be down to category level, and might approach a dual statistical approach (this category is often used with those properties) and taxonomized approach (once we manage a way to generate associated properties at scale, possibly using LLMs - see: openfoodfacts/openfoodfacts-ai#296)

[teolemon]

I don't see this as a strict blocker for the deployment.

[alexgarel]

Looking at it again, @CharlesNepote another way could be to keep solution 1 adding a prefix by product type (plus an eventual general one) but:

1. add a prefix filter (multiple prefix possible) to the /keys API (very easy to do)
2. abstract this prefix thing at the javascript and interface level (the user does not see the prefix, but a meaningful interface: prefix is not visible, and maybe you separate items between food and general ones (two part of the table), when you autocomplete you search on off+general prefix, but you display it as a property type)

[teolemon]

There are properties common to food and non food, thus I would really use ML/Statistics/categories

[alexgarel]

There are properties common to food and non food, thus I would really use ML/Statistics/categories

It's a good point, for this to happen we would need to have a statistic by product type/category on a different table, that maybe we update from time to time. It would solve the problem by an external factor (the stats table). If done well, updating this stat table can be done in a differential way (only fetch products newly updated).

The new table will contain one entry by property + product type (+ eventually category) with the number of products.

Every day, we get the newly added / removed properties, we fetch corresponding products to change statistics according to them.

To also account for product changes we could use open food facts events to recompute changed products (a bit trickier), or, easier, every now and then, update stats from the data dump.

I think this is not hard to implement and has the big advantage to offer more flexibility, and it can help being even more specific when we propose properties, based on the product type.

Although this will work better if we don't mess keys and values

[teolemon]

@stephanegigandet has already coded something very similar for packaging components