📶 🤳 Offline scanning (Tracker) #18
Comments
|
@teolemon I'm interested! |
|
👋 @monsieurtanuki And that's it. If you'd like to have an idea of additional complexities and sophistications, there's a (somewhat confusing) Google Doc: There's also the Swift code for the iOS implem if you'd like to have a look: openfoodfacts/openfoodfacts-ios@20723da |
|
also, wanna join the channel on slack ? |
|
(for the moment I have no access to the google docs file) The csv file is about 80Mb big, has 1,555,491 lines plus a header, and the header is
Some stats (with a Unix command like
That means that there are only 577,722 lines with an attached value. In addition to that, I saw tons of products in non-latin alphabets. For the moment I don't see the use case... |
|
aha… |
|
Non latin alphabets: that's because that's the whole db for all countries |
|
I've made a fix server side @monsieurtanuki |
That's my point about not understanding the use-case: what's the point of pre-downloading the data of all countries? (disclaimer: I have a smartphone with limited memory and disk space) What I had in mind is: I'm in a supermarket and I want to check the different scores of - say - breakfast cereals, but I have a bad internet connection. Why would I think about downloading in advance data about foods sold in Russia or Japan? There's no "just in case" argumentation. What may make sense is to select food categories and countries, and to focus on that. Maybe automatically: I scan a Kellog's in France, therefore there's a good chance I'm interested both in breakfast cereals and products sold in France, and that's the range of foods we could cache and refresh periodically. |
|
It was not a just in case choice, but a speed one:
I've shared the design document, where there's a discussion on how to implement properly all of that. |
|
@monsieurtanuki @stephanegigandet says we should start using this country-specific route as you suggested: |
|
About https://fr.openfoodfacts.org/api/v2/search?fields=code,product_name&page_size=1000:
I still don't understand the use-case: downloading 60Mb of world data for a limited added value (barcode => name). And we need to refresh it altogether time after time. @teolemon As I don't understand the purpose I think you should find someone else for this issue (e.g. someone who understands it) - that's why I removed my assignment to this issue about a year ago. |
|
Speed up or enable (in worst case scenario) scan in supermarkets. |
@teolemon what's the end goal? To pre-download just (barcodes => name) or the whole product. If it's just names, I agree with @monsieurtanuki. If it's the whole product it's going to take a lot of memory, and the database is always increasing. We would also need a syncing mechanism. Yes I did have issues where I scanned products and then I was just waiting for it to load, but my phone was also very heated up so I'm not really sure if the root cause was the heating up or network issues. Even if it was network issues, I would try to do a thorough analysis before working on this. Since this is a non trivial task I would first validate if we really need this, I would add some metrics on how often product fetch calls timeout and if it's higher than a permissible threshold I would go for this feature. If needed, I would also suggest compressing the response payload as much as possible.
|
|
@teolemon There are separate problems here:
cf. https://world.openfoodfacts.org/api/v0/product/093270067481501.json and its 19558 bytes ("all" fields here) a good product for you, in json{
"code": "093270067481501",
"product": {
"_id": "093270067481501",
"_keywords": ["fact", "open", "good", "beverage", "fruit", "you", "plant-based", "based", "eu", "organic", "meal", "trade", "soup", "product", "for", "and", "food", "vegetable", "fair", "france"],
"added_countries_tags": [],
"additives_n": 0,
"additives_old_n": 0,
"additives_old_tags": [],
"additives_original_tags": [],
"additives_tags": [],
"allergens": "",
"allergens_from_ingredients": "",
"allergens_from_user": "(en) ",
"allergens_hierarchy": [],
"allergens_lc": "en",
"allergens_tags": [],
"amino_acids_tags": [],
"brands": "Open Food Facts",
"brands_imported": "Open Food Facts",
"brands_tags": ["open-food-facts"],
"carbon_footprint_from_known_ingredients_debug": "en:butternut-squash 50% x 0.3 = 15 g - en:carrot 10% x 0.3 = 3 g - ",
"carbon_footprint_percent_of_known_ingredients": 60,
"categories": "Plant-based foods and beverages,Plant-based foods,Fruits and vegetables based foods,Meals,Soups,Vegetable soups",
"categories_hierarchy": ["en:plant-based-foods-and-beverages", "en:plant-based-foods", "en:fruits-and-vegetables-based-foods", "en:meals", "en:soups", "en:vegetable-soups"],
"categories_imported": "Plant-based foods and beverages, Plant-based foods, Fruits and vegetables based foods, Meals, Soups, Vegetable soups",
"categories_lc": "en",
"categories_properties": {
"agribalyse_proxy_food_code:en": "25903",
"ciqual_food_code:en": "25969"
},
"categories_properties_tags": ["all-products", "categories-known", "agribalyse-food-code-unknown", "agribalyse-proxy-food-code-25903", "agribalyse-proxy-food-code-known", "ciqual-food-code-25969", "ciqual-food-code-known", "agribalyse-known", "agribalyse-25903"],
"categories_tags": ["en:plant-based-foods-and-beverages", "en:plant-based-foods", "en:fruits-and-vegetables-based-foods", "en:meals", "en:soups", "en:vegetable-soups"],
"checkers_tags": [],
"cities_tags": ["saint-maur-des-fosses-val-de-marne-france"],
"code": "093270067481501",
"codes_tags": ["code-15", "093270067481xxx", "09327006748xxxx", "0932700674xxxxx", "093270067xxxxxx", "09327006xxxxxxx", "0932700xxxxxxxx", "093270xxxxxxxxx", "09327xxxxxxxxxx", "0932xxxxxxxxxxx", "093xxxxxxxxxxxx", "09xxxxxxxxxxxxx", "0xxxxxxxxxxxxxx"],
"compared_to_category": "en:vegetable-soups",
"complete": 0,
"completeness": 0.8,
"correctors_tags": ["org-openfoodfacts", "stephane"],
"countries": "France",
"countries_hierarchy": ["en:france"],
"countries_imported": "France",
"countries_lc": "en",
"countries_tags": ["en:france"],
"created_t": 1641562431,
"creator": "org-openfoodfacts",
"data_quality_bugs_tags": [],
"data_quality_errors_tags": [],
"data_quality_info_tags": ["en:packaging-data-incomplete", "en:ingredients-percent-analysis-ok", "en:carbon-footprint-from-known-ingredients-but-not-from-meat-or-fish"],
"data_quality_tags": ["en:packaging-data-incomplete", "en:ingredients-percent-analysis-ok", "en:carbon-footprint-from-known-ingredients-but-not-from-meat-or-fish"],
"data_quality_warnings_tags": [],
"data_sources": "Producers, Producer - openfoodfacts",
"data_sources_imported": "Producers, Producer - openfoodfacts, Producers, Producer - openfoodfacts",
"data_sources_tags": ["producers", "producer-openfoodfacts"],
"debug_param_sorted_langs": ["en", "fr"],
"ecoscore_data": {
"adjustments": {
"origins_of_ingredients": {
"aggregated_origins": [{
"origin": "en:france",
"percent": 100
}],
"epi_score": 93,
"epi_value": 4,
"origins_from_origins_field": ["en:france"],
"transportation_scores": {
"ad": 57,
"al": 0,
"at": 38,
"ax": 67,
"ba": 14,
"be": 85,
"bg": 21,
"ch": 69,
"cy": 40,
"cz": 48,
"de": 61,
"dk": 39,
"dz": 45,
"ee": 71,
"eg": 35,
"es": 37,
"fi": 69,
"fo": 62,
"fr": 100,
"gg": 78,
"gi": 4,
"gr": 49,
"hr": 30,
"hu": 26,
"ie": 47,
"il": 34,
"im": 50,
"is": 53,
"it": 47,
"je": 76,
"lb": 39,
"li": 64,
"lt": 63,
"lu": 82,
"lv": 71,
"ly": 56,
"ma": 60,
"mc": 52,
"md": 29,
"me": 37,
"mk": 29,
"mt": 57,
"nl": 77,
"no": 20,
"pl": 25,
"ps": 42,
"pt": 13,
"ro": 31,
"rs": 7,
"se": 15,
"si": 38,
"sj": 53,
"sk": 24,
"sm": 40,
"sy": 26,
"tn": 9,
"tr": 7,
"ua": 40,
"uk": 68,
"us": 0,
"va": 29,
"world": 0,
"xk": 28
},
"transportation_values": {
"ad": 9,
"al": 0,
"at": 6,
"ax": 10,
"ba": 2,
"be": 13,
"bg": 3,
"ch": 10,
"cy": 6,
"cz": 7,
"de": 9,
"dk": 6,
"dz": 7,
"ee": 11,
"eg": 5,
"es": 6,
"fi": 10,
"fo": 9,
"fr": 15,
"gg": 12,
"gi": 1,
"gr": 7,
"hr": 5,
"hu": 4,
"ie": 7,
"il": 5,
"im": 8,
"is": 8,
"it": 7,
"je": 11,
"lb": 6,
"li": 10,
"lt": 9,
"lu": 12,
"lv": 11,
"ly": 8,
"ma": 9,
"mc": 8,
"md": 4,
"me": 6,
"mk": 4,
"mt": 9,
"nl": 12,
"no": 3,
"pl": 4,
"ps": 6,
"pt": 2,
"ro": 5,
"rs": 1,
"se": 2,
"si": 6,
"sj": 8,
"sk": 4,
"sm": 6,
"sy": 4,
"tn": 1,
"tr": 1,
"ua": 6,
"uk": 10,
"us": 0,
"va": 4,
"world": 0,
"xk": 4
},
"values": {
"ad": 13,
"al": 4,
"at": 10,
"ax": 14,
"ba": 6,
"be": 17,
"bg": 7,
"ch": 14,
"cy": 10,
"cz": 11,
"de": 13,
"dk": 10,
"dz": 11,
"ee": 15,
"eg": 9,
"es": 10,
"fi": 14,
"fo": 13,
"fr": 19,
"gg": 16,
"gi": 5,
"gr": 11,
"hr": 9,
"hu": 8,
"ie": 11,
"il": 9,
"im": 12,
"is": 12,
"it": 11,
"je": 15,
"lb": 10,
"li": 14,
"lt": 13,
"lu": 16,
"lv": 15,
"ly": 12,
"ma": 13,
"mc": 12,
"md": 8,
"me": 10,
"mk": 8,
"mt": 13,
"nl": 16,
"no": 7,
"pl": 8,
"ps": 10,
"pt": 6,
"ro": 9,
"rs": 5,
"se": 6,
"si": 10,
"sj": 12,
"sk": 8,
"sm": 10,
"sy": 8,
"tn": 5,
"tr": 5,
"ua": 10,
"uk": 14,
"us": 4,
"va": 8,
"world": 4,
"xk": 8
}
},
"packaging": {
"non_recyclable_and_non_biodegradable_materials": 0,
"packagings": [{
"ecoscore_material_score": "81",
"ecoscore_shape_ratio": "1",
"material": "en:glass",
"recycling": "en:recycle",
"shape": "en:bottle"
}, {
"ecoscore_material_score": "76",
"ecoscore_shape_ratio": "0.1",
"material": "en:steel",
"recycling": "en:recycle",
"shape": "en:bottle-cap"
}],
"score": 78.6,
"value": -2
},
"production_system": {
"labels": ["en:eu-organic"],
"value": 15
},
"threatened_species": {}
},
"agribalyse": {
"agribalyse_proxy_food_code": "25903",
"co2_agriculture": "0.099163319",
"co2_consumption": "0.0079267752",
"co2_distribution": "0.025340163",
"co2_packaging": "0.098786634",
"co2_processing": "0.13521664",
"co2_total": "0.49805172",
"co2_transportation": "0.1316182",
"code": "25903",
"dqr": "2.42",
"ef_agriculture": "0.023834065",
"ef_consumption": "0.004012423",
"ef_distribution": "0.0095369029",
"ef_packaging": "0.014974093",
"ef_processing": "0.030870892",
"ef_total": "0.093465939",
"ef_transportation": "0.010237564",
"is_beverage": 0,
"name_en": "Soup, mixed vegetables, prepacked, to be reheated",
"name_fr": "Soupe aux légumes variés, préemballée à réchauffer",
"score": 97
},
"grade": "a",
"grades": {
"ad": "a",
"al": "a",
"at": "a",
"ax": "a",
"ba": "a",
"be": "a",
"bg": "a",
"ch": "a",
"cy": "a",
"cz": "a",
"de": "a",
"dk": "a",
"dz": "a",
"ee": "a",
"eg": "a",
"es": "a",
"fi": "a",
"fo": "a",
"fr": "a",
"gg": "a",
"gi": "a",
"gr": "a",
"hr": "a",
"hu": "a",
"ie": "a",
"il": "a",
"im": "a",
"is": "a",
"it": "a",
"je": "a",
"lb": "a",
"li": "a",
"lt": "a",
"lu": "a",
"lv": "a",
"ly": "a",
"ma": "a",
"mc": "a",
"md": "a",
"me": "a",
"mk": "a",
"mt": "a",
"nl": "a",
"no": "a",
"pl": "a",
"ps": "a",
"pt": "a",
"ro": "a",
"rs": "a",
"se": "a",
"si": "a",
"sj": "a",
"sk": "a",
"sm": "a",
"sy": "a",
"tn": "a",
"tr": "a",
"ua": "a",
"uk": "a",
"us": "a",
"va": "a",
"world": "a",
"xk": "a"
},
"score": 122,
"scores": {
"ad": 122,
"al": 114,
"at": 120,
"ax": 122,
"ba": 116,
"be": 122,
"bg": 117,
"ch": 122,
"cy": 120,
"cz": 121,
"de": 122,
"dk": 120,
"dz": 121,
"ee": 122,
"eg": 119,
"es": 120,
"fi": 122,
"fo": 122,
"fr": 122,
"gg": 122,
"gi": 115,
"gr": 121,
"hr": 119,
"hu": 118,
"ie": 121,
"il": 119,
"im": 122,
"is": 122,
"it": 121,
"je": 122,
"lb": 120,
"li": 122,
"lt": 122,
"lu": 122,
"lv": 122,
"ly": 122,
"ma": 122,
"mc": 122,
"md": 118,
"me": 120,
"mk": 118,
"mt": 122,
"nl": 122,
"no": 117,
"pl": 118,
"ps": 120,
"pt": 116,
"ro": 119,
"rs": 115,
"se": 116,
"si": 120,
"sj": 122,
"sk": 118,
"sm": 120,
"sy": 118,
"tn": 115,
"tr": 115,
"ua": 120,
"uk": 122,
"us": 114,
"va": 118,
"world": 114,
"xk": 118
},
"status": "known"
},
"ecoscore_extended_data": {
"impact": {
"ef_single_score_log_stddev": 0.0800212307271874,
"likeliest_impacts": {
"Climate_change": 0.012534284630903,
"EF_single_score": 0.00409189070065541
},
"likeliest_recipe": {
"en:butternut_squash": 50.0548448998379,
"en:carrot": 10.0109689799676,
"en:leek": 2.20570054418209,
"en:potato": 1.14589787403023,
"en:salt": 0.392984356259901,
"en:sweet_potato": 3.56236029441026,
"en:water": 32.7369328509879
},
"mass_ratio_uncharacterized": 0,
"uncharacterized_ingredients": {
"impact": [],
"nutrition": []
},
"uncharacterized_ingredients_mass_proportion": {
"impact": 0,
"nutrition": 0
},
"uncharacterized_ingredients_ratio": {
"impact": 0,
"nutrition": 0
},
"warnings": []
}
},
"ecoscore_extended_data_version": "4",
"ecoscore_grade": "a",
"ecoscore_score": 122,
"ecoscore_tags": ["a"],
"editors_tags": ["stephane", "ecoscore-impact-estimator", "org-openfoodfacts"],
"emb_codes": "FR 94.068.042 EC",
"emb_codes_imported": "FR 94.068.042 CE",
"emb_codes_orig": "EMB 53062, FR 94.068.042 EC",
"emb_codes_tags": ["fr-94-068-042-ec"],
"entry_dates_tags": ["2022-01-07", "2022-01", "2022"],
"environment_impact_level": "",
"environment_impact_level_tags": [],
"expiration_date": "",
"food_groups": "en:soups",
"food_groups_tags": ["en:fruits-and-vegetables", "en:soups"],
"generic_name": "",
"generic_name_en": "",
"generic_name_fr": "",
"grades": {},
"id": "093270067481501",
"informers_tags": ["org-openfoodfacts"],
"ingredients": [{
"id": "en:butternut-squash",
"percent": "50",
"percent_estimate": "50",
"percent_max": "50",
"percent_min": "50",
"rank": 1,
"text": "Butternut squash",
"vegan": "yes",
"vegetarian": "yes"
}, {
"id": "en:water",
"percent_estimate": 27.5,
"percent_max": 45,
"percent_min": "10",
"rank": 2,
"text": "water",
"vegan": "yes",
"vegetarian": "yes"
}, {
"id": "en:carrot",
"percent": "10",
"percent_estimate": "10",
"percent_max": "10",
"percent_min": "10",
"rank": 3,
"text": "carrots",
"vegan": "yes",
"vegetarian": "yes"
}, {
"id": "en:sweet-potato",
"percent_estimate": 5,
"percent_max": "10",
"percent_min": 0,
"rank": 4,
"text": "sweet potato",
"vegan": "yes",
"vegetarian": "yes"
}, {
"id": "en:leek",
"percent_estimate": 3.75,
"percent_max": "10",
"percent_min": 0,
"rank": 5,
"text": "leeks",
"vegan": "yes",
"vegetarian": "yes"
}, {
"id": "en:potato",
"percent_estimate": 1.875,
"percent_max": "10",
"percent_min": 0,
"rank": 6,
"text": "potatoes",
"vegan": "yes",
"vegetarian": "yes"
}, {
"id": "en:salt",
"percent_estimate": 1.875,
"percent_max": 8,
"percent_min": 0,
"rank": 7,
"text": "salt",
"vegan": "yes",
"vegetarian": "yes"
}],
"ingredients_analysis_tags": ["en:palm-oil-free", "en:vegan", "en:vegetarian"],
"ingredients_from_or_that_may_be_from_palm_oil_n": 0,
"ingredients_from_palm_oil_n": 0,
"ingredients_from_palm_oil_tags": [],
"ingredients_hierarchy": ["en:butternut-squash", "en:vegetable", "en:water", "en:carrot", "en:root-vegetable", "en:sweet-potato", "en:leek", "en:potato", "en:salt"],
"ingredients_n": 7,
"ingredients_n_tags": ["7", "1-10"],
"ingredients_original_tags": ["en:butternut-squash", "en:water", "en:carrot", "en:sweet-potato", "en:leek", "en:potato", "en:salt"],
"ingredients_percent_analysis": 1,
"ingredients_tags": ["en:butternut-squash", "en:vegetable", "en:water", "en:carrot", "en:root-vegetable", "en:sweet-potato", "en:leek", "en:potato", "en:salt"],
"ingredients_text": "Butternut squash 50%, water, carrots 10%, sweet potato, leeks, potatoes, salt.",
"ingredients_text_en": "Butternut squash 50%, water, carrots 10%, sweet potato, leeks, potatoes, salt.",
"ingredients_text_en_imported": "Butternut squash 50%, water, carrots 10%, sweet potato, leeks, potatoes, salt.",
"ingredients_text_fr": "",
"ingredients_text_with_allergens": "Butternut squash 50%, water, carrots 10%, sweet potato, leeks, potatoes, salt.",
"ingredients_text_with_allergens_en": "Butternut squash 50%, water, carrots 10%, sweet potato, leeks, potatoes, salt.",
"ingredients_text_with_allergens_fr": "",
"ingredients_that_may_be_from_palm_oil_n": 0,
"ingredients_that_may_be_from_palm_oil_tags": [],
"ingredients_with_specified_percent_n": 2,
"ingredients_with_specified_percent_sum": 60,
"ingredients_with_unspecified_percent_n": 5,
"ingredients_with_unspecified_percent_sum": 40,
"interface_version_created": "import_csv_file - version 2019/09/17",
"interface_version_modified": "20150316.jqm2",
"known_ingredients_n": 9,
"labels": "Organic,EU Organic,Fair trade",
"labels_hierarchy": ["en:organic", "en:eu-organic", "en:fair-trade"],
"labels_imported": "Organic, EU Organic, Fair trade",
"labels_lc": "en",
"labels_tags": ["en:organic", "en:eu-organic", "en:fair-trade"],
"lang": "en",
"lang_imported": "en",
"languages": {
"en:english": 4,
"en:french": 1
},
"languages_codes": {
"en": 4,
"fr": 1
},
"languages_hierarchy": ["en:english", "en:french"],
"languages_tags": ["en:english", "en:french", "en:2", "en:multilingual"],
"last_edit_dates_tags": ["2022-01-07", "2022-01", "2022"],
"last_editor": "ecoscore-impact-estimator",
"last_modified_by": "ecoscore-impact-estimator",
"last_modified_t": 1641563795,
"lc": "en",
"lc_imported": "en",
"link": "",
"main_countries_tags": [],
"manufacturing_places": "",
"manufacturing_places_tags": [],
"minerals_tags": [],
"misc_tags": ["en:nutrition-fruits-vegetables-nuts-estimate-from-ingredients", "en:nutrition-all-nutriscore-values-known", "en:nutriscore-computed", "en:ecoscore-extended-data-computed", "en:ecoscore-extended-data-version-4", "en:ecoscore-no-missing-data", "en:ecoscore-computed", "en:main-countries-new-product"],
"no_nutrition_data": "",
"nova_group": 3,
"nova_group_debug": " -- categories/en:meals : 3",
"nova_groups": "3",
"nova_groups_tags": ["en:3-processed-foods"],
"nucleotides_tags": [],
"nutrient_levels": {
"fat": "low",
"salt": "moderate",
"saturated-fat": "low",
"sugars": "low"
},
"nutrient_levels_tags": ["en:fat-in-low-quantity", "en:saturated-fat-in-low-quantity", "en:sugars-in-low-quantity", "en:salt-in-moderate-quantity"],
"nutriments": {
"carbohydrates": 4,
"carbohydrates_100g": 4,
"carbohydrates_unit": "g",
"carbohydrates_value": 4,
"carbon-footprint-from-known-ingredients_100g": 18,
"carbon-footprint-from-known-ingredients_product": 36,
"energy": 120,
"energy-kj": 120,
"energy-kj_100g": 120,
"energy-kj_unit": "kJ",
"energy-kj_value": 120,
"energy_100g": 120,
"energy_unit": "kJ",
"energy_value": 120,
"fat": 0.5,
"fat_100g": 0.5,
"fat_unit": "g",
"fat_value": 0.5,
"fiber": 3,
"fiber_100g": 3,
"fiber_unit": "g",
"fiber_value": 3,
"fruits-vegetables-nuts-estimate-from-ingredients_100g": 60,
"fruits-vegetables-nuts-estimate-from-ingredients_serving": 60,
"nova-group": 3,
"nova-group_100g": 3,
"nova-group_serving": 3,
"nutrition-score-fr": -3,
"nutrition-score-fr_100g": -3,
"proteins": 1,
"proteins_100g": 1,
"proteins_unit": "g",
"proteins_value": 1,
"salt": 0.4,
"salt_100g": 0.4,
"salt_unit": "g",
"salt_value": 0.4,
"saturated-fat": 0.2,
"saturated-fat_100g": 0.2,
"saturated-fat_unit": "g",
"saturated-fat_value": 0.2,
"sodium": 0.16,
"sodium_100g": 0.16,
"sodium_unit": "g",
"sodium_value": 0.16,
"sugars": 0.5,
"sugars_100g": 0.5,
"sugars_unit": "g",
"sugars_value": 0.5
},
"nutriscore_data": {
"energy": 120,
"energy_points": 0,
"energy_value": 120,
"fiber": 3,
"fiber_points": 3,
"fiber_value": 3,
"fruits_vegetables_nuts_colza_walnut_olive_oils": 60,
"fruits_vegetables_nuts_colza_walnut_olive_oils_points": 1,
"fruits_vegetables_nuts_colza_walnut_olive_oils_value": 60,
"grade": "a",
"is_beverage": 0,
"is_cheese": 0,
"is_fat": 0,
"is_water": 0,
"negative_points": 1,
"positive_points": 4,
"proteins": 1,
"proteins_points": 0,
"proteins_value": 1,
"saturated_fat": 0.2,
"saturated_fat_points": 0,
"saturated_fat_ratio": 40,
"saturated_fat_ratio_points": 6,
"saturated_fat_ratio_value": 40,
"saturated_fat_value": 0.2,
"score": -3,
"sodium": 160,
"sodium_points": 1,
"sodium_value": 160,
"sugars": 0.5,
"sugars_points": 0,
"sugars_value": 0.5
},
"nutriscore_grade": "a",
"nutriscore_score": -3,
"nutriscore_score_opposite": 3,
"nutrition_data": "on",
"nutrition_data_per": "100g",
"nutrition_data_per_imported": "100g",
"nutrition_data_prepared": "",
"nutrition_data_prepared_per": "100g",
"nutrition_data_prepared_per_imported": "100g",
"nutrition_grade_fr": "a",
"nutrition_grades": "a",
"nutrition_grades_tags": ["a"],
"nutrition_score_beverage": 0,
"nutrition_score_warning_fruits_vegetables_nuts_estimate_from_ingredients": 1,
"nutrition_score_warning_fruits_vegetables_nuts_estimate_from_ingredients_value": 60,
"obsolete": "",
"obsolete_imported": "0",
"obsolete_since_date": "",
"origin": "France",
"origin_en": "France",
"origin_en_imported": "France",
"origin_fr": "",
"origins": "France",
"origins_hierarchy": ["en:france"],
"origins_imported": "France",
"origins_lc": "en",
"origins_tags": ["en:france"],
"other_nutritional_substances_tags": [],
"owner": "org-openfoodfacts",
"owner_fields": {
"brands": 1641562722,
"carbohydrates": 1641562722,
"categories": 1641562722,
"countries": 1641562722,
"data_sources": 1641562722,
"emb_codes": 1641562722,
"energy-kj": 1641562722,
"fat": 1641562722,
"fiber": 1641562722,
"ingredients_text_en": 1641562722,
"labels": 1641562722,
"lang": 1641562722,
"lc": 1641562722,
"nutrition_data_per": 1641562722,
"nutrition_data_prepared_per": 1641562722,
"obsolete": 1641562722,
"origin_en": 1641562431,
"origins": 1641562722,
"packaging_text_en": 1641562722,
"product_name_en": 1641562722,
"product_name_fr": 1641562722,
"proteins": 1641562722,
"quantity": 1641562722,
"salt": 1641562722,
"saturated-fat": 1641562722,
"sugars": 1641562722
},
"owners_tags": "org-openfoodfacts",
"packaging": "",
"packaging_tags": [],
"packaging_text": "Glass bottle to recycle\r\nSteel cap to recycle",
"packaging_text_en": "Glass bottle to recycle\r\nSteel cap to recycle",
"packaging_text_en_imported": "Glass bottle to recycle\nSteel cap to recycle",
"packaging_text_fr": "",
"packagings": [{
"material": "en:glass",
"recycling": "en:recycle",
"shape": "en:bottle"
}, {
"material": "en:steel",
"recycling": "en:recycle",
"shape": "en:bottle-cap"
}],
"photographers_tags": [],
"pnns_groups_1": "Fruits and vegetables",
"pnns_groups_1_tags": ["fruits-and-vegetables", "known"],
"pnns_groups_2": "Soups",
"pnns_groups_2_tags": ["soups", "known"],
"popularity_key": 0,
"product_name": "A good product for you",
"product_name_en": "A good product for you",
"product_name_en_imported": "A good product for you",
"product_name_fr": "Un bon produit pour vous",
"product_name_fr_imported": "Un bon produit pour vous",
"product_quantity": "200",
"purchase_places": "",
"purchase_places_tags": [],
"quantity": "200 g",
"quantity_imported": "200 g",
"removed_countries_tags": [],
"rev": 5,
"scores": {},
"sources": [{
"fields": ["product_name_en", "product_name_fr", "packaging_text_en", "quantity", "brands", "categories", "labels", "emb_codes", "countries", "origin_en", "data_sources", "nutrition_data_per", "nutrition_data_prepared_per", "ingredients_text_en", "nutrients.carbohydrates_unit", "nutrients.carbohydrates_value", "nutrients.fat_unit", "nutrients.fat_value", "nutrients.fiber_unit", "nutrients.fiber_value", "nutrients.proteins_unit", "nutrients.proteins_value", "nutrients.salt_unit", "nutrients.salt_value", "nutrients.saturated-fat_unit", "nutrients.saturated-fat_value", "nutrients.sugars_unit", "nutrients.sugars_value"],
"id": "org-openfoodfacts",
"images": [],
"import_t": 1641562431,
"manufacturer": 1,
"name": "openfoodfacts",
"url": null
}, {
"fields": ["origins"],
"id": "org-openfoodfacts",
"images": [],
"import_t": 1641562503,
"manufacturer": 1,
"name": "openfoodfacts",
"url": null
}, {
"fields": ["emb_codes"],
"id": "org-openfoodfacts",
"images": [],
"import_t": 1641562722,
"manufacturer": 1,
"name": "openfoodfacts",
"url": null
}],
"states": "en:to-be-completed, en:nutrition-facts-completed, en:ingredients-completed, en:expiration-date-to-be-completed, en:packaging-code-completed, en:characteristics-to-be-completed, en:origins-completed, en:categories-completed, en:brands-completed, en:packaging-to-be-completed, en:quantity-completed, en:product-name-completed, en:photos-to-be-uploaded",
"states_hierarchy": ["en:to-be-completed", "en:nutrition-facts-completed", "en:ingredients-completed", "en:expiration-date-to-be-completed", "en:packaging-code-completed", "en:characteristics-to-be-completed", "en:origins-completed", "en:categories-completed", "en:brands-completed", "en:packaging-to-be-completed", "en:quantity-completed", "en:product-name-completed", "en:photos-to-be-uploaded"],
"states_tags": ["en:to-be-completed", "en:nutrition-facts-completed", "en:ingredients-completed", "en:expiration-date-to-be-completed", "en:packaging-code-completed", "en:characteristics-to-be-completed", "en:origins-completed", "en:categories-completed", "en:brands-completed", "en:packaging-to-be-completed", "en:quantity-completed", "en:product-name-completed", "en:photos-to-be-uploaded"],
"stores": "",
"stores_tags": [],
"teams": "pain-au-chocolat,shark-attack",
"teams_tags": ["pain-au-chocolat", "shark-attack"],
"traces": "",
"traces_from_ingredients": "",
"traces_from_user": "(en) ",
"traces_hierarchy": [],
"traces_lc": "en",
"traces_tags": [],
"unknown_ingredients_n": 0,
"unknown_nutrients_tags": [],
"update_key": "ing2",
"vitamins_tags": []
},
"status": 1,
"status_verbose": "product found"
} |
|
@teolemon @jasmeet0817 For the low connectivity use-case, I suggest that we add in dev mode a switch between the current set of extracted fields, and a minimum set of fields. And then we send @jasmeet0817 go shopping :) (bad luck, it's crowded on Saturdays) Anyway, it's a bit paradoxical to use scores for fast assessment of products ("it's A, it's D") and flood users with tons of the detailed data from which the scores were computed. In a "more about it..." button, fair enough. But when you're in a busy supermarket with low (or expensive) connectivity, the faster the better. And more or less that's similar to the OP: a downgraded mode, and a full mode. The difference is that "my" downgraded mode does not handle offline queries and doesn't imply pre-loading tons of data. Following is the current list of product fields we extract for products:
PS: my understanding of offline scanning hasn't changed, as described in the following video :) |
|
|
Probably this to display only the summary card ? |
|
We could remove images alltogether. What would be the cost of having 1K to 10K of those ? Edit: 2,7kb*1-10k >> 30mb (possibly times 3 to 4 if we want to show which images are available) |
|
I've just simplified again the attributes, this time "à la SQL", and it looks less poetic but much more compact (should take half the space - for attributes): SQL versionAs I said before you don't change a SQL database structure like you do with JSON files, therefore we should be very careful with what we really want:
|
|
@teolemon @g123k continuing from #5392 I would compute the number of products (1M for France?) multiplied by the size of each product (in a mini version: barcode, name, main image), and then I'll realize that the server doesn't accept more than 10 queries a minute. That would also mean a specific MiniProduct table. Actually I don't know the use-case, or more precisely: how minified should the product version be? |
|
I think we should drop images altogether, in favor of less size or more info (eg image status, attributes…)
We should steer away from search queries, and generate a one-size fits all mini dump for each country with product_name, nutriscore, ecoscore, nova_group, and possibly: attributes, states. We could slice the mini-dump based on user prefs (and remove some attributes) |
That could even be the subtitle of a new app!
In both cases, if we want to put that inside Smoothie that should be in distinct pages, at least in a first step:
|
|
@teolemon you can emulate the no-network-scan-session after downloading a significative set of products (cf. dev mode / offline) and switching to flight mode. Immediate results. |
What
Tasks
Why
Part of
The text was updated successfully, but these errors were encountered: