feat(Django): daily sync of products from OFF

[raphodn]

What

Installed django-q2 : https://django-q2.readthedocs.io/

poetry add django-q2 blessed croniter

• with 2 extra packages
• using the DB as message broker
• why this package ? See Rewrite the app in Python Django 🚀  #355 for a comparison

Commands

Create scheduled task (how to create automatically ?)

from django_q.models import Schedule

Schedule.objects.create(func="open_prices.common.openfoodfacts.import_product_db_full", schedule_type=Schedule.CRON, cron="0 15 * * *")

Run django-q2 in the background

poetry run python manage.py qcluster

Sync product flavor manually

// python manage.py shell_plus
import openfoodfacts
from open_prices.common import openfoodfacts as common_openfoodfacts

common_openfoodfacts.import_product_db(flavor=openfoodfacts.Flavor.opf)

[raphodn]

in the previous script we had if buffer_len % batch_size == 0:
but 0 % 1000 == 0
so it seems like it was commiting the db transactions for EVERY product and not in batch 😅

that's why I added if len(products_to_create_or_update)