discuss data storage priorities

[berteh]

some thoughts relating to data storage

• be able to use reference plants data (eg from scrapers you already made) loaded by the app.
• all permadesigner to define/refine his own pet plants, local varieties or local calendars that vary a bit from the reference. They should not replace but rather 'superseed' the reference data (like an 'inheritance' mechanism). That should be usable/copyable/refinable in the app in some fast way, maybe with an ability to 'suggest for curation' to the reference dataset, either automatically (as a 'give back to community' mechanism) or manually.

About the data storage technology : how big do you expect it will become? How many plants and designs would be stored in the user's app?

If the sum is less than 2000 then the technology doesn't matter much I think.

Do you need instant online sync amongst all users? I think an update every once in a while is enough and thus wouldn't go for online CRUD like restdb, but I'd rather stick to offline & local file with some import/update mechanism.

In case of local files : SQLite is easy enough but really low level, opt. use sqlalchemy for ORM or MVC or later backend change... but I wouldn't invest too much time there in now. Parquet file is a good alternative to CSV, that natively loads to pandas data frame.

[berteh]

from discussions:

• size database ~ 10k plants
• size designs ~ 30-200 pants

pandas dataframe for storing the design
sqlite to store the plants