An application to easily manage your food to reduce waste and shop economically.
- Description
- Video Demo
- App Usage
- Solution Architecture
- Installation / Deployment
- Project Roadmap
- Authors
- License
One third of the world’s food is wasted, with Australia's largest driver of food waste being consumer households. Reducing food waste plays a critical component in tackling climate change as the processing and disposal of food wastes resources such as water, land, energy and increases carbon dioxide emissions. It is more important now, than ever, to ensure we have responsible consumption and production (UN Sustainable Development Goal 12).
Existing food tracking solutions all suffer from the same problem: as the average consumer purchases a large number of food products each week, adding and managing items requires too much manual effort. As a result, many consumers are discouraged from incorporating food tracking and waste management solutions into their daily lifestyle.
Plenti is a mobile application that aims to address the UN Sustainable Development Goal of Responsible Consumption and Production. We apply the two principles of preventing food waste used by many small cafes - inventory tracking and waste measurement. This solution aims to remind consumers about what’s in their kitchen by providing greater visibility on food items that can be tucked away for too long, as well as allowing consumers to measure what they’re throwing out to access personalised insights on how to improve their wasteful behaviours. Where consumers previously needed to track their inventory on lists or spreadsheets, our solution aims to provide an easy way for consumers to stay on top of their groceries and reduce waste. An added social element motivates users and aims to reinforce that tackling food waste is a collective effort.
Plenti's AI-driven categorisation, expiry approximation and receipt scanning features improve on existing solutions by providing more in-depth tracking and insights while requiring significantly less manual input. This is simply our first step towards making food tracking an easy and efficient process which consumers can readily adopt. In the future, we plan to introduce interfacing with digital receipt services, and acquire partnerships with supermarkets to achieve seamless integration with the consumer shopping experience.
To do so, Plenti digitises the kitchen inventory and uses expiry date approximations to remind people to finish their food on time. A user can input data into the application through receipt scanning, which scans products off a store receipt through a text recognition model, and classifies specific store products into general food item categories through a natural language classifier implemented using IBM Watson.
Insights on one’s level of food waste in terms of amounts, types of food wasted and costs are also provided to help consumers make smarter decisions. Users can see both personal and community insights, while a leaderboard displays users that have most improved their food savings. This quantifies the positive environmental impact each person can make while using the app and integrates social elements to engage consumers.
To get started with using the app, first create an account. This can be done by clicking the register button on the login screen, and filling in the required details.
Items can be added to the inventory using the add button at the top of the screen. By default, the item will be added with the auto-classify functionality turned on. When auto-classify is on, only the name, price and quantity fields for the item need to be filled out; this information is fed to the classification model, and the results of the classication are used to automatically determine additional information on the product (e.g. food class, category, expiry date, emoji etc.). The auto-classify feature is designed to significantly reduce manual user input. This feature can also be turned off by toggling the auto-classify button at the top of the add item screen. Plenti's classification model currently supports 54 different food classes, and is trained on product data collected from various Australian supermarkets.
Clicking on an item in the inventory screen will bring up the edit screen. Here, all the stored information on the item can be viewed, and modifications can be made to any of the item fields.
A common method to deal with uneaten food which is almost at expiry is to store it in the freezer. Plenti provides the freeze feature to continue tracking food which has been moved to the freezer. Updating a food item to the frozen state will appropriately modify the expiry approximation. To freeze an item, swipe left on the item and click the freeze button.
Each item in the inventory displays a coloured circle representing the time left until the item expires, which is calculated using the purchase date, the classifier model predictions, and a database of food expirations for various food types. These expiry approximations may not be completely representive of the actual expiry date, as the actual expiry date of an item is subject to many additional factors which aren't accounted for in our model. Instead, these indicators should serve solely as a reminder to check on food items that might be expiring soon.
There are three actions for removing items from the inventory: eat, throw out, and delete. Eat should be used for items which the user has consumed. Throw out moves the item to the bin, and should be used for food which has been thrown away by the user. Delete should be used for items which have been mistakenly added to the inventory, such that the removal of the item should not be counted for waste tracking.
The dashboard screen provides insights into the user's food waste, a bin feature to view wasted food, and a community leaderboard.
The food waste overview section of the dashboard screen displays user food waste trends for the month, and shows a comparison with food wasted in previous months. By clicking on the month, the user can select previous months to revisit their food waste history. The bar below the food waste trends visualises the distribution of wasted food across different food categories, so the user can monitor which types of food are contributing to the most waste.
The bin allows for users to view food items which were recently wasted, and retrieve items back to the inventory in case an item was mistakenly moved to the bin.
The ranking/leaderboard screen is accessed by clicking the ranking button on the dashboard. This displays a community leaderboard of food waste rankings to further incentivise users to contribute towards the collective effort to reduce food waste.
The settings functionality is currently not implemented. This page will be used for users to manage their account details, and will be implemented before the application is made publicly available.
- The user interacts with a mobile application to view and manage their food inventory.
- The React Native application communicates with the back end through an API managed by API Gateway.
- API Gateway accesses a serverless back end implemented through various actions in Cloud Functions.
- Each Cloud Functions action may interact with other cloud services to perform data processing and store information in the database.
- The Cloudant database stores user data for back-end access.
- An optical character recognition (OCR) model is deployed in the cloud to convert user-scanned receipt images to text data.
- Watson Natural Language Classifier is used to semantically classify products into different food categories.
Our repository is split between our mobile application built with Expo and React Native and our backend deployed as individual functions on IBM's Cloud Functions.
To get started with our application you will need to configure your IBM cloud.
Create an IBM cloud account and set up the following services.
Create a Cloudant database and a non-partitioned table called "users". Create a service credential and note down the apiKey and Service URL.
Create a cloud functions namespace and note down all the environment variables such as region and namespace. Ensure you have downloaded the IBM CLI tool on your device, downloaded the cloud functions plugin set the correct namespace.
Create a Watson Natural Language Classifier instance. To train the classifier, first load a dataset of training samples and labels in a csv format into your data assets. Our dataset consists of approximately 5000 training samples spanning 54 different food classes, collected from various Australian supermarkets. This dataset can be accessed in the dataset folder of this repository. After loading the dataset, train the model on the data. This training process may take some time. Note down the API key, service URL and classifier ID for the model instance, as this is needed for external access to the classifier.
Follow instructions to deploy your own OCR in your IBM cloud environment using the Kubernetes Cluster service or use the sample provided by IBM for testing purposes. Note down the endpoint to access the service.
Simply clone the repo and inside /backend/actions/ copy the .env.sample to a .env file and fill out the specified information.
Now inside /backend run an npm install and for each folder in actions run npm run build (this builds the index.js file with dependencies into a single file) and run npm run deploy which will deploy each of the functions to your IBM Cloud Functions Namespace.
Inside the IBM Cloud Functions dashboard, head to API and click create API. From there configure like so. Or optionally import the API.yaml file in the root of the project.
Go to /app and copy app.json.sample to app.json and fill out the apiUrl from the Cloud Functions page and add the OCR endpoint.
We're all set up now!
Simply go to /app and run npm install and npm start to view the application on your device with Expo Go.
With funding, we will continue to develop our application to ensure it will be ready for launch.
Our goal is to launch Plenti on app stores in late 2021, around October/November.
Our app will operate with a freemium model, where users can purchase our pro offerings such as syncing to multiple accounts.
Digital receipts are the future. We aim to integrate this into Plenti by building partnerships with digital receipt fintechs like Slyp. To further streamline the process for our users, we will seek partnerships to build further direct connectivity to retailer’s Point-of-Sale systems.
Once we collect additional data, Plenti can be scaled to different countries for household food waste. We also envision the technology to be applied to different industries, such as the hospitality industry or other inventory management use cases for perishable items.
Food problems are planet problems, and our vision is to become the solution for sustainable decision-making. There is plenti of food in our homes, so let’s all adopt clever eating for greener living 🌱
This project is licensed under the Apache 2 License - see the LICENSE for details.