Skip to content

Latest commit

 

History

History
115 lines (97 loc) · 11.8 KB

App.md

File metadata and controls

115 lines (97 loc) · 11.8 KB
Raw

FoodieShoot->Documentation->App

The Application

All the files for the application can be found in Applications/FoodDetection. If you notice, in Applications we have two folders, FoodDetection and FoodClassification, when we started the project we were doing image clasification which was not what needed to be implemented, that folder (FoodeClassification) is deprecated and remains only as a reference.

To incorporate the Tensorflow model in our app we needed to implement the object detection android API, we used Tensorflow's example app as a reference for our implementation, using the Tensorflow object detection api we had to set our target SDK to be of at least level 28.

For the design the initial plans can be assed here we would seperate our app into two major parts:

  • Anonymous usage
  • Authenticated usage

In the Anonynous usage we woul provide the login and sign up screens and provided an option to use the camera for a quick real time prediction, if the user wanted to analyse the foods it would need to authenticate.

We will now describe the overall structure of our app and will later present the instructions and demo of the app

Structure

The app will need to interact with the user's camera, will have to perform json requests and parse json requests. Though we will perform request to the server we also want to provide imediate caloric info, to do so we store a json file with the calories for each ofthe model's class which then is loaded to memory, here is the overall structure:

  • The Auxiliar/ folder is responsible for the auxilar classes for network requests and response parser as well as form validators
  • The Configs/Configurations.java file contains all the server major configurations, the necessary endpoints and the logout request
  • The Authentication/ folder has the login and sign up classes
  • The Authenticated/ folder has all the activity classes that are only accessible once the user authenticates
    • The Authenticated/Posts/ folder contains the preview and save request for a server analysis as well as all the classes responsible for finding the user's last known location
  • The SplashActivity.java file is the main activity that at startup checks if the user saved the token (keep me logged in) and if so try to authenticate the user
  • All the other java files and foldersare responsble for the integration of Tensorflow object detection api and model into our android app

Instructions and Demo

As we have already said the app is comprised of two modes, the mode with no authentication where the user has limite access to the application functionalities and thus,can only detect the food but not analyse or save it. We will show the instructions for first time use and for authenticated use if you want to see the demo video go here

Important: This demo will be in English as the phone used for the demo had an English system however, we have added support for four languages: English, German, Portuguese and Spanish, most of the app has translations for these languages except for the post response are the messages are given by the server.

First Use

If you are using the app for the first time you will see the splash screen

At which point you are presented with the login screen and sign up screen(to get there scroll down on the text for the credentials in login sreen

You can authenticate or if you want to use the app with limited access you can click on the "Continue with no account", if you do so you will see the camera and be presented with the request permissions

Once you give the permission the camera will initiate the prediction for individual objects in an image, do consider that the model only works with 300x300 images as such the app will scale down your image if necessary so be pattient

If you scroll up on the up arrow you sould see more options an information, whenever the model preicts something we output all of its predictions and the total calories into a table to facilitate the visualization

You can then analyse the imageby clicking analyse image however, since you are not authenticated you cannot use this function

Authenticated Use

After you have logged in or signed up (you don't need to always do this if you check the Keep me logged in) you have access to the full app, in the main page you will have two navbars, a side one and a bottom one with only the home, camera and posts options, the main home page will have these same instructions only in a more compact way

If you scroll down you can always refresh the app. If you go to posts, since we haven't yet made any posts it is empty, even after refreshing

You can start the post by opening the camera and detecting objects, after it has detected objects (not empty) and you have clicked the analyse image, you will reveive a notification informing you that we have started the analysis and once it is done you will be moved to the preview page, since we also want to store the user's location you will need to grant the app access to location, after all permissions are set you can visualize the post priview and can either discard the post (ignores it and goes back to the camera) or save it to the server

After saving the post you will be redirected to the post page and will see all your posts (after refreshing) ordered by most recent, if the app was able to find your location you will also see it

You can also delete a posts

You can also visit the about page

In the app you can also logout by opening the nav bar and clicking on logout, after that all your data saved in the device (the auth token) will be deleted and you will need to authentiate to access the full version of the app

Video Demo

The next gif is the demo for the use of the application without authentication, if you want to see to full demo video go here

For the no user demo: