Augmented Reality Skills Training

This repository is hosted as a part of FSI Hackathon for Autism (https://github.com/fsi-hack4autism)

Objective: Teach basic job skills to people with autism using Augmented Reality

This use case will focus on the use of Augmented Reality for teaching job skills to people with Autism, particularly those aging out of the supports provided in the school environment. Unemployment for those with Autism is significant and the supports in the work environment are limited. The use case will use Unity engine to build Augmented Reality solutions that can be used on platforms like HoloLens to help with basic job skills by presenting visual cues on how to complete task such as stocking shelves or preparing food. Additonally, the use case will make use of Gen AI, to manage, evaluate, and provide feedback to the student on how well the skill is being performed. It can also be used to generated the slight variants and increasingly complex scenarios to practice and introduce gamification. This will also address the issue of scaability - moving away from 1:1 support to a 1:N as an a student becomes an adult.

Sample Scenarios

The skills th application will focus on can include one or more of the following:

• Stacking shelves in a store – e.g., making sure all the labels are facing outwards, etc.
• Setting up a table in a restaurant – making sure all the cutlery is there an arranged properly
• Setting up a catering tray – e.g., making sure all items are placed properly

Code repository setup

Sample HoloLens

• This is a sample hololens project that you can use as a reference to get started with Unity 3D development

Note: Create all projects in a folder under https://github.com/fsi-hack4autism/augmented-reality-skills-training/tree/main/Sample%20Hololens

Hackathon 2022

• This is the code from Hackathon 2022

Note: Create all projects in a folder under https://github.com/fsi-hack4autism/augmented-reality-skills-training/tree/main/2022

Development best practices

• Create a subfolder under data-ingestion for your project
• Add a readme.md file to explain the overall objective of your project
• Indicate the tech stack you are using
• You can upload further artifacts if you want - such as presentations, documents, excel/csv files etc.