An open-source project developed with Unity3D and Photon to create a collaborative 3D virtual environment for academic and educational purposes.
Now powered by AI!
The main goal of this project was to develop an AI infrastructure for the SENEM Metaverse, where Smart Students powered by AI interact with the class and enhance the educational experience of the Metaverse itself.
- Creation and connection to a room. The platform offers real-time connection with other users, implemented through a room system, allowing users to choose whom to connect with. The first user intending to connect will create the room, providing it with a password, and then share this information with interested individuals. The user may decide to spawn in Smart Students that will interact with the class and if this interaction should be by text chat or via audio.
- Free three-dimensional exploration. Freedom of movement and interaction characterize the virtual environment, allowing users to navigate any classroom area and engage with objects seamlessly through their avatar. Users can walk, sit, and interact freely with various objects in the scene. They can rotate their view and adjust the zoom level of their perspective as well.
- Avatar's customization. The user has the option to customize their avatar in various ways. The prototype includes a dedicated interface where users can make different appearance choices for their avatar, including skin color, uniform type and color, eye shape and color, eyebrows and possible beard, hairstyle, and a selection of some additional cosmetics like glasses or eye patches.
- Realistic voice communication. Users can engage in real-time communication with each other through voice chat. Using their microphones and voices, they can make their avatars speak and hear others as in genuine verbal communication. The voice chat is equipped with proximity and three-dimensionality features, allowing the tone of voice to vary based on the distance and position of the interlocutor.
- Text communication. It is also possible to communicate through text chat, which is readily available and visible in the platform's user interface.
- Smart Students. Users can interact with AI powered students, which are able to understand and communicate with them. More on that later.
- Non-verbal interaction. Various animations are available for avatars, and users can perform actions such as waving, clapping, or raising their hands.
- Projection and presentation of multimedia content. The virtual classroom is equipped with a projector to transmit multimedia content. Users can upload their slides or images and display them during the platform's runtime. The user can navigate the content on the projector with the presenter role. Each user can use the projector to showcase or present their materials and alternate this role with others during the session.
- Interaction with the whiteboard. The classroom also has a whiteboard where users can type and write using a keyboard. One user at a time can approach the whiteboard and start writing, and what they write will be visible to everyone, supporting interaction and communication.
This repository contains:
- Metaverse_Classroom_2: The complete Unity Project of the application.
- Windows Build: The built application ready to use for Windows.
- MacOS Build.app: The built application for MacOS.
Windows:
- Download the Windows Build folder.
- Run the Metaverse_Classroom executable.
MacOS:
- Download the MacOS Build.app and extract it intto a folder
- Follow these steps to allow your device to run the application:
- Open the terminal and navigate to the folder that contains the extracted files.
- Type the following command
chmod -R +x MacOS Build.app/Contents/MacOSinto the terminal. - Type the following command
xattr -cr MacOS Build.appinto the terminal. - Double-click on MacOS Build.app.
Requirements:
- Unity3D version: 2021.3.22f1
- Unity Hub
- Smart Student Server (optional, runs locally)
Steps:
- Download Metaverse_Classroom_2.
- Open the Unity Hub, go in the Projects tab and click on the Open button.
- Choose the path where you downloaded the Metaverse_Classroom_2.
- The project should now appear in your projects list. Click on its name and Unity will install all the needed libraries to make it work.
How do I make the students work?
The students run on the Smart Student Server, which runs locally (this will probably change in the future to allow direct access without modifying the code), by default the server runs on port 5000, to correct changes done on the server's location, reference the QuestionDispatcher.cs file.
The students run through a chain of interactions by commercially available AIs and their interaction can be either verbal or written, you can choose this during while creating a room.