Been There, Seen That (BTST): Visualization of 3D eye tracking data from real world environments

This Project allows you to analyze 3D gaze and movement data sets recorded using the HoloLens 2. It provides you with a gaze replay visualization, which is linked to a space-time cube visualization to show an overview of the behavioral data and inspect important events in more detail.

Requirements

We recommend using Unity version 2020.3.24f to run this project.

Setup

Once the project is opened in Unity, drag the scenes Gaze Replay and STC to the Hierachy and Unload the scene Gaze Replay. The Scenes can be found within the project Folder in Assets/Scenes.

Structure

The project includes a folder called Assets, which contains the necessary scripts and data. The important scripts and the data are listed below.

.
└── MyScripts
    └── General
        |── ButtonFunctionalities # the code for UI elements
        |── ReadData            # the code for loading the data
        |── Trajectory            # visualizes movement within space-time cube(STC) 
        |── StackedHeatMap    # visualizes cube heatmap within STC
        |── HeatmapWall          # visualizes heatmap within gaze replay        
        |── ReplayManager_General   # visualizes participants within gaze replay    
└── Resources
    └── CSVFiles
       └──AnchorFile   #contains the files needed to transform the data into one   
                                  coordinate system
       └──GazeData   #contains the recorded gaze data of the participants
└── Scenes
    |── GazeReplay          # Scene for gaze replay 
    |── STC                      # Scene for STC 

Background

As mentioned above the data was recorded using the HoloLens 2. We generated a Unity project and utilized the ARETT package from Kapp et al., 2020 to record the gaze data of different participants. Afterwards the ARETT-R-Package was used to obtain fixation data, which were then utilzed for the visualization. To avoid copyright infringement we replaced the photogrammetry mesh with a standard cube and added the artwork collections, to visualize the contextual information.
For more details regarding the individual visualization techniques, we refer to our paper.

Reference

Pathmanathan, N., Öney, S., Becher, M., Sedlmair, M., Weiskopf, D. and Kurzhals, K. (2023), Been There, Seen That: Visualization of Movement and 3D Eye Tracking Data from Real-World Environments. Computer Graphics Forum, 42: 385-396. https://doi.org/10.1111/cgf.14838