Skip to content

Commit

Permalink
Merge pull request #82 from pupil-labs/develop
Browse files Browse the repository at this point in the history
Update master to v1.0

Check v1.0 release notes for information about the new features and high level changes.
  • Loading branch information
fx-lange authored Sep 6, 2019
2 parents 8b28d8a + 689bb4b commit c96b6b5
Show file tree
Hide file tree
Showing 2,995 changed files with 32,396 additions and 417,034 deletions.
9 changes: 7 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
dev_Refactor-and-demos/Library/
dev_Refactor-and-demos/Temp/
*.pyc
plugin/Tests*

# Local gitignores
*/Assets/.gitignore

# VSCode
*/**/.vscode/*
321 changes: 0 additions & 321 deletions Developer.md

This file was deleted.

Binary file removed GettingStarted/BeforeStartingCalibration.png
Binary file not shown.
Binary file removed GettingStarted/VRBuildAndPlayerSettings.png
Binary file not shown.
49 changes: 17 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,46 +3,31 @@ hmd-eyes

Building blocks for eye tracking in Augmented Reality `AR` and Virtual Reality `VR` or more generally Head Mounted Displays `HMD`. The purpose of this repository is do demonstrates how to implement Pupil with Unity3D. For details on Pupil computations, please have a look at the related repositories.


## VR Getting Started

This guide walks you through a first time setup for your Pupil Labs VR add-on.

1. [HTC Vive Add-on setup](https://docs.pupil-labs.com/#htc-vive-add-on) - Install your Pupil eye tracking add-on into your HMD and connect the add-on to your computer.
1. [Download Pupil Software](https://github.com/pupil-labs/pupil/releases/latest). Extract the Pupil Capture app to your Desktop.
1. Start Pupil Capture via the `pupil_capture.exe` in the Pupil Capture App folder. Pupil Capture performs the actual eye tracking and provides real-time data access via a network API.
1. [Download and start hmd-eyes demo app](https://github.com/pupil-labs/hmd-eyes/releases/latest) - This demo runs a VR experience. The demo app talks to Pupil software in the background. You will use it to calibrate and visualize gaze data within a demo scene. The demo app will start with settings dialog. <!-- insert image of app demo dialog -->
1. Start the demo with default values. You will see a view of the left eye of the hmd.
1. This would be a good point to put said device on your head.
2. Use the displayed realtime videos of your eyes to make sure they are as centered as possible and in focus.
3. Press 'c' on your keyboard to start a calibration and focus your gaze on the displayed marker as it changes position.
4. After a successful calibration, this scene will be loaded

![Market Scene Demo](https://github.com/pupil-labs/hmd-eyes/blob/master/GettingStarted/2DMarketScene.png)


## HoloLens Getting Started

1. [HoloLens Add-on setup](https://docs.pupil-labs.com/#hololens-add-on) - Set up the Pupil eye tracking add-on with your HMD and connect it to a PC.
1. [Download Pupil Software](https://github.com/pupil-labs/pupil/releases/latest). Extract the Pupil Capture app to your Desktop.
1. Start Pupil Capture via the `pupil_capture.exe` in the Pupil Capture App folder. A window like this will appear when service is running. Capture does pupil detection from eye images.
1. Select the `Plugin Manager` in `Pupil Capture` and start the `HoloLens Relay` plugin ![Pupil Capture with HoloLens Relay](https://github.com/pupil-labs/hmd-eyes/blob/master/GettingStarted/PupilCaptureWithHoloLensRelay.png)
1. [Download and install Unity 3D](https://store.unity.com/).
1. [Download the hmd-eyes source code](https://github.com/pupil-labs/hmd-eyes/releases/latest). Extract the Unity project sources for HoloLens, located in `unity_pupil_plugin_hololens`.
1. In Unity3d open the `unity_pupil_plugin_hololens` as a project. Double click the `Shark Demo/2D Calibration Demo` to load the scene and open the `Holographic Emulation` tab (`Menu/Window/Holographic Emulation`)
1. In the `Holographic` tab, select `Remote to Device` as `Emulation Mode`
1. Read more about Unity's `Holographic Emulation` [here](https://docs.unity3d.com/550/Documentation/Manual/windowsholographic-emulation.html).
1. Start the `Holographic Remoting Player` on your HoloLens device. Enter the displayed device IP in the `Holographic` tab under `Remote machine`. Click the `Connect` button.
1. Once connected, press `Play` in Unity Editor.
1. Follow the on device instructions to open the menu (double air tap) and select `Connect to Pupil` by looking straight at the button and confirming with a single air tap. ![HoloLens Menu](https://github.com/pupil-labs/hmd-eyes/blob/master/GettingStarted/HoloLensMenu.png)
1. Once the connection between Unity3d and `Pupil Capture` is established two eye windows will open. Use these windows to adjust the eye cameras for good tracking.
1. The `Start Calibration` will now be enabled on the HoloLens display.
1. Calibrate - Start the calibration. Focus your gaze on the displayed marker for each displayed position.
1. After a successful calibration, you should see a rotating 3D model and a visualization of your gaze ![Shark Demo with Gaze Visualization](https://github.com/pupil-labs/hmd-eyes/blob/master/GettingStarted/2DDemoHoloLens.png)
2. [Download Pupil Software](https://github.com/pupil-labs/pupil/releases/latest). Extract the Pupil Capture app to your Desktop.
3. Start Pupil Capture via the `pupil_capture.exe` in the Pupil Capture App folder.
4. Please refer to the Pupil [getting started](https://docs.pupil-labs.com/#capture-workflow) and [user docs](https://docs.pupil-labs.com/#pupil-detection) to ensure that eyes are well captured and that the pupil detection runs with high confidence (~0.8).
5. [Download and start hmd-eyes demo app](https://github.com/pupil-labs/hmd-eyes/releases/latest) - This demo runs a VR experience. The demo app talks to Pupil software in the background. You will use it to calibrate and visualize gaze data within a demo scene. The demo app will start with settings dialog.
6. Start the demo with default values. You will see a view of the left eye of the hmd.
7. This would be a good point to put said device on your head.
8. Use the displayed realtime videos of your eyes to make sure they are as centered as possible and in focus.
9. Press 'c' on your keyboard to start a calibration and focus your gaze on the displayed marker as it changes position.
10. After a successful calibration, the example scene will appear again and the gaze estimate will be visualized.

## Develop, integrate, and extend

Check out the [developer docs](https://github.com/pupil-labs/hmd-eyes/blob/master/Developer.md) to learn how to set up dev envirmoment, make changes to code, and integrate with your own Unity3D project.
Download the *.unitypackage from the [latest release](https://github.com/pupil-labs/hmd-eyes/releases/latest) and
check out the [developer docs](./docs/Developer.md) to learn how to set up dev environment, make changes to code, and integrate with your own Unity3D project.

**Dependencies**:

* Unity 2018.3+
* `ProjectSettings/Player/Configuration/Scripting Runtime Verion` set to **.NET 4.x Equivalent**.
* Due to an issue with MessagePack, the default project setting for `ProjectSettings/Player/Configuration/API Compatibility Level` is not supported and needs to be set to **.NET 4.x**

## Community

Expand Down
File renamed without changes
File renamed without changes
File renamed without changes
Binary file added docs/BeforeStartingCalibration.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/BlinkHero.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes.
Binary file added docs/CalibrationController.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/CalibrationResources.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit c96b6b5

Please sign in to comment.