Skip to content

PyrusMed/Pear-Interaction-Engine-HoloLens

Repository files navigation

pear interaction engine

What is PIE?

PIE was made to give designers and developers an easier way to create, share and modify user interactions. At its core, PIE is a set of cross-device platform independent Unity components that make creating intuitive interactions easier. PIE currently supports mouse and keyboard, Leap Motion, Oculus Touch, and HoloLens gestures, with more interaction support coming soon. Use PIE if you want a quick way to give your users intuitive interactions in VR, AR, on the desktop and on mobile. Contribute to PIE if you want to make interactions better for all developers, designers, and end users. We hope you find this useful!

leap hover-small leap grab-small touch motion rotate-small touch motion zoom-small touchmotion drop controller-small leap zoom-small

Why PIE?

Interactions are a core component of every user facing software application, yet there are very few pattern guidelines, best practices and tools that can be used to design and develop interactions in virtual (VR) and mixed reality (MR). We've explored interactions using cheap headsets, expensive headsets, hand controllers, full body motion capture suits, and just about everything in between. In the process, we created PIE, a set of tools we've used for both rapid prototyping and production quality products. We're happy to share these tools and welcome all contributions as we look forward to the next generation of human-computer interactions.

PIE Architecture

PIE consists of modular components that can be swapped in and out. These components revolve around the Controller, ControllerBehavior and Interaction classes.

  • Controller represents a user input device, such as the user's eyes
  • ControllerBehavior represents something a controller can do, like gazing at an object
  • Interaction represents how input affects objects, such as highlighting an object when its looked at

Example - Fade On Hover (3 steps)

First, we define a Controller on the camera, because the user's head is a source of input. add controller-optimized

Next, we add a ControllerBehavior that lets other objects know when the head is looking (or gazing) at them. add gaze behavior-optimized

Finally, we add an Interaction to the object we want to fade that links the "looking" logic to the "fading" logic. add interaction-optimized

We can even link a single fade action to multiple Interactions to keep things clean and consistent across multiple objects. link multiple interactions-optimized

Why was it designed this way?

Controllers, ControllerBehaviors and Interactions represent how we interact with objects in the real world. For example, if you were to pick up a mug, you would use your hand (Controller) to grab (ControllerBehavior) the mug that would react to your grab (Interaction). As babies we have hands (Controllers), but we don't know how to use them (i.e. we don't have any ControllerBehaviors). Over time we learn to use our hands, body parts and tools to interact with objects, like a mug. Right now PIE is in its infancy, but as we grow our set of Controllers, ControllerBehaviors and Interactions, we'll be able to quickly add complex interactions to our applications using our collective knowledge.

Oh, one more really cool thing. ControllerBehaviors and Interactions can be mixed and matched. Using our baby analogy, once a child learns how to grab a mug it will quickly apply its grab logic to other objects, like things it shouldn't put in its mouth :/. PIE was designed similarly, but is much safer. Once we develop a ControllerBehavior we can use it to create countless interactions. For example, we've created a ControllerBehavior for grabbing with the Leap Motion hand, so that ControllerBehavior can be used to pick up any object, whether it's a cube, a mug, or anything else. It can also be used to open a door, or flip on a light switch...the possibilities are endless since we can link the grab logic to any Intraction.

If you couldn't tell, we're pretty excited about this :).

Getting Started with PIE

Clone and Run A Branch

  1. Clone one of our several branches
  2. Open one of the example scenes in Assets/Pear.InteractionEngine */Examples/...
  3. Run!

Import PIE Into Your Mouth! (I mean...into your Own Project)

Download our latest package from the releases section and import it into your Unity project

Contributing

If you like PIE, want to make it better, or just want to work on something cool, help us out! We think user interactions are extremely important, so we'd love to work with others to improve how we all design, develop and use interactions in VR, MR and everywhere else. Fork this repo to get started!

Who is using PIE?

Hopefully this list will continue to grow ;)

Questions?

If you have any questions please log an issue or contact us directly

Thanks!

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages