An event-driven Leap Motion plugin for the Unreal Engine 4.
You can use convenience blueprints to just select and play or use a custom approach via blueprints or C++. See relevant sections for details
You can extend functionality to any blueprint through adding the LeapInterfaceEvent interface then adding the LeapController component to that blueprint. This same architecture is available to C++ if you prefer, which also supports both event-driven and polling style use.
See unreal thread for version downloads and development updates.
Since 4.11 this is now the official plugin!
- Create new or open a project.
- Select Window->Plugins. Click on the Input Devices category and you should see a plugin called Leap Motion. Select Enabled. The Editor will warn you to restart, click restart.
- The plugin should be enabled and ready to use.
Since 0.7.10 the plugin includes convenience content for easy reference of say a rigged setup. Since 0.9 these are automatically included as plugin content.
To try the rigged or floating hands characters, change your game mode to use LeapRiggedCharacter or LeapFloatingHandsCharacter as your default pawn.
That's it! hit play to try it out!
If you wish to toggle floating hand types, add ChangeHands to your input action mapping, e.g. using the keyboard button H
If you're using the plugin with a Vive in VR mode, adjust your camera offset to make the hands match where you should see them
- Add a camera to your pawn/character, parent e.g. LeapRiggedEchoHandsActor or LeapImageHandsActor to that camera as a child actor.
- Optionally modify parameters
- Hit play!
Since 2.0 the LeapRiggedCharacter and LeapFloatingHandsCharacter automatically have collision enabled.
If you want to modify the passthrough character or your own sub-class to have collision simply change one setting in your pawn.
First select your mesh
Change the collision preset to PhysicsActor
This will use the BasicBody_Physics PhysicsAsset included with the plugin content for collision. You can modify the collision shapes there to modify what part of your mesh collides with the world.
Using collision you can now interact directly with movable actors with physics simulation turned on.
Pass-through is supported from 0.9, useful if you wish to blend real-world for AR purposes or simply wish to see where you type in vr. Simply select LeapPassthroughCharacter as your pawn.
Play to try out AR/VR transition with a simple gesture
###Convenience Setup and Notes###
Convenience content is built with re-usability in mind. Below is a diagram showing how all the blueprint classes relate.
The LeapBasicRiggedCharacter has only one skeletal mesh and can be used as a basis for basic non-VR characters. If you're rigging a VR character you should split the mesh head from the body so that it can be hidden from the character in order to alleviate clipping issues, this is the functionality that the LeapRiggedCharacter extends from the basic version. Finally if you want to have automatic access to passthrough mode, use the LeapPassthroughCharacter.
What this setup gives you in flexibility is that you have many entry points from which to modify any aspect of the rigging. E.g. If you wish to change post process material being used you would change it in the PPChanger blueprint while still retaining every other functionality. Or if you want to adjust the default mode offset (where your leap is in relation to you when place on the table) you would modify the LeapAnimBodyConnector component which specifies its location. If you want to use your own skeletal mesh, simply replace the mesh, but retain the animation blueprint (use animation retargeting if you're using different skeletons). In fact the rigging setup doesn't even need the leap motion to work, any input which would change the AnimBody skeleton to whatever pose you wish to attain would have the same rigged behavior. The setup is the beginning part of a BodyInput plugin which will support a similar setup for various VR inputs in the future.
All rigged characters automatically swap the appropriate leap offset to display the hands in both default mode (leap on table facing up) and HMD mode by detecting when the user enables their HMD (e.g. by going fullscreen in UE4).
Should you wish to adjust the offset when using the leap on the table facing up, modify the position of the DefaultModeSprite in the LeapAnimBodyConnector. VR mode needs no adjustment as it is always 1:1 due to being attached to your HMD .
- Open desired blueprint where you want to receive the leap events.
- Click on Class Settings and Scroll down in the details panel to Interfaces.
- Add the interface called LeapEventInterface
- Add Leap Controller Component through option A or B
- Option B, (Optional, recommended) Drag from the return value, save this as a blueprint variable e.g. Leap.
- Both, (Optional) Drag from the return value or your Leap blueprint variable (get), type 'Opt' select the only node which is OptimizeForHMD. Select true for the boolean input (red input), this will optimize the leap for top-down use. Keep auto rotate and auto shift if you want the plugin to automatically shift the leap positions to account for HMD movement.
- You're ready to go.
Option A:
Add directly through Add Components
Option B: Place BeginPlay event on your event graph, drag out from the execute node typing 'Leap', select Add LeapController.
The easiest way to use the Leap Motion is to use the input mapping system for simple, joystick-like maneuvers.
E.g. by adding the one input mapping key and 2 axis to the Rolling Template, you can roll the ball by the pitch and roll of your hand and jump by closing your fist or pinching.
You can Right click anywhere on Event graph and type 'Leap Event' and you will see all the available events you can receive.
with
Supports memory efficient event driven api and leap api style polling
For both cases you need to inform the plugin that you want to receive images.
Note that you can control whether events are being emitted by toggling Emit Image events. It is recommended to disable both when not in use for performance and bandwidth reasons.
Example of a toggle
Which gives the following billboard results (note the finger and palm debug visualization is a result of a separate graph)
Distortion texture maps and other image parameters can be obtained by branching the LeapImage node (branched in the example graph to obtain the image id).
All the Blueprint nodes and functions keep the same name as specified in the documentation of leapmotion, refer to this documentation for help with the API. Events are custom named, but follow action oriented naming. Type 'Leap Event' in your blueprint event graph to see all possible events after adding the LeapEventInterface to your blueprint interfaces.
Not an exhaustive list, but common use list. See the leap motion documentation for full list.
The same above result can be achieve by using Leap API style polling like this
Note that a difference between the two forms occurs when image event toggle is off. The plugin reclaims memory and uses no extra performance in the case of events, whereas polling based example will continue attempting to get a texture even if the allow images is off and will cause the plugin to warn the user to enable images every tick. To get the same functionality as the event driven format a separate boolean would need to be used to branch the execution to stop the log warning spam.
Require enabling before receiving events, below is an example of how to get the Circle gesture to emit and how to filter it.
With 0.7.2 the plugin supports the gesture sub-classes directly, here is an example of showing a debug sphere around the circle you are gesturing. Note that you can use the generic gesture detection event and cast the gesture to your sub-gesture or use the sub-gesture specific event. Both are equivalent.leap-ue4
An event-driven Leap Motion plugin for the Unreal Engine 4.
You can use convenience blueprints to just select and play or use a custom approach via blueprints or C++. See relevant sections for details
You can extend functionality to any blueprint through adding the LeapInterfaceEvent interface then adding the LeapController component to that blueprint. This same architecture is available to C++ if you prefer, which also supports both event-driven and polling style use.
See unreal thread for version downloads and development updates.
Since 4.11 this is now the official plugin!
- Create new or open a project.
- Select Window->Plugins. Click on the Input Devices category and you should see a plugin called Leap Motion. Select Enabled. The Editor will warn you to restart, click restart.
- The plugin should be enabled and ready to use.
Since 0.7.10 the plugin includes convenience content for easy reference of say a rigged setup. Since 0.9 these are automatically included as plugin content.
To try the rigged or floating hands characters, change your game mode to use LeapRiggedCharacter or LeapFloatingHandsCharacter as your default pawn.
That's it! hit play to try it out!
If you wish to toggle floating hand types, add ChangeHands to your input action mapping, e.g. using the keyboard button H
- Add a camera to your pawn/character, parent e.g. LeapRiggedEchoHandsActor or LeapImageHandsActor to that camera as a child actor.
- Optionally modify parameters
- Hit play!
Since 2.0 the LeapRiggedCharacter and LeapFloatingHandsCharacter automatically have collision enabled.
If you want to modify the passthrough character or your own sub-class to have collision simply change one setting in your pawn.
First select your mesh
Change the collision preset to PhysicsActor
This will use the BasicBody_Physics PhysicsAsset included with the plugin content for collision. You can modify the collision shapes there to modify what part of your mesh collides with the world.
Using collision you can now interact directly with movable actors with physics simulation turned on.
Pass-through is supported from 0.9, useful if you wish to blend real-world for AR purposes or simply wish to see where you type in vr. Simply select LeapPassthroughCharacter as your pawn.
Play to try out AR/VR transition with a simple gesture
### Convenience Setup and Notes
Convenience content is built with re-usability in mind. Below is a diagram showing how all the blueprint classes relate.
The LeapBasicRiggedCharacter has only one skeletal mesh and can be used as a basis for basic non-VR characters. If you're rigging a VR character you should split the mesh head from the body so that it can be hidden from the character in order to alleviate clipping issues, this is the functionality that the LeapRiggedCharacter extends from the basic version. Finally if you want to have automatic access to passthrough mode, use the LeapPassthroughCharacter.
What this setup gives you in flexibility is that you have many entry points from which to modify any aspect of the rigging. E.g. If you wish to change post process material being used you would change it in the PPChanger blueprint while still retaining every other functionality. Or if you want to adjust the default mode offset (where your leap is in relation to you when place on the table) you would modify the LeapAnimBodyConnector component which specifies its location. If you want to use your own skeletal mesh, simply replace the mesh, but retain the animation blueprint (use animation retargeting if you're using different skeletons). In fact the rigging setup doesn't even need the leap motion to work, any input which would change the AnimBody skeleton to whatever pose you wish to attain would have the same rigged behavior. The setup is the beginning part of a BodyInput plugin which will support a similar setup for various VR inputs in the future.
All rigged characters automatically swap the appropriate leap offset to display the hands in both default mode (leap on table facing up) and HMD mode by detecting when the user enables their HMD (e.g. by going fullscreen in UE4).
Should you wish to adjust the offset when using the leap on the table facing up, modify the position of the DefaultModeSprite in the LeapAnimBodyConnector. VR mode needs no adjustment as it is always 1:1 due to being attached to your HMD .
- Open desired blueprint where you want to receive the leap events.
- Click on Class Settings and Scroll down in the details panel to Interfaces.
- Add the interface called LeapEventInterface
- Add Leap Controller Component through option A or B
- Option B, (Optional, recommended) Drag from the return value, save this as a blueprint variable e.g. Leap.
- Both, (Optional) Drag from the return value or your Leap blueprint variable (get), type 'Opt' select the only node which is OptimizeForHMD. Select true for the boolean input (red input), this will optimize the leap for top-down use. Keep auto rotate and auto shift if you want the plugin to automatically shift the leap positions to account for HMD movement.
- You're ready to go.
Option A:
Add directly through Add Components
Option B: Place BeginPlay event on your event graph, drag out from the execute node typing 'Leap', select Add LeapController.
The easiest way to use the Leap Motion is to use the input mapping system for simple, joystick-like maneuvers.
E.g. by adding the one input mapping key and 2 axis to the Rolling Template, you can roll the ball by the pitch and roll of your hand and jump by closing your fist or pinching.
You can Right click anywhere on Event graph and type 'Leap Event' and you will see all the available events you can receive.
with
Supports memory efficient event driven api and leap api style polling
For both cases you need to inform the plugin that you want to receive images.
Note that you can control whether events are being emitted by toggling Emit Image events. It is recommended to disable both when not in use for performance and bandwidth reasons.
Example of a toggle
Which gives the following billboard results (note the finger and palm debug visualization is a result of a separate graph)
Distortion texture maps and other image parameters can be obtained by branching the LeapImage node (branched in the example graph to obtain the image id).
All the Blueprint nodes and functions keep the same name as specified in the documentation of leapmotion, refer to this documentation for help with the API. Events are custom named, but follow action oriented naming. Type 'Leap Event' in your blueprint event graph to see all possible events after adding the LeapEventInterface to your blueprint interfaces.
Not an exhaustive list, but common use list. See the leap motion documentation for full list.
The same above result can be achieve by using Leap API style polling like this
Note that a difference between the two forms occurs when image event toggle is off. The plugin reclaims memory and uses no extra performance in the case of events, whereas polling based example will continue attempting to get a texture even if the allow images is off and will cause the plugin to warn the user to enable images every tick. To get the same functionality as the event driven format a separate boolean would need to be used to branch the execution to stop the log warning spam.
Require enabling before receiving events, below is an example of how to get the Circle gesture to emit and how to filter it.
With 0.7.2 the plugin supports the gesture sub-classes directly, here is an example of showing a debug sphere around the circle you are gesturing. Note that you can use the generic gesture detection event and cast the gesture to your sub-gesture or use the sub-gesture specific event. Both are equivalent.
When you create a class blueprint you can add a component called LeapController, this contains all the functionality that the LeapController has specified in their API. From the controller you can get the frame and do what you want with features currently available.
Available blueprint classes branching from LeapController:
- Arm
- Bone
- CircleGesture
- Finger
- FingerList
- Frame
- Gesture
- GestureList
- Hand
- HandList
- InteractionBox
- KeyTapGesture
- LeapImage
- LeapImageList
- Pointable
- PointableList
- ScreenTapGesture
- SwipeGesture
- Tool
- ToolList
- If you get errors complaining 'Leap Motion' plugin cannot be found, you do not have code in your project.
- If you get errors complaining the 'Leap.dll' is missing, you did not drag binaries to the correct location.
- C++ how to
Post issues to this github or the unreal thread.
Event-driven fork by Getnamo.
Original Plugin by Marc Wieser.