Leap Motion’s hand tracking modern technology makes grabbing and also poking digital objects practically as basic as in genuine life. Combine this v ZED Mini AR passthrough capabilities to lug your AR endure to the following level.

You are watching: Leap motion htc vive

building a food chain#mixedreality #vr #leapmotion #zedcamera #oculusrift pic.twitter.com/2C0hQX5kZ0

— Yujin Ariza (

Getting Started

In this guide, we’ll add basic hand tracking and interactions come our unit plugin’s Planetarium scene. This will let you knock planets around your room with your hands as though they were balloons.


Oculus Rift or HTC Vive VR headsetUnity 2017.1 or greater

Hardware Setup

We’ll i think you currently have a ZED Mini attached come an Oculus Rift or HTC Vive. If not, monitor this overview for the Rift and this guide for the Vive, skipping the 3rd test action if friend wish.

The setup guide for her Leap activity tells friend to affix it come the center, where you’ve currently placed her ZED Mini. Instead, connect it just listed below the ZED Mini. We’ll compensate for this later on in Unity.


If friend haven’t already, make sure you’ve mounted the ZED SDKand Leap activity Orion.

Setting up Unity

Enable VR by walk toEdit->Project Settings->Player, climate in thePlayermenu, openingXR Settingsand checkingEnable XROpen the ZED Planetarium instance scene via the Project window viaZED ->Examples->Planetarium->SceneIn the job window, navigate toLeapMotion->Core->Prefabsand drag the Leap Rig prefab into the scene.

Your scene and also hierarchy have to look choose this:


Combining the Rigs

The Planetarium instance scene currently has the ZED_Rig_Stereo prefab in it, which handles the ZED’s image, tracking and headset output. By default, the Leap activity rig also tracks the VR headset, making use of its very own Unity camera. We have to switch the to follow the ZED rig instead.

ExpandZED_Rig_Stereo, right click theCamera_eyesobject and clickCreate Empty. Surname it “Leap Offset”ExpandLeap Rig andclick onMain CameraIn the Inspector, disable theCameracomponent – we’ll be utilizing the ZED’s camera because that the last image instead.In theLeap XR business Providercomponent, setDevice balance out Modeto “Transform”. This reasons the rig to monitor a certain object instead of the VR camera.Drag and also drop the brand-new Leap balance out object native the hierarchy into the machine Origin fieldZED Mini and also Leap need an balance out correction

That’s since the Leap motion is an installed below the ZED, therefore their pictures don’t line up. Fixing this is easy, thanks to the Leap offset object we made.

Offsetting the Leap Hand Tracking Rig

Without closing the scene, take off the headset, selectLeap Offsetand adjust its transforms to line it up properly. The Y place is most important, yet make certain to move your hand roughly to make certain all axes are appropriately aligned. If using a Vive, friend may additionally need to readjust the X rotation slightly.When you have the right to see in the Game home window thatLeap Offsetis in a good place, click the equipment icon in the top right ofLeap Offset‘s change component, and clickCopy ComponentStop the sceneClick ~ above that same gear, adhered to byPaste component ValuesSave the scene

Now when you operation the scene, the wires should already be lined up v your hands.

ZED Mini and Leap are now aligned in Unity

Adding Physics

The default “hands” offered in the Leap rig let you check out where they are, yet they can’t collide with anything in the scene. We desire the the opposite – invisible hands that reason collisions – so that it shows up your real-life hands are colliding with the online objects.

See more: I Dont Know But Ive Been Told ' Army Song? I Don'T Know, But I'Ve Been Told Lyrics

In the project window, navigate toLeapMotion->Core->Prefabs->HandModelsPhysicalDrag and also dropRigidRoundHand_LandRigidRoundHand_Rto theHand Modelsobject in theLeap Rigto do them kids of itSelect theHand Modelsobject.In the Inspector underHand design Manager, expandModel PoolandCapsule HandsIn theLeft/Right Modelfields, change the Capsule hands with the RigidRoundHands you simply added.Delete the Capsule Hands the are kids ofHand Models

The last step is to provide something because that the brand-new physics-empowered hands to communicate with. Adding colliders to the planets will do the trick.

Expand the Planetarium object, and theEarthobject listed below thatSelect every the planets, theSun, and theMoon(a boy ofEarth) in ~ once. In the Inspector, include a Rigidbody componentUncheck Use Gravity and check Is Kinematic on these new Rigidbody componentsWith the objects still selected, also add a Sphere Collider componentSelect the objects personally and change the Radius value of the ball Collider until you watch its border fit the earth in the step window

Save, run the scene, and put on your headset. With out and also poke a planet. Dribble it in between your palms. Punch the sun to send it flying toward your buddy. Earth and also the rest of the solar device is now in her hands!