Unity: Oculus HMD Tracking

Main Page → OptiTrack Unity Plugin → Unity: Oculus HMD Tracking

---

Oculus / OptiTrack Unity plugin Integration At the time of writing, there has been a problem reported with using OptiTrack Unity plugin for Oculus Rift HMD integration. An update to the asynchronous spacewarp feature in the updated Oculus SDK has been reported to cause problems with rendering of the HMD view. This is working correctly in Unreal Engine plugin. As a workaround, please use OptiTrack Unreal Engine 4 Plugin or HTC Vive HMDs as a workaround. This note will be removed once this problem has been fixed.

Page Scope

This page provides instructions on how to use the OptiTrack Unity Plugin for tracking of the Oculus head-mounted display (HMD) within Unity.

Overview

Oculus HMDs can be tracked by OptiTrack motion capture system and integrated into Unity using the OptiTrack Unity Plugin. Once this is set up, the motion capture system will override the Oculus Tracker to track the HMD from the attached markers for unique VR experiences. For tracking of objects other than HMDs, please read through the OptiTrack Unity Plugin page for details.

Note This guide assumes that the motion capture volume has been set up and optimized for tracking. If the tracking volume is not set up yet, please go through the Getting Started guide to set up and calibrate the camera system. Also, if you plan to install any obstacles (e.g. walls) inside the capture volume, make sure they are non-reflective. Also, make sure the cameras are placed and oriented so that every corner is thoroughly captured by multiple cameras.

Supported Systems

• Oculus Rift DK2
• Oculus Rift CV1
• Oculus GearVR

Oculus HMD Setup

---

The following section provides instructions on setting up the HMD for tracking in VR applications. This includes placing markers on the HMD for tracking and creating a corresponding rigid body definition in Motive.

General Setup Steps

1. Attach the markers on the HMD
2. Create a rigid body asset in Motive
3. Calibrate the Pivot Point of the rigid body
4. Configure the rigid body settings in Motive

Marker Attachment

There are two different types of markers that can be used to track an object with OptiTrack systems: Passive markers and Active markers. Passive markers are retroreflective markers that reflect infrared light emitted from the IR LEDs on the camera. On the other hand, the active markers are LED markers that emit the IR light. Either type of markers can be used to track HMDs.

Use appropriate adhesives and marker posts to securely attach the markers to the HMD. For active markers, the active Tag must be attached also in order to synchronize and power the LEDs. For Oculus CV1 HMDs, the active CV1 clip enclosure can be used to easily place 8 active markers over the HMD. Depending on which markers you are using to track the HMD, the setup instructions are slightly different. For most applications, using the HMD clip is strongly recommended.

Oculus CV1 HMD ClipPassive Markers

Active Markers

Attachment & Placement

With the HMD bracket, simply attach the active marker bracket onto an CV1 HMD and use the active markers on it to define a rigid body.

When using the active markers, the markers are actively labeled individually from each of their own unique illumination patterns, and only the matching active markers get contributed to the rigid body solve. The biggest benefit from the individual labeling is that markers can be placed in perfectly symmetrical placements among multiple rigid body assets.

• 

  HMD markers with passive markers
• 

  Putting the HMD bracket on
• 

  When detaching the bracket from the HMD, first pull up the two latches behind the bracket and slowly pull it from the top.

Passive Markers

Attachment

HMD markers with passive markers

When attaching retroreflective markers, make sure markers are securely attached and readily captured by the cameras. For attaching the markers, we recommend using our 20 mm wide and 30 mm tall M4 threaded plastic marker bases with Acrylic adhesives, available at the webstore, to attach the markers onto the HMD.

Placement

A markered HMD will be defined as a rigid body in Motive. When placing markers, make sure the placement asymmetry is respected in the arrangement within the HMD. Also, the marker arrangements between multiple HMDs must be incongruent. For more details, read about marker placement from the Rigid Body Tracking page.

Create an HMD Rigid Body

HMD with Active ClipHMD with Attached Markers

Creating Active Clip HMD

Creating an HMD rigid body in the Builder pane.

This feature can be used only with HMDs that have the OptiTrack Active HMD clips mounted.

For using OptiTrack system for VR applications, it is important that the pivot point of HMD rigid body gets placed at the appropriate location, which is at the root of the nose in between the eyes. When using the HMD clips, you can utilize the HMD creation tools in the Builder pane to have Motive estimate this spot and place the pivot point accordingly. It utilizes known marker configurations on the clip to precisely positions the pivot point and sets the desired orientation.

HMDs with passive markers can utilize the External Pivot Alignment tool to calibrate the pivot point.

Steps

1. First of all, make sure Motive is configured for tracking active markers.
2. Open the Builder pane under View tab and click Rigid Bodies.
3. Under the Type drop-down menu, select HMD. This will bring up the options for defining an HMD rigid body.
4. If the selected marker matches one of the Active clips, it will indicate which type of Active Clip is being used.
5. Under the Orientation drop-down menu, select the desired orientation of the HMD. The orientation used for streaming to Unity is +Z forward and Unreal Engine is +X forward, or you can also specify the expected orientation axis on the client plugin side.
6. Hold the HMD at the center of the tracking volume where all of the active markers are tracked well.
7. Select the 8 active markers in the 3D viewport.
8. Click Create. An HMD rigid body will be created from the selected markers and it will initiate the calibration process.
9. During calibration, slowly rotate the HMD to collect data samples in different orientations.
10. Once all necessary samples are collected, the calibrated HMD rigid body will be created.

Creating HMD from the markers

In Motive, select the markers that are attached to the HMD and create a rigid body (Hotkey: CTRL + T) as you would do for other rigid bodies. Once this is set up, use the External Pivot Alignment tool in the Builder pane (rigid body → Edit) to calibrate the pivot point of the created rigid body:

For using OptiTrack system for VR applications, it is important that the pivot point of HMD rigid body gets placed at the appropriate location, which is at the root of the nose in between the eyes. External Pivot Alignment tool utilizes both the OptiTrack mocap system and the HMD's own tracking system (Oculus Tracker / Vive Base Stations) for precisely calculating the location of the pivot point of an HMD. When using this feature, both systems must be tracking the HMD simultaneously.

HMD Compatibility Notes: For Motive versions 2.1 Final and later: Oculus and Vive HMDs are supported. For older versions of Motive: Only Oculus HMDs are supported. The corresponding native HMD tracker must be tracking the HMD along with the motion capture system simultaneously. For CV1 active HMD clips, the rigid body definition will be provided separately.

External Pivot Calibration Tips: Do not move around the HMD during calibration. You will want to keep it at one position and slowly rotate the HMD so that it can calibration from different orientations. When rotating, please rotate it slowly. Camera Exposure: Lower down the camera exposure setting to where the exposure is minimized but the HMD is still tracked well in Motive. This will help with preventing IR interference between the VR HMD's own tracking system and the motion capture system during the pivot calibration process. Set this to around 30us ~ 100us. Frame rate: Lower down the system frame rate down to 30 FPS for better sampling. Calibration Result: For best VR experience, please aim for the error result below ~1mm/marker.

Steps

1. First of all, connect the HMD Tracker (Oculus Tracker / Vive Base Stations) and have its corresponding software running in the background.
2. Launch Motive.
3. [Motive] Create a rigid body asset from the markers on the HMD.
4. [Motive] Open the Builder pane and access editing options for rigid bodies.
5. [Motive] Select the HMD rigid body in Motive.
6. [Motive → Builder pane] Under Detected Type, it should indicate the type of HMD you will be using.
7. [Motive → Builder pane] Set the desired number of sample counts (Default: 750) and the desired orientation.
8. [Motive → Builder pane] Press start to initiate the sampling process.
9. [Motive → Builder pane] During calibration, make sure both the mocap system and the HMD tracker (Oculus Tracker / Vive Base Stations) is tracking the HMD.
10. [Motive → Builder pane] Once it starts collecting the samples, point the HMD towards the tracker and slowly rotate it and sample different orientations.
11. [Motive → Builder pane] When it finishes collecting samples, it will display the offset distance, in mm, between positions of the pivot point detected by the HMD tracking system and the motion capture system.
12. [Motive → Builder pane] If the results are satisfying, press Apply to calibrate the HMD rigid body.

Oculus Positional Sensor

Oculus Rift Sensor In order to use the Oculus Rift HMDs, it's positional sensor must be connected to the hosting computer for the HMD to work. When this sensor is connected, Oculus HMDs will start illuminating its own IR LEDs embedded on its frontal panel. These LEDs can add noise in the IR spectrum and interfere with tracking. If you are using the CV1 clip, this shouldn't matter as the HMD's frontal panel will be already covered up. If you are manually attaching the markers, it's suggested to cover up the HMD's frontal panel to block these LEDs once the corresponding rigid body definition has been created in Motive.

Rigid Body Properties

Once the rigid body is created, select the asset and open the Properties pane. The following settings need to be configured for HMD tracking.

Rigid body properties for the HMD asset.

Deflection

Deflection setting is the tolerable distance, in millimeters, that a rigid body marker may deviate from its expected position before it is unlabeled and unassociated with the rigid body. The deflection is set to 4 mm by default. For HMD tracking applications, we recommended lowering this value to 3 or 2. This will reduce the amount of computation required for labeling, and overall latency may be reduced.

Tracking Algorithm

The tracking algorithm setting determines which protocol to use for solving the rigid body in Motive.

Passive Markers

Use the ray-based tracking algorithm for HMDs with passive markers. This algorithm is recommended because it allows tracking to be less susceptible to marker occlusions and solves rigid bodies more robustly without introducing trajectory gaps. Using ray-based solving will increase processing load, but since it can stabilize the tracking a lot, this is more suitable for tracking the passive marker HMDs where even minor occlusions can acutely affect the experience.

Active Markers

Use the marker-based tracking algorithm for HMDs using active marker clips. Ray-based tracking is not supported for active markers since active markers themselves already provide stable tracking from active labeling using unique illumination patterns. Thus, marker-based tracking will provide robust tracking for HMDs with active markers.

Streaming ID

The User Data value for each rigid body is used as an identification number to reference the rigid body in external applications. Log this value for each rigid body (including the HMD(s)) in the scene. This number will be used to associate the assets in the client applications using the plugin devices.

Min Marker Count (Passive Markers only)

This setting may need to be modified when tracking HMDs with passive markers. To prevent the swapping of the rigid body definition, set this setting to 4. When tracking multiple HMDs, there could be limitations to a variety of unique marker arrangements that could be achieved. If this value is set to a lower value, a set of three markers on an HMD may be congruent to another set in a different HMD, and the rigid body definitions may be switched in Motive.

Recalibrating Pivot Point

After the rigid body definition have been created for the HMD, the position and orientation of the rigid body pivot point should get placed precisely on the root of the player's nose with the respective orientation axis pointing the forward direction. If needed, you can use the Builder pane edit tools to recalibrate the HMD pivot point positions.

• The pivot point is placed on the bridge of the nose; specifically, right at the midpoint between two eyes.
• Orientation axes of the rigid body should precisely align with the physical HMD component.

• 

  Uncalibrated HMD rigid body definition. The pivot point is not positioned nor oriented properly.
• 

  Calibrated HMD rigid body definition. x-axis forward for UE4 integration.

Manually Calibrating the HMD Pivot Point

In most cases, you will be utilizing the previously described calibration tool for precisely positioning and orienting the HMD rigid body. However, if, for some reason, you want to manually position the pivot point without using the HMD Calibration feature, follow the instruction on this page: Manually Calibrating the HMD Pivot Point.

Export the Rigid Body

Tip: Once you have the rigid body asset for the HMD configured, you can export the asset into a MOTIVE file (*.motive) for future uses. Importing the MOTIVE file (e.g. CV1.tra) will load the rigid body (HMD) asset and make it available for use; however, the marker placement must remain unchanged in order to re-load previously created rigid bodies.

Streaming into Unity

---

The plugin also features HMD tracking integration for virtual reality experiences. For general instructions on developing VR application in Unity, refer to the Unity documentation: Unity Virtual Reality tutorial. Please note that the following instructions assume that the OptiTrack Unity Plugin has been set up for streaming. This means the plugin must have been installed and the Client Origin should have been imported onto the scene already. For information on setting up the OptiTrack Unity Plugin

Oculus DK2, CV1, and Gear VR HMD models are supported

OculusAndroid: GearVR

Oculus DK2, CV1

1. [Motive] First of all, follow the HMD Tracking Setup section and prepare HMD rigid bodies in Motive.
2. Make sure the HMD software is running.
3. [Unity] In the Unity project, check the Virtual Reality Supported box from the player settings (Edit → Project Settings → Player), as shown in the image below.
4. [Unity] From the Prefabs folder (OptiTrack/Prefabs) of the imported plugin package, import the HMD prefab (HMD - OptiTrack) object into the scene.
5. [Unity] In the Streaming Client entry, specify the object which the streaming client is attached to. By default, it searches for an existing client instance, but this must be specified when there are more than one streaming client objects.
6. [Unity] For the Rigid Body ID entry, enter the User Data ID value given for the HMD rigid body in Motive.
7. [Unity] For the Rigid Body Orientation entry, enter the orientation of the rigid body coordinate axis that is defined in Motive.
8. [Unity] Play the scene. When the HMD is properly connected, the scene will be visible from the HMD.

• 

  HMD object and its components in Unity
• 

  VR Supported setting in Unity.

OVRManager Having OVRManager enabled within the project may introduce an additional offset to the position of the HMD. If you are seeing positional offset in the HMD position, remove the OVRManager object.

Android: Gear VR

Gear VR devices can also be integrated and the tracking information can be streamed via wifi connections using a router with sufficient bandwidth. The required bandwidth will vary depending on many factors (e.g. router type, the number of tracked object, etc.). For more specific information on this setup, contact us. The following settings must be configured in addition to the above HMD settings for developing Gear VR experiences using the plugin. For more information on developing Gear VR applications in Unity, refer to Unity documentation.

Related Pages

---

• OptiTrack Unity Plugin
• Unity: Vive HMD Tracking
• Rigid Body Tracking
• OptiTrack Unreal Engine 4 Plugin