Difference between revisions of "OptiTrack Unreal Engine 4 Plugin"

Line 6: Line 6:
 
'''Plugin Overview'''
 
'''Plugin Overview'''
 
<div class="padded">
 
<div class="padded">
The [http://optitrack.com/downloads/plugins.html OptiTrack Unreal Engine Plugin] allows you to stream real-time rigid body, skeleton, and HMD tracking data from Motive into Unreal Engine 4 (UE4). This article focuses on how to set up those different streaming workflows. For instructions on setting up the the motion capture system, please refer to the [[Quick_Start_Guide:_Getting_Started|Getting Started]] guide instead.
+
The [http://optitrack.com/downloads/plugins.html OptiTrack Unreal Engine Plugin] allows you to stream real-time tracking data from Motive into Unreal Engine 4 (UE4).
 +
Using this plugin, you can stream rigid body tracking data, skeleton tracking data, and/or HMD tracking data into a UE4 project. This article focuses on how to set up for those different streaming workflows. For basic instructions on how to set up a motion capture system, please refer to the [[Quick_Start_Guide:_Getting_Started|Getting Started]] guide instead.
  
The OptiTrack Unreal Engine Plugin is divided into two plugins:
+
The OptiTrack Unreal Engine Plugin is divided into two parts:
 
* The '''OptiTrack - NatNet Streaming Client''' which allows you to stream rigid body and skeleton data from Motive onto Unreal Engine.
 
* The '''OptiTrack - NatNet Streaming Client''' which allows you to stream rigid body and skeleton data from Motive onto Unreal Engine.
* The '''OptiTrack - OculusVR plugin''' which allows you to implement HMD tracking onto your scenes.
+
* The '''OptiTrack - OculusVR plugin''' which allows you to implement tracking of Oculus HMDs onto your scenes.
  
Once the plugins are enabled, they receive the streaming data from Motive through an ''OptiTrack Client Origin'' object in your Unreal Engine level.  
+
Once the plugins are enabled, they can be used to connect and receive streaming data from Motive by setting up an ''OptiTrack Client Origin'' object in the Unreal Engine level.  
 
</div>
 
</div>
 +
 +
 +
'''HMD Compatibility'''
 +
<div class="padded">
 +
*As of version 1.2.1, the OptiTrack - OculusVR plugin supports Oculus DK2, CV1, and Gear VR HMD models.
 +
*Tracking of Vive HMDs can be integrated using the [[Unreal Engine: Vive HMD Tracking|OptiTrack OpenVR Driver]].
 +
</div>
 +
  
 
'''Version Requirements'''
 
'''Version Requirements'''
Line 72: Line 81:
 
|style = "padding:0.5em;"|Latest
 
|style = "padding:0.5em;"|Latest
 
|}
 
|}
{{Info|'''Notes on HMD Compatibility'''
 
 
*As of OptiTrack Unreal Engine 4 Plugin 1.2.1, the OptiTrack Oculus plugin supports Oculus DK2, CV1, and Gear VR HMD models.
 
*Vive HMDs can be integrated through the [[OptiTrack OpenVR Driver]].
 
}}
 
 
</div>
 
</div>
  
Line 82: Line 86:
 
=Motive Data Streaming Setup (Server)=
 
=Motive Data Streaming Setup (Server)=
 
----
 
----
First of all, follow the below instructions and set up the data streaming settings in Motive. Once this is configured, Motive will be broadcasting tracking data onto a designated network interface where client applications can receive them.  
+
First of all, follow the below instructions to set up the data streaming settings in Motive. Once this is configured, Motive will be broadcasting tracking data onto a designated network interface where client applications can receive them.  
  
==Streaming in Motive==
+
==Streaming Settings==
 
<div class="padded">
 
<div class="padded">
 
[[File:UEP_MotiveStream_20.png|right|Broadcast Frame Data set to true for streaming.]]
 
[[File:UEP_MotiveStream_20.png|right|Broadcast Frame Data set to true for streaming.]]
Open the [[Data Streaming pane|streaming settings]] in Motive and set the values below:
+
Open the [[Data Streaming pane]] in Motive and set the following settings:
* Enable '''Broadcast Frame Data''' setting at the top
+
* Enable the '''Broadcast Frame Data''' setting at the top
* From '''Local Interface''' drop-down menu, select a network address that you wish to stream the data over to.  
+
* From the '''Local Interface''' drop-down menu, select a network address that you wish to stream the data over to.  
* Select desired data types to stream under streaming options. Streaming of rigid body is required. Streaming of skeleton is optional for skeleton tracking. For tracking HMDs, disabling the ''Marker'' streaming is adviced.
+
* Select desired data types to stream under streaming options:
 +
::- Streaming of rigid bodies is required.  
 +
::- Streaming of skeleton is optional for skeleton tracking.  
 +
::- For tracking HMDs, disabling the ''Marker'' streaming is adviced.
 
* Set the '''Skeleton Coordinates''' to ''Local''.
 
* Set the '''Skeleton Coordinates''' to ''Local''.
* When streaming skeletons, set the appropriate bone naming convention for the client application. (If the character uses the FBX naming convention, this will need to be set to FBX.)
+
* When streaming skeletons, set the appropriate bone naming convention for the client application. If the character uses the FBX naming convention, this will need to be set to FBX.
 
* For streaming over a wifi network, setting the '''Transmission Type''' to ''Unicast'' is strongly adviced.
 
* For streaming over a wifi network, setting the '''Transmission Type''' to ''Unicast'' is strongly adviced.
  
 
{{Info2|'''Additional Tips'''
 
{{Info2|'''Additional Tips'''
* In order to stream data from Edit mode, a capture-recording must be playing back in Motive.
 
 
* For best results, it is advised to run Motive and Unreal Engine separately on different computers, so that they are not competing for processing resources.
 
* For best results, it is advised to run Motive and Unreal Engine separately on different computers, so that they are not competing for processing resources.
 
* When streaming the data over a wifi network, ''Unicast'' transmission must be used.
 
* When streaming the data over a wifi network, ''Unicast'' transmission must be used.
 +
* In order to stream data from the Edit mode, a capture-recording must be playing back in Motive.
 
*For additional information on data streaming in general, read through the [[Data Streaming]] page.
 
*For additional information on data streaming in general, read through the [[Data Streaming]] page.
 
}}
 
}}
Line 105: Line 112:
 
=Unreal Engine Setup (Client)=
 
=Unreal Engine Setup (Client)=
 
----
 
----
Next step is to set up the client.  Follow the below instructions to install and configure the Unreal Engine plugin to receive the streamed tracking data.  
+
Next step is to configure the client plugin inside UE4Please follow below instructions to install and configure the OptiTrack Unreal Engine plugin to receive the streamed tracking data.  
  
==Setting Up the Plugin==
+
==Enable the Plugin==
# Download the plugin ZIP file.
+
# Download the plugin ZIP file: [https://optitrack.com/downloads/plugins.html download page]
 
# Extract the contents from the ZIP file.
 
# Extract the contents from the ZIP file.
# Open the OptiTrack folder, transfer the '''OptiTrackNatNet''' and '''OptiTrackOculusRift''' folders into the Unreal Engine plugin directory. By default, this directory is installed in the <code>C:\Program Files\Epic Games\4.##\Engine\Plugins</code> folder. There will be other plugins installed in the folder already. Simply copy and paste the OptiTrack plugin folders into this directory.
+
# Open the extracted OptiTrack folder, transfer the '''OptiTrackNatNet''' and '''OptiTrackOculusRift''' folders into the Unreal Engine's plugin directory. By default, this directory is installed in the <code>C:\Program Files\Epic Games\4.##\Engine\Plugins</code> folder. There will be other plugins installed in the folder already. Simply copy and paste the OptiTrack plugin folders into this directory.
 
# Open/Create a UE4 project.
 
# Open/Create a UE4 project.
# Under the Edit menu, open the ''Plugin Panel''.
+
# Under the Edit menu, click ''Plugin'' to open up the panel where all of the available plugins are listed.
# Browse to Virtual Reality section and enable the OptiTrack plugins (OptiTrack - NatNet Streaming Client and OptiTrack - OculusVR) that are added to the folder.
+
# Browse to OptiTrack section and enable the OptiTrack - NatNet Streaming Client.
 +
# (optional) For Oculus HMD users, browse to OptiTrack section and enable OptiTrack - OculusVR plugin for Oculus HMD tracking. After this is enabled, please make sure to disable the built-in Oculus Rift and Stream VR HMD plugins under virtual reality plugins category. Otherwise, the editor may crash.
 +
# Click ''Apply'' to submit the changes. It may require the UE4 project to be restarted.
  
 
<div class="padded">
 
<div class="padded">
 
[[Image:UEP_PluginPanel.png|thumb|700 px|center|UE4 plugins panel listing the enabled OptiTrack plugins.]]
 
[[Image:UEP_PluginPanel.png|thumb|700 px|center|UE4 plugins panel listing the enabled OptiTrack plugins.]]
 
{{Info|'''Note:''' When using the Oculus Plugin, make sure to disable the built-in Oculus Rift (or the Gear VR) and Stream VR HMD plugins under virtual reality plugins category. Otherwise, the editor may crash, or they may end up having a higher priority than the OptiTrack Oculus Plugin, and if enabled, will be used in preference to it.}}
 
 
</div>
 
</div>
  
==Client Origin Setup==
+
==Set up the Client Origin==
 
<div class="padded">
 
<div class="padded">
 +
Once the OptiTrack - NatNet Streaming Client plugin is enabled, OptiTrack Client Origin will be available in the Modes panel within UE4. This is the baseline node for making the connection to Motive.
 +
 
====OptiTrack Client Origin====
 
====OptiTrack Client Origin====
 
[[Image:UEP_ModesPanel.png|thumb|Once the plugin is properly added, the Client Origin Object can be found under Modes panel in UE4.]]
 
[[Image:UEP_ModesPanel.png|thumb|Once the plugin is properly added, the Client Origin Object can be found under Modes panel in UE4.]]
 
<div class="padded">
 
<div class="padded">
The OptiTrack Client Origin enables the Unreal Engine (client) to communicate with the NatNet server where the Motive’s rigid body, skeleton, and HMD tracking data are streamed.
+
The ''OptiTrack Client Origin'' class enables the Unreal Engine (client) to communicate with Motive where the rigid body, skeleton, and HMD tracking data are streamed from.
  
Add this object simply by dragging and dropping the OptiTrack Client Origin from the Modes panel into the scene. Once the OptiTrack Client Origin object is placed in the scene, its position and orientation will reconcile the global origin of Motive in Unreal Engine. In other words, the tracking data will be represented according to where this Client Origin object is positioned and how it is oriented.
+
This can be added simply by drag-and-dropping the OptiTrack Client Origin from the Modes panel into the level in UE4. Once the client origin is placed within the level, its position and orientation will reconcile the global origin of Motive in Unreal Engine. In other words, the tracking data will be represented according to where this Client Origin object is positioned and how it is oriented.
  
 
{{Info2|'''Global Origin:''' Both position and orientation of the OptiTrackClientOrigin will represent the global origin of the tracking volume within Motive.}}
 
{{Info2|'''Global Origin:''' Both position and orientation of the OptiTrackClientOrigin will represent the global origin of the tracking volume within Motive.}}
Line 155: Line 164:
 
{{Info|
 
{{Info|
 
'''Advance settings: Auto-initialize'''
 
'''Advance settings: Auto-initialize'''
:In the plugin version 1.8 and above, you can set up the project so at the client origin gets initialized at a user-defined event. This setting is enabled by default. But when there is a need to provide the server IP address dynamically within the game, you can disable the ‘’auto-initialize’’ feature and configure using blueprint how and when the client origin gets initialized.
+
:By default, the auto-initialize feature is enabled and the client origin will get auto-initialized whenever the scene is played. But when needed, you can disable this and set up the project so the client origin gets initialized when a user-defined event is triggered. For example, if there is a need to provide the server IP address dynamically within a game, you can disable this and configure using blueprint how and when the client origin gets initialized. ''This is available in plugin version 1.8 and above.''
  
 
[[Image:UEP_AutoInitialize_420.png|center]]
 
[[Image:UEP_AutoInitialize_420.png|center]]
Line 164: Line 173:
 
<div class="padded">
 
<div class="padded">
 
====OptiTrack Rigid Body Actor====
 
====OptiTrack Rigid Body Actor====
: Actor objects in Unreal Engine can be animated using rigid body tracking data from Motive. Once the ''OptiTrack - NatNet Streaming Client'' plugin is enabled in the project, ''OptiTrack Rigid Body'' component will be available to use. By attaching this component to an actor, you can animate its ''child'' actors according to the movement of a rigid body in Motive. Each rigid body component is given a ''Trackable ID'' value and rigid bodies in Motive with the matching [[Properties:_Rigid_Body#General_Settings|Streaming ID]] will be used to update the transform of the target actor in Unreal Engine. Please note that the ''Hide on Invalid Definition'' component property is set to true, by default, and the rigid body actor will be visible within the scene only when the corresponding rigid body data is streamed and received by the client.
+
<div class="padded">
 +
Actor objects in Unreal Engine can be animated using rigid body tracking data from Motive. Once the ''OptiTrack - NatNet Streaming Client'' plugin is enabled in the project, ''OptiTrack Rigid Body'' component will be available to use. By attaching this component onto an actor, you can animate its ''child'' actors according to the movement of a rigid body in Motive. Each rigid body component is given a ''Trackable ID'' value, and tracking information of rigid body assets, in Motive, with the matching [[Properties:_Rigid_Body#General_Settings|Streaming ID]] will be used to update the transform of the target actor in Unreal Engine.  
  
: You can also use the ''OptiTrack Rigid Body Actor'', which will be available on the Modes panel. This actor is simply a blank actor with just the ''OptiTrack Rigid Body'' component attached to it, and you can use this to easily associate the objects within the scene:
+
You can also use the ''OptiTrack Rigid Body Actor'' prefab under the Modes panel. This is simply a blank actor with just the ''OptiTrack Rigid Body'' component attached to it, and you can use this to easily associate the objects within the scene:
 +
</div>
  
<div class="padded">
+
====Set Up Steps====
'''Steps'''
+
# '''[UE4]''' From the Modes panel, search for ''OptiTrack Rigid Body Actor''. Then drag-and-drop the actor into the scene.
# '''[UE4]''' From the Modes panel, search for ''OptiTrack Rigid Body Actor''. Then drag and drop the actor into the scene.
+
# '''[UE4]''' Under this rigid body actor, attach the target actor that you wish to animate. Please make sure the target actor's transformation is set to movable.  
# '''[UE4]''' Under this rigid body actor, attach the target actor that you wish to animate.
+
# '''[UE4]''' Set the relative locations and rotations to all zeroes on this target actor. This actor should be listed as a child of the rigid body actor.
# '''[UE4]''' Set the relative location and rotation to zero on this target actor, which should be a child of the rigid body actor at this point.
 
 
# '''[Motive]''' In Motive, assign a value to ''Streaming ID'' property for the target rigid body.
 
# '''[Motive]''' In Motive, assign a value to ''Streaming ID'' property for the target rigid body.
 
# '''[UE4]''' In the properties of the ''OptiTrack Rigid Body Actor'' component, match the Tracking ID with the Streaming ID of the rigid body asset in Motive.
 
# '''[UE4]''' In the properties of the ''OptiTrack Rigid Body Actor'' component, match the Tracking ID with the Streaming ID of the rigid body asset in Motive.
# Make sure both Motive and ''OptiTrack Client Origin'' is set up for streaming, hit ''Play'', and the attached actor object will be animated according to the live-streamed rigid body tracking data.</div>
+
# Make sure both Motive and ''OptiTrack Client Origin'' is set up for streaming, hit ''Play'', and the attached actor object will be animated according to the live-streamed rigid body tracking data.
  
 
<center><ul>
 
<center><ul>
Line 183: Line 193:
 
</ul></center>
 
</ul></center>
  
 
{{Info|
 
 
====RigidBodyComponent Properties====
 
====RigidBodyComponent Properties====
 
{{Indent|
 
{{Indent|
Line 201: Line 209:
 
=====Respect Parent Transform=====
 
=====Respect Parent Transform=====
 
:When this is set to true, the rigid body transform data from Motive will be applied in respect to the parent actor's pivot coordinates. By default, this is set to false, and all of the tracking data will be applied in respect to the pivot axis of the client origin.
 
:When this is set to true, the rigid body transform data from Motive will be applied in respect to the parent actor's pivot coordinates. By default, this is set to false, and all of the tracking data will be applied in respect to the pivot axis of the client origin.
}}}}
+
}}
 
</div>
 
</div>
  
 
==Animating Skeleton==
 
==Animating Skeleton==
 
<div class="padded">
 
<div class="padded">
Characters in Unreal Engine can be animated using [[Skeleton Tracking|skeleton tracking]] data in Motive. There are two different approaches to this: skeleton bone mapping and skeleton retargetting.
+
Characters in Unreal Engine can be animated using [[Skeleton Tracking|skeleton tracking]] data from Motive. There are two different approaches to this: skeleton bone mapping and skeleton retargetting.
  
 
====Bone Mapping====
 
====Bone Mapping====
 
<div class="padded">
 
<div class="padded">
For plugin versions above 1.9.2, the skeleton component can be used to allow streaming of the skeleton via bone mapping. This means that the motion of each skeleton bone in Motive can be projected to the bones of the character model. Please note that this is different from skeleton retargeting, the bone segments in Motive get mapped directly to the character model (bone mapping) instead of being translated into something that is usable by a more abstract biped model (retargeting). The instructions on setting up the skeleton component are documented in the following page:
+
For plugin versions above 1.9.2, the skeleton component can be used to allow streaming of the skeleton via bone mapping. This means that the motion of each skeleton bone in Motive can be projected to the bones of the character model. Please note that this is different from skeleton retargeting, the bone segments in Motive get mapped directly to the character model (bone mapping) instead of being translated into something that is usable by a more abstract biped model (retargeting). The instructions on setting up the skeleton component are documented under the following page:
  
 
Go to: [[Unreal Engine Skeleton Streaming]]  
 
Go to: [[Unreal Engine Skeleton Streaming]]  
Line 217: Line 225:
 
====Skeleton Retargetting====
 
====Skeleton Retargetting====
 
<div class="padded">
 
<div class="padded">
Skeleton tracking data can also be integrated into Unreal Engine via a 3D retargeting process. This can be accomplished through a third party software that retargets the 3D data to produce realistic motion in a scene. For more information, [http://optitrack.com/contact/ contact us].
+
Skeleton tracking data can also be integrated into Unreal Engine via a 3D retargeting process. This can be accomplished through a third party software, such as IKinema, that retargets captured 3D data onto a character to produce realistic motion in a scene. For more information, [http://optitrack.com/contact/ contact us].
  
 
{{Info|'''Notes for IKinema LiveAction for UE 4.18.'''
 
{{Info|'''Notes for IKinema LiveAction for UE 4.18.'''
Line 239: Line 247:
 
</div>
 
</div>
  
==Integrating HMD (Oculus DK2, CV1)==
+
=Intergrating HMDs=
<div class="padded">
+
----
To integrate an HMD, make sure the '''OptiTrack – OculusVR''' is enabled in your project under the Plugins panel and follow the below steps to link the HMD tracking data.
+
OptiTrack motion capture systems can be used to track head mounted displays (HMD) and integrate the tracking data into Unreal Engine for VR applications. For instructions on integrating HMD tracking data into Unreal Engine, please refer to the corresponding pages:
  
[[Image:UEP_OculusVRPlugin.png|center|700px]]
+
* [[Unreal Engine: Oculus HMD Tracking]]
 +
* [[Unreal Engine: Vive HMD Tracking]]
  
====Linking the HMD Tracking Data====
+
{{Info|
# '''[UE4]''' Enable the OptiTrack – Oculus Rift plugin from the plugins panel.
+
====Supported HMDs====
# '''[UE4]''' Open the ''World Settings'' panel under the ''Window'' tab.
+
{{Indent|At the time of writing, the following HMDs are supported:  
# '''[UE4]''' If not done already, create a new GameMode by adding a new blueprint under the Game Mode section.
+
*HTC Vive
# '''[UE4]''' Change the ''Default Pawn Class'' to OptiTrackHmdPawn. You may need to restart the editor before this change takes effect.
+
*HTC Vive Pro
# '''[UE4]''' Select the OptitrackClientOrigin and access its settings under the Details panel.
+
*Oculus DK2
# '''[UE4]''' Expand the OptiTrack properties section.
+
*Oculus CV1
# '''[UE4]''' In the ''HMD Rigid Body ID'' entry, input the respective [[Properties:_Rigid_Body#Streaming ID|streaming ID]] of the HMD rigid body in Motive.
+
*Oculus GearVR
# '''[UE4]''' In the HMD Orientation section, make sure the axis orientation matches the axis orientation of the HMD rigid body in Motive.
+
}}}}
# '''[UE4]''' Now the tracking data is linked to the Unreal Engine project, and the HMD will display the scene when it is played.
 
 
 
<center><ul>
 
<li class="thumblist">[[Image:UEP_PawnClass.png|thumb|Defining the default pawn class from World Setttings: GameMode in UE4.]]</li>
 
<li class="thumblist">[[Image:UEP_UserDataHMD_420.png|thumb|Inputting the Streaming ID information for tracking the HMD.]]</li>
 
</ul></center>
 
</div>
 
 
 
==Integrating HMD (Android: GearVR)==
 
[[Image:ClientOriginGearVR_420.png|thumb|Client origin configuration for GearVR]]
 
<div class="padded">
 
The plugin also support integration of GearVR HMDs. The HMD tracking data can be streamed over a WiFi network, and in order to do so, a router with sufficient bandwidth must be used. The required bandwidth will vary depending on many factors (e.g. router type, the number of tracked object, etc.). The following settings must be configured in addition to the above HMD settings for developing Gear VR experiences using the plugin. For more information on developing Gear VR applications in Unreal Engine, refer to UE 4 documentations.
 
 
 
====Settings====
 
:*'''[UE 4]''' For developing Android applications in Unreal Engine, make sure the project environment is set up for Android development: [https://docs.unrealengine.com/latest/INT/Platforms/Android/GettingStarted/ Android Quick Start Guide ].
 
:*'''[UE4]''' Follow the [https://docs.unrealengine.com/latest/INT/Platforms/GearVR/QuickStart/index.html GearVR Quick Start Guide] and configure the project for GearVR development.
 
:*'''[UE4]''' Generate and include the OSIG file (oculus signature) in the directory.  See: [https://developer.oculus.com/osig/ Oculus Signature File Generator].
 
:* '''[Motive]''' Under the [[Data Streaming pane]], set the Network Type setting under the Advanced Settings to '''''Unicast'''''. Note that the plugin currently only supports the Unicast broadcasting for streaming onto Android, multicasting will be supported in the future releases.
 
 
 
====Connecting via Wifi====
 
<div class="padded">
 
:1. '''[Android]''' Connect the smartphone to the internet router which the host PC (server) is connected to.
 
:2. '''[UE4]''' Configure the Client object.
 
:: - Use Connection Type: Unicast.
 
:: - Enable ''Auto Connect'', unless there is a need to designate the network address.
 
:3. '''[UE4]''' Connect a smartphone into the PC, build and run the project.
 
:4. Make sure Motive is streaming, and the Gear VR HMD tracking data will be streamed into the UE4 application.
 
 
 
{{Info|'''Connecting to a Designated Network.
 
: - Set the Local IP address to (0.0.0.0). The local address is not required for unicast connections.
 
: - For the Server Address, connect to the IP address that the host PC (server) is connected to. This could also be found from the Android OS when the smartphone is connected to the wifi router.
 
}}
 
</div>
 
</div>
 
 
 
{{HMD Setup}}
 
 
 
==Export the Rigid Body==
 
<div class="padded">
 
{{Info|'''Tip:''' Once you have the rigid body asset for the HMD configured, you can export the asset into a [[Motive_Basics#Motive_User_Profile_.28.MOTIVE.29|MOTIVE file (*.motive)]] for future uses. Importing the MOTIVE file (e.g. CV1.tra) will load the rigid body (HMD) asset and make it available for use; however, the marker placement '''must''' remain unchanged in order to re-load previously created rigid bodies.}}
 
</div>
 
  
 
=Live Streaming=
 
=Live Streaming=

Revision as of 15:57, 15 July 2019

Back to the Main page


Plugin Overview

The OptiTrack Unreal Engine Plugin allows you to stream real-time tracking data from Motive into Unreal Engine 4 (UE4). Using this plugin, you can stream rigid body tracking data, skeleton tracking data, and/or HMD tracking data into a UE4 project. This article focuses on how to set up for those different streaming workflows. For basic instructions on how to set up a motion capture system, please refer to the Getting Started guide instead.

The OptiTrack Unreal Engine Plugin is divided into two parts:

  • The OptiTrack - NatNet Streaming Client which allows you to stream rigid body and skeleton data from Motive onto Unreal Engine.
  • The OptiTrack - OculusVR plugin which allows you to implement tracking of Oculus HMDs onto your scenes.

Once the plugins are enabled, they can be used to connect and receive streaming data from Motive by setting up an OptiTrack Client Origin object in the Unreal Engine level.


HMD Compatibility

  • As of version 1.2.1, the OptiTrack - OculusVR plugin supports Oculus DK2, CV1, and Gear VR HMD models.
  • Tracking of Vive HMDs can be integrated using the OptiTrack OpenVR Driver.


Version Requirements

This plugin is supported for the following versions of the Unreal Engine, and it also requires the corresponding version of Oculus runtime as indicated in the table.

Plugin Version UE Version Oculus SDK
1.0 UE 4.10 0.8
1.1 UE 4.11 1.3
1.2 UE 4.12 Latest
1.3 UE 4.13 Latest
1.4 UE 4.14 Latest
1.5 UE 4.15 Latest
1.6 UE 4.16 Latest
1.7 UE 4.17 Latest
1.8 UE 4.18.3 Latest
1.9 UE 4.19.2 Latest
1.20 UE 4.20.3 Latest
1.21 UE 4.21.2 Latest


Motive Data Streaming Setup (Server)


First of all, follow the below instructions to set up the data streaming settings in Motive. Once this is configured, Motive will be broadcasting tracking data onto a designated network interface where client applications can receive them.

Streaming Settings

Broadcast Frame Data set to true for streaming.

Open the Data Streaming pane in Motive and set the following settings:

  • Enable the Broadcast Frame Data setting at the top
  • From the Local Interface drop-down menu, select a network address that you wish to stream the data over to.
  • Select desired data types to stream under streaming options:
- Streaming of rigid bodies is required.
- Streaming of skeleton is optional for skeleton tracking.
- For tracking HMDs, disabling the Marker streaming is adviced.
  • Set the Skeleton Coordinates to Local.
  • When streaming skeletons, set the appropriate bone naming convention for the client application. If the character uses the FBX naming convention, this will need to be set to FBX.
  • For streaming over a wifi network, setting the Transmission Type to Unicast is strongly adviced.

Info2.png

Additional Tips

  • For best results, it is advised to run Motive and Unreal Engine separately on different computers, so that they are not competing for processing resources.
  • When streaming the data over a wifi network, Unicast transmission must be used.
  • In order to stream data from the Edit mode, a capture-recording must be playing back in Motive.
  • For additional information on data streaming in general, read through the Data Streaming page.

Unreal Engine Setup (Client)


Next step is to configure the client plugin inside UE4. Please follow below instructions to install and configure the OptiTrack Unreal Engine plugin to receive the streamed tracking data.

Enable the Plugin

  1. Download the plugin ZIP file: download page
  2. Extract the contents from the ZIP file.
  3. Open the extracted OptiTrack folder, transfer the OptiTrackNatNet and OptiTrackOculusRift folders into the Unreal Engine's plugin directory. By default, this directory is installed in the C:\Program Files\Epic Games\4.##\Engine\Plugins folder. There will be other plugins installed in the folder already. Simply copy and paste the OptiTrack plugin folders into this directory.
  4. Open/Create a UE4 project.
  5. Under the Edit menu, click Plugin to open up the panel where all of the available plugins are listed.
  6. Browse to OptiTrack section and enable the OptiTrack - NatNet Streaming Client.
  7. (optional) For Oculus HMD users, browse to OptiTrack section and enable OptiTrack - OculusVR plugin for Oculus HMD tracking. After this is enabled, please make sure to disable the built-in Oculus Rift and Stream VR HMD plugins under virtual reality plugins category. Otherwise, the editor may crash.
  8. Click Apply to submit the changes. It may require the UE4 project to be restarted.
UE4 plugins panel listing the enabled OptiTrack plugins.

Set up the Client Origin

Once the OptiTrack - NatNet Streaming Client plugin is enabled, OptiTrack Client Origin will be available in the Modes panel within UE4. This is the baseline node for making the connection to Motive.

OptiTrack Client Origin

Once the plugin is properly added, the Client Origin Object can be found under Modes panel in UE4.

The OptiTrack Client Origin class enables the Unreal Engine (client) to communicate with Motive where the rigid body, skeleton, and HMD tracking data are streamed from.

This can be added simply by drag-and-dropping the OptiTrack Client Origin from the Modes panel into the level in UE4. Once the client origin is placed within the level, its position and orientation will reconcile the global origin of Motive in Unreal Engine. In other words, the tracking data will be represented according to where this Client Origin object is positioned and how it is oriented.

Info2.png

Global Origin: Both position and orientation of the OptiTrackClientOrigin will represent the global origin of the tracking volume within Motive.

Connecting Unreal Engine to Motive

  1. [Motive] First of all, make sure the Broadcast Frame Data entry is checked in the Data Streaming pane of Motive.
  2. [UE4] Once the plugin is added and enabled in the project, OptiTrack Client Origin class will be available from the Modes panel.
  3. [UE4] Drag and drop the OptiTrack Client Origin into the scene.
  4. [UE4] Place the OptiTrack Client Origin to the desired tracking origin within the scene.
  5. [UE4] Select the instantiated OptiTrackClientOrigin object from the World Outliner panel.
  6. [UE4] In the Details panel, make sure its Auto Connect setting is checked. This configures the client origin to automatically search the network and connect to Motive.
  7. Now that the client origin is set, the client origin will attempt to connect to Motive and start receiving the tracking data whenever the scene is played.
  • Data streaming settings in Motive
  • OptiTrack Client Origin properties defined with corresponding server and client address.

Info2.png

Connecting to a designated IP address

If you wish to connect to a server on a specific network address, you can uncheck the Auto Connect setting and enter the IP address of the client where the instance of UE is running and also the IP address of the server which was designated in the Data Streaming pane in Motive. You may need to run the ipconfig command in the command prompt to obtain an appropriate IP address of the client.
UEP DesignateIP.png

Info2.png

Advance settings: Auto-initialize

By default, the auto-initialize feature is enabled and the client origin will get auto-initialized whenever the scene is played. But when needed, you can disable this and set up the project so the client origin gets initialized when a user-defined event is triggered. For example, if there is a need to provide the server IP address dynamically within a game, you can disable this and configure using blueprint how and when the client origin gets initialized. This is available in plugin version 1.8 and above.
UEP AutoInitialize 420.png

Animating Rigid Body

OptiTrack Rigid Body Actor

Actor objects in Unreal Engine can be animated using rigid body tracking data from Motive. Once the OptiTrack - NatNet Streaming Client plugin is enabled in the project, OptiTrack Rigid Body component will be available to use. By attaching this component onto an actor, you can animate its child actors according to the movement of a rigid body in Motive. Each rigid body component is given a Trackable ID value, and tracking information of rigid body assets, in Motive, with the matching Streaming ID will be used to update the transform of the target actor in Unreal Engine.

You can also use the OptiTrack Rigid Body Actor prefab under the Modes panel. This is simply a blank actor with just the OptiTrack Rigid Body component attached to it, and you can use this to easily associate the objects within the scene:

Set Up Steps

  1. [UE4] From the Modes panel, search for OptiTrack Rigid Body Actor. Then drag-and-drop the actor into the scene.
  2. [UE4] Under this rigid body actor, attach the target actor that you wish to animate. Please make sure the target actor's transformation is set to movable.
  3. [UE4] Set the relative locations and rotations to all zeroes on this target actor. This actor should be listed as a child of the rigid body actor.
  4. [Motive] In Motive, assign a value to Streaming ID property for the target rigid body.
  5. [UE4] In the properties of the OptiTrack Rigid Body Actor component, match the Tracking ID with the Streaming ID of the rigid body asset in Motive.
  6. Make sure both Motive and OptiTrack Client Origin is set up for streaming, hit Play, and the attached actor object will be animated according to the live-streamed rigid body tracking data.
  • Streaming ID of a selected rigid body asset in Motive.
  • Once the OptiTrack NatNet streaming client plugin is properly installed, the OptiTrack Rigid Body component will be available within the components list in UE4.
  • Within the OptiTrack Rigid Body Component, input the Streaming ID for the corresponding rigid body asset in Motive.

RigidBodyComponent Properties

Tracking ID
ID of the rigid body used to derive the position and orientatation transform of the attached actor. This ID must match with the Streaming ID of the respective rigid body in Motive.
Hide on Invalid Definition
When this is checked, the corresponding rigid body actor will be hidden from the level until the associated rigid body data is streamed out from Motive and received by the plugin.
Disable Low Latency Update
Low latency update feature allows rigid body position and orientation transform to be updated immediately before rendering minimizing the latency. This is enabled by default. For debugging, you can check this setting to disable this behavior.
Tracking Origin
This sets a specific client origin to use for receiving tracking data. When this is unset, the plugin will default to the first client origin that it finds in the scene.
Respect Parent Transform
When this is set to true, the rigid body transform data from Motive will be applied in respect to the parent actor's pivot coordinates. By default, this is set to false, and all of the tracking data will be applied in respect to the pivot axis of the client origin.

Animating Skeleton

Characters in Unreal Engine can be animated using skeleton tracking data from Motive. There are two different approaches to this: skeleton bone mapping and skeleton retargetting.

Bone Mapping

For plugin versions above 1.9.2, the skeleton component can be used to allow streaming of the skeleton via bone mapping. This means that the motion of each skeleton bone in Motive can be projected to the bones of the character model. Please note that this is different from skeleton retargeting, the bone segments in Motive get mapped directly to the character model (bone mapping) instead of being translated into something that is usable by a more abstract biped model (retargeting). The instructions on setting up the skeleton component are documented under the following page:

Go to: Unreal Engine Skeleton Streaming

Skeleton Retargetting

Skeleton tracking data can also be integrated into Unreal Engine via a 3D retargeting process. This can be accomplished through a third party software, such as IKinema, that retargets captured 3D data onto a character to produce realistic motion in a scene. For more information, contact us.

Info2.png

Notes for IKinema LiveAction for UE 4.18.

When using IKinema LiveAction for UE 4.18 along with OptiTrack UE plugin 4.18.3, you will need to replace NatNet library file (NatNetLIb.dll / NatNetLib.lib) included in the IKinema LiveAction plugin directory with the library files included within the Optitrack UE plugin package. This is required only for this particular version of the LiveAction because it uses a slightly different version of the NatNet in its package. Copy both the LIB and DLL files for the corresponding platform, 32-bit or 64-bit, from the OptiTrack plugin and replace the library files included on the LiveAction plugin package:
  • The OptiTrack UE plugin library files exist under the following folder:
C:\Program Files\Epic Games\UE_4.18\Engine\Plugins\UE_4.18\OptiTrack\OptitrackNatNet\ThirdParty\NatNetSDK\lib
  • The LiveAction plugin library files exists under the following folder:
C:\Program Files\Epic Games\UE_4.18\Engine\Plugins\LiveAction\Source\ThirdParty\NatNet\NatNetSDK\lib\

Drawing Markers

When needed, you can also draw streamed labeled marker data from Motive into the scene in UE. In most applications, you do not have to draw the markers into the scene as rigid body data and the skeleton data will be mainly used; however, getting markers generated in the scene may be helpful for debugging purposes. To enable drawing of the markers, expand the OptiTrackClientOrigin component properties, and set the Draw Markers to true, and also, the Labeled Markers in the Motive's data streaming pane must be set to true as well.

Labeled markers shown in UE.

Intergrating HMDs


OptiTrack motion capture systems can be used to track head mounted displays (HMD) and integrate the tracking data into Unreal Engine for VR applications. For instructions on integrating HMD tracking data into Unreal Engine, please refer to the corresponding pages:

Info2.png

Supported HMDs

At the time of writing, the following HMDs are supported:

  • HTC Vive
  • HTC Vive Pro
  • Oculus DK2
  • Oculus CV1
  • Oculus GearVR

Live Streaming


Once you have connected the client object and configured the script components on the assets from the scene. Play the scene, and if the properties are correctly configured, the assets in the game scene will be animated according to the position and orientation of the rigid bodies and the HMD in Motive. When playing the scene, the HMD and the assigned actor objects will be placed in respect to their location in Motive’s coordinate system disregards to their position in the editor.

Wireless Multiplayer Setup


When setting up multiplayer games on wireless clients, it is more beneficial for each client to make direct connection to both the tracking-server (Motive) and the game-server, rather than rebroadcasting the streamed tracking data through the game-server. Then, any of the game related actions that interacts with the tracking data can be processed on the game-server, and this server can send out the corresponding updates to the wireless clients. This allows the wireless clients to only receive both the tracking data or updates without having to send back any information; in other words, minimizing the number of data transfers needed. If wireless clients are sending data there will be a minimum of two transfers on the wireless network, and each transfer of data through wireless network is at risk of latency or lost packets.

WirelessClientSetup.png