Difference between revisions of "Template:LabelingSamples"

(Sample Scenarios)
 
(21 intermediate revisions by 2 users not shown)
Line 1: Line 1:
The following steps describe typical labeling workflows. Note that the steps may vary depending on the capture, and alternative approaches could also be used. Utilize the reconstruction and auto-labeling pipelines in combination with the [[Labeling pane]] to best reconstruct and label the 3D data of your capture. From Motive 1.10 and above, standalone auto-labeling pipeline has been added, and it can be utilized to more efficiently and precisely label the markers.
+
==Sample Scenario 1 :: All Good==
 
+
{{#ev:vimeo|167944412|500x280|right|Labeling Tutorial 1. How to check marker labels in Motive|frame}}
 
+
{{Tip|
+
'''Labeling Tips'''
+
----
+
* Use the [[Timeline pane#Timeline Editor|Timeline Editor]] to monitor occlusion gaps and labeling errors as you post-process capture ''Takes''
+
* When using the [[Labeling pane]], choose the most appropriate labeling setting (all, selected, spike, or fragment) to efficiently label selected trajectories. See more from the [[Labeling pane]] page.
+
* When working with skeleton assets, label the hip segment first. The hip segment is the main parent segment, top of the segments hierarchy, where all other child segments are associated to. Manually assigning hip markers sometimes helps the auto-labeler to label entire assets.
+
* Show/Hide skeleton visibility [[Image:View_ShowHideSKL.png]] in the [[View pane#Perspective View|perspective view]] to have better view on the markers when assigning marker labels.
+
* Toggle skeleton selectability [[Image:Viewport14.png]] in the [[View pane#Perspective View|perspective view]] to use the skeleton as a visual aid without it getting in the way of marker data.
+
* Show/Hide skeleton sticks and marker colors under the visual aids [[Image:Viewport16.png]] in the [[View pane#Perspective View|perspective view]] options for intuitive identification of labeled markers as you tag through skeleton markers.
+
}}
+
 
+
<table cellspacing="0" cellpadding="0" style="margin:auto;  width:100%">
+
<tr>
+
<td style = "vertical-align: top; width: 50%; padding-left: 2em; padding-right: 2em;">
+
===Combined Reconstruction and Auto-label===
+
----
+
{{Indent|
+
'''Step 1.'''  In the [[Project pane]], '''Reconstruct and auto-label''' the take with all of the desired assets enabled.
+
 
+
'''Step 2.'''  In the [[Timeline pane|Timeline]], examine the trajectories and navigate to the frame where labeling errors are frequent.
+
 
+
'''Step 3.'''  Open the [[Labeling pane]].
+
 
+
'''Step 4.'''  Select an asset that you wish to label.
+
 
+
'''Step 5.'''  From the label columns, Click on a marker label that you wish to re-assign.
+
 
+
'''Step 6.'''  Inspect behavior of a selected trajectory and its labeling errors and set the appropriate labeling settings (allowable gap size, maximum spike and applied frame ranges).
+
 
+
'''Step 7.'''  Switch to the QuickLabel mode (Hotkey: D).
+
 
+
'''Step 8.'''  On the [[View pane|Perspective View]], assign the labels onto the corresponding marker reconstructions by clicking on them.
+
 
+
'''Step 9.'''  When all markers have been labeled, switch back to the Select Mode.
+
}}
+
</td>
+
<td style = "vertical-align: top;width: 50%; padding-left: 2em; padding-right: 2em;">
+
===Standalone Reconstruction and Auto-label===
+
----
+
{{Indent|
+
'''Step 1.'''  Start with 2D data of a captured ''Take'' with model assets (skeletons and rigid bodies).
+
 
+
'''Step 2.'''  '''Reconstruct and Auto-Label''', or just '''Reconstruct''',  the ''Take'' with all of the desired assets enabled under the [[Project pane]]. If you use reconstruct only, you can skip step 3 and 5 for the first iteration.
+
 
+
'''Step 3.'''  Examine the reconstructed 3D data, and inspect the frame range where markers are mislabeled.
+
 
+
'''Step 4.'''  Using the [[Labeling pane]], manually fix/assign marker labels, paying attention to your label settings (direction, max gap, max spike, selected duration).
+
 
+
'''Step 5.'''  Unlabel all trajectories you want to re-auto-label.
+
 
+
'''Step 6.'''  '''Auto-Label''' the ''Take'' again. Only the unlabeled markers will get re-labeled, and all existing labels will be kept the same.
+
 
+
'''Step 7.'''  Re-examine the marker labels. If some of the labels are still not assigned correctly from any of the frames, repeat the steps 3-6 until complete.
+
}}
+
</td>
+
</tr>
+
</table>
+
 
+
 
+
{{Indent|
+
===Labeling Error Fix===
+
----
+
[[File:Label.gif|thumb|280 px|Labeling an unlabeled marker.]]
+
{{Indent|
+
The general process for resolving labeling error is:
+
# Identify the trajectory with the labeling error.
+
# Determine if the error is a swap, an occlusion, or unlabeled.
+
# Resolve the error with the correct tool.
+
{{Indent|
+
* Swap : Use the Swap Fix tool ( Edit Tools ) or just re-assign each label ( Labeling Panel ).
+
** When manually labeling markers to fix swaps, utilize the direction, max spike and selected range settings as appropriate.
+
* Occlusion : Use the Gap Fill tool ( Edit Tools ).
+
* Unlabeled : Manually label an unlabeled trajectory with the correct label ( Labeling Panel ).}}
+
 
+
For more data editing options, read through the [[Data Editing]] page.     
+
}}}}
+
 
+
 
+
=Sample Scenarios=
+
----
+
 
+
 
+
====Sample Scenario 1 :: All Good====
+
[[Image:Labeling_Scenario1|thumb|All good (need to pick)]]
+
 
{{Indent|
 
{{Indent|
When all markers are well reconstructing and there are no significant occlusions, you can just simply Reconstruct and Auto-label the Take to obtain 3D data and label all of the skeleton and rigid body markers. If reconstructed and auto-labeled 3D data is acceptable, you can proceed without post-processing.
+
When recorded 3D data have been labeled properly and entirely throughout the ''Take'', you will not need to edit marker labels. If you don't have 3D data recorded, you can reconstruct and auto-label the ''Take'' to obtain 3D data and label all of the skeleton and rigid body markers. If all of the markers are well reconstructed and there are no significant occlusions, auto-labeled 3D data may be acceptable right away. In this case, you can proceed without post-processing of marker labels.
  
# '''Reconstruct and Auto-label''' works perfectly the first time without additional post-processing.
+
# Recorded 3D data has no gaps in the labels, or the '''Reconstruct and Auto-label''' works perfectly the first time without additional post-processing.
# Examine the Take(s). Check the [[Labeling pane]], or the [[Timeline pane#Tracks View|tracks view]], to make sure no occlusion exists within the capture, and all markers are consistently labeled.
+
# Examine the Take(s). Check the [[Labeling pane]], or the [[Graph View pane#Tracks View|tracks view]], to make sure no occlusion exists within the capture, and all markers are consistently labeled.
 
# Done.
 
# Done.
 
}}
 
}}
  
 
+
==Sample Scenario 2 :: Labeling errors in the middle of a ''Take''==
 
+
====Sample Scenario 2 :: Labeling errors in the middle of a ''Take''====
+
[[Image:Labeling_Scenario2|thumb|Jumping Jack]]
+
 
{{Indent|
 
{{Indent|
When skeleton markers are mislabeled only within a specific frame range of the Take, you will have to manually re-label the markers during that range. This may occur when a subject performs dynamic movements or come into contact with another object during the recorded Take. After correcting the mislabeled markers, you can auto-label the take again to assign remaining missing labels.
+
When skeleton markers are mislabeled only within specific frame ranges of a ''Take'', you will have to '''manually''' re-label the markers. This may occur when a subject performs dynamic movements or come into contact with another object during the recorded Take. After correcting the mislabeled markers, you can also use the auto-labeler to assign remaining missing labels.
# '''Reconstruct and auto-label''' the Take.
+
# Start with recorded 3D data or '''Reconstruct and auto-label''' the Take to obtain newly labeled 3D data.
 
# Inspect the ''Take'' to pick out the frame ranges with bad tracking.
 
# Inspect the ''Take'' to pick out the frame ranges with bad tracking.
# If necessary, unlabel the bad frame range or the entire ''Take''.
+
# If markers are mislabeled during majority of the capture, unlabel all markers from the entire capture by right-clicking on the ''Take'' in [[Data Management pane]] and click '''Delete Marker Labels'''. You can do this on selected frame ranges as well.
 
# Scrub the timeline to a frame just before the bad tracking frame range.  
 
# Scrub the timeline to a frame just before the bad tracking frame range.  
# Using the [[Labeling pane]], manually label the skeleton. Depending on severity of the mislabels, you can either label the entire skeleton or just the key segments starting from the hip.
+
# Using the [[Labeling pane]], manually label the skeleton. Depending on the severity of the mislabels, you can either label the entire skeleton or just the key segments starting from the hip.
 
# Scrub the timeline to a frame after the bad tracking frame range.
 
# Scrub the timeline to a frame after the bad tracking frame range.
 
# Manually label the same skeleton.
 
# Manually label the same skeleton.
# '''Auto-label''' the Take (Motive 1.10 and above).
+
# '''Auto-label''' the Take.
 
# Check the frames again and correct any remaining mislabels using the [[Labeling pane]].
 
# Check the frames again and correct any remaining mislabels using the [[Labeling pane]].
 
}}
 
}}
  
 
+
==Sample Scenario 3 :: Skeletons never acquire perfectly throughout a ''Take''==
====Sample Scenario 3 :: Skeletons never acquire perfectly throughout a ''Take''====
+
{{#ev:vimeo|167944844|500x280|right|Labeling Tutorial 3. Auto-label.|frame}}
[[Image:Labeling_Scenario3|thumb|Undecided]]
+
 
{{Indent|
 
{{Indent|
For Take(s) where skeletons are never perfectly tracked and the markers are consistently mislabeled, you will need to manually assign the correct labels for the skeleton asset(s). Situations like this could happen when the skeleton(s) are never in an easily trackable pose throughout the Take (e.g. captures where the actors are rolling on the ground).  It is usually recommended that all skeleton ‘’Takes’’’ start and end with T-pose in order to easily distinguish the skeleton markers. This also helps the skeleton solver to correctly auto-label the associated markers; however, in some cases, only specific section of a Take needs be trimmed out, or including the calibration poses might not be possible. Manually assigning labels can help the auto-labeler to correctly label markers and have skeletons acquire properly in a Take. You will get best results if you manually label the entire skeleton, but doing so can be time consuming. You can also label only the mislabeled segment or the key segment (hip bone) and run the auto-labeler to see if it correctly assigns the labels with the small help.
+
For Take(s) where skeletons are never perfectly tracked and the markers are consistently mislabeled, you will need to manually assign the correct labels for the skeleton asset(s). Situations like this could happen when the skeleton(s) are never in an easily trackable pose throughout the Take (e.g. captures where the actors are rolling on the ground).  It is usually recommended that all skeleton ‘’Takes’’’ start and end with T-pose in order to easily distinguish the skeleton markers. This also helps the skeleton solver to correctly auto-label the associated markers; however, in some cases, only specific section of a Take needs be trimmed out, or including the calibration poses might not be possible. Manually assigning labels can help the auto-labeler to correctly label markers and have skeletons acquire properly in a Take. You will get best results if you manually label the entire skeleton, but doing so can be time-consuming. You can also label only the mislabeled segment or the key segment (hip bone) and run the auto-labeler to see if it correctly assigns the labels with the small help.
  
# '''Reconstruct''' the Take.
+
# Start with recorded 3D data or '''Reconstruct''' the Take.
# At a certain point of the Take (usually at a frame where you can best identify the pose of the skeleton), use the [[Labeling pane]] to manually assign the marker labels for skeletons that are not labeling correctly. Depending on severity of the mislabels, you can either label the entire skeleton or only the key segments starting from the hip.
+
# At a certain point of the Take (usually at a frame where you can best identify the pose of the skeleton), use the [[Labeling pane]] to manually assign the marker labels for skeletons that are not labeling correctly. Depending on the severity of the mislabels, you can either label the entire skeleton or only the key segments starting from the hip.
# After manually assigning the labels, '''auto-label''' the Take. Make sure the corresponding assets are enabled in the [[Project pane]].
+
# After manually assigning the labels, '''auto-label''' the Take. Make sure the corresponding assets are enabled in the [[Assets pane]].
 
# Check to see if all markers are correctly assigned throughout the take. If not, re-label or unlabel, any mislabeled markers and run auto-label again if needed.
 
# Check to see if all markers are correctly assigned throughout the take. If not, re-label or unlabel, any mislabeled markers and run auto-label again if needed.
 
}}
 
}}
  
 
+
==Sample Scenario 4 :: Mislabeling/unlabeling of a marker due to occlusions==
====Sample Scenario 4 :: Mislabeling/unlabeling of a marker due to occlusions====
+
 
{{Indent|
 
{{Indent|
 
Marker occlusions can be critical to the auto-labeling process. After having a gap for multiple frames, occluded markers can be unlabeled entirely, or nearby reconstructions can be mistakenly recognized as the occluded marker and result in labeling swaps or mislabels. Skeleton and rigid body asset definitions may accommodate labeling for such occlusions, but in some cases, labeling errors may persist throughout the Take. The following steps can be used to re-assign the labels in this case.
 
Marker occlusions can be critical to the auto-labeling process. After having a gap for multiple frames, occluded markers can be unlabeled entirely, or nearby reconstructions can be mistakenly recognized as the occluded marker and result in labeling swaps or mislabels. Skeleton and rigid body asset definitions may accommodate labeling for such occlusions, but in some cases, labeling errors may persist throughout the Take. The following steps can be used to re-assign the labels in this case.
Line 130: Line 40:
 
''If tracked markers are relatively stationary during the occluded frames, you may want to increase the Maximum Marker Label Gap value under the Auto-Labeler settings in the [[Reconstruction pane]] to allow the occluded marker to maintain its label after auto-labeling the ''Take''. However, note that adjusting this setting will not be useful if the marker is moving dynamically beyond the Prediction Radius (mm) settings during occlusion.''
 
''If tracked markers are relatively stationary during the occluded frames, you may want to increase the Maximum Marker Label Gap value under the Auto-Labeler settings in the [[Reconstruction pane]] to allow the occluded marker to maintain its label after auto-labeling the ''Take''. However, note that adjusting this setting will not be useful if the marker is moving dynamically beyond the Prediction Radius (mm) settings during occlusion.''
  
# '''Reconstruct and auto-label''' the ''Take''
+
# Start with recorded 3D data or '''Reconstruct and auto-label''' the ''Take''
 
# Examine through the ''Take'', and go to a frame where markers are mislabeled right after an occlusion.
 
# Examine through the ''Take'', and go to a frame where markers are mislabeled right after an occlusion.
# In the [[Labeling pane]], disable the Increment Label Selection [[Image:Label_Increment.png]] option, and select a marker set and a label that is frequently occluded.  
+
# In the [[Labeling pane]], disable the Increment Label Selection [[Image:Label Increment 20.png]] option, and select a marker set and a label that is frequently occluded.  
 
# In the [[Labeling pane]], disable the ''Apply Labels to Previous Frames'' [[Image:Label_ApplyBefore.png]] option, and leave only the ''Apply Labels to Upcoming Frames'' [[Image:Label_ApplyAfter.png]] option enabled.
 
# In the [[Labeling pane]], disable the ''Apply Labels to Previous Frames'' [[Image:Label_ApplyBefore.png]] option, and leave only the ''Apply Labels to Upcoming Frames'' [[Image:Label_ApplyAfter.png]] option enabled.
 
# Using the Quick Label Mode, correct the labeling errors.
 
# Using the Quick Label Mode, correct the labeling errors.

Latest revision as of 23:53, 25 July 2018

Sample Scenario 1 :: All Good[edit]

Labeling Tutorial 1. How to check marker labels in Motive

When recorded 3D data have been labeled properly and entirely throughout the Take, you will not need to edit marker labels. If you don't have 3D data recorded, you can reconstruct and auto-label the Take to obtain 3D data and label all of the skeleton and rigid body markers. If all of the markers are well reconstructed and there are no significant occlusions, auto-labeled 3D data may be acceptable right away. In this case, you can proceed without post-processing of marker labels.

  1. Recorded 3D data has no gaps in the labels, or the Reconstruct and Auto-label works perfectly the first time without additional post-processing.
  2. Examine the Take(s). Check the Labeling pane, or the tracks view, to make sure no occlusion exists within the capture, and all markers are consistently labeled.
  3. Done.

Sample Scenario 2 :: Labeling errors in the middle of a Take[edit]

When skeleton markers are mislabeled only within specific frame ranges of a Take, you will have to manually re-label the markers. This may occur when a subject performs dynamic movements or come into contact with another object during the recorded Take. After correcting the mislabeled markers, you can also use the auto-labeler to assign remaining missing labels.

  1. Start with recorded 3D data or Reconstruct and auto-label the Take to obtain newly labeled 3D data.
  2. Inspect the Take to pick out the frame ranges with bad tracking.
  3. If markers are mislabeled during majority of the capture, unlabel all markers from the entire capture by right-clicking on the Take in Data Management pane and click Delete Marker Labels. You can do this on selected frame ranges as well.
  4. Scrub the timeline to a frame just before the bad tracking frame range.
  5. Using the Labeling pane, manually label the skeleton. Depending on the severity of the mislabels, you can either label the entire skeleton or just the key segments starting from the hip.
  6. Scrub the timeline to a frame after the bad tracking frame range.
  7. Manually label the same skeleton.
  8. Auto-label the Take.
  9. Check the frames again and correct any remaining mislabels using the Labeling pane.

Sample Scenario 3 :: Skeletons never acquire perfectly throughout a Take[edit]

Labeling Tutorial 3. Auto-label.

For Take(s) where skeletons are never perfectly tracked and the markers are consistently mislabeled, you will need to manually assign the correct labels for the skeleton asset(s). Situations like this could happen when the skeleton(s) are never in an easily trackable pose throughout the Take (e.g. captures where the actors are rolling on the ground). It is usually recommended that all skeleton ‘’Takes’’’ start and end with T-pose in order to easily distinguish the skeleton markers. This also helps the skeleton solver to correctly auto-label the associated markers; however, in some cases, only specific section of a Take needs be trimmed out, or including the calibration poses might not be possible. Manually assigning labels can help the auto-labeler to correctly label markers and have skeletons acquire properly in a Take. You will get best results if you manually label the entire skeleton, but doing so can be time-consuming. You can also label only the mislabeled segment or the key segment (hip bone) and run the auto-labeler to see if it correctly assigns the labels with the small help.

  1. Start with recorded 3D data or Reconstruct the Take.
  2. At a certain point of the Take (usually at a frame where you can best identify the pose of the skeleton), use the Labeling pane to manually assign the marker labels for skeletons that are not labeling correctly. Depending on the severity of the mislabels, you can either label the entire skeleton or only the key segments starting from the hip.
  3. After manually assigning the labels, auto-label the Take. Make sure the corresponding assets are enabled in the Assets pane.
  4. Check to see if all markers are correctly assigned throughout the take. If not, re-label or unlabel, any mislabeled markers and run auto-label again if needed.

Sample Scenario 4 :: Mislabeling/unlabeling of a marker due to occlusions[edit]

Marker occlusions can be critical to the auto-labeling process. After having a gap for multiple frames, occluded markers can be unlabeled entirely, or nearby reconstructions can be mistakenly recognized as the occluded marker and result in labeling swaps or mislabels. Skeleton and rigid body asset definitions may accommodate labeling for such occlusions, but in some cases, labeling errors may persist throughout the Take. The following steps can be used to re-assign the labels in this case.

If tracked markers are relatively stationary during the occluded frames, you may want to increase the Maximum Marker Label Gap value under the Auto-Labeler settings in the Reconstruction pane to allow the occluded marker to maintain its label after auto-labeling the Take. However, note that adjusting this setting will not be useful if the marker is moving dynamically beyond the Prediction Radius (mm) settings during occlusion.

  1. Start with recorded 3D data or Reconstruct and auto-label the Take
  2. Examine through the Take, and go to a frame where markers are mislabeled right after an occlusion.
  3. In the Labeling pane, disable the Increment Label Selection Label Increment 20.png option, and select a marker set and a label that is frequently occluded.
  4. In the Labeling pane, disable the Apply Labels to Previous Frames Label ApplyBefore.png option, and leave only the Apply Labels to Upcoming Frames Label ApplyAfter.png option enabled.
  5. Using the Quick Label Mode, correct the labeling errors.
  6. Move onto next occluded frames. When the marker reappears, correct the labels.
  7. After correcting the labels, Auto-label the Take again.
  8. Use the Fill Gaps tool in the Editing tools to interpolate the occluded trajectories.