Model Parts Tracking in Unity

Level: Advanced

Introduction

VisionLib allows you to use 3D models from Unity's hierarchy for tracking (for more information on this, see › Using Models from the Unity Hierarchy). This enables you to dynamically add or remove parts of the tracking model depending on the situation. You can also switch between entirely different tracking geometries. Our SwitchingModels Unity sample scene shows you how to do this.

Tracking Mesh

In VisionLibs model tracking, an individual tracking target is represented by a TrackingAnchor. This target's tracking geometry comprises one or more meshes (MeshFilters), each located on children of the TrackingAnchor in the hierarchy. A mesh is added to tracking by adding a TrackingMesh component to the same GameObject. The settings on the TrackingMesh control their role in their parent tracking target, as explained in the following sections.

You can add the TrackingMesh automatically to all GameObjects containing a MeshFilter by pressing the Add TrackingMesh components to all models button in the TrackingAnchor.

Note: In some cases, including many of our sample scenes in Unity, the same model is used for tracking and visualization. Despite this, the tracking geometry is generally independent of the geometry rendered by Unity (i.e. what is visible in the game view). You can use arbitrary meshes for visualization purposes. For more information about displaying different models for augmentation and/or init pose guidance, please consult Using Different Augmentation and Init Pose Guide.

Enabled

If a TrackingMesh is located on the same GameObject as a MeshFilter, the model is added to VisionLib. To use the model for tracking, its TrackingMesh component has to be enabled. Toggling the enabled state of a TrackingMesh dynamically adds/removes the corresponding model to/from the current tracking geometry. E.g., you can progressively assemble your tracking geometry during runtime by starting with disabled TrackingMeshes and enabling them one by one.

Occluder

An enabled TrackingMesh is generally used as tracking geometry. Some use cases call for tracking to ignore portions of the real tracked object. This is achieved by placing an appropriate geometry such that it covers these portions and set the occluder property in the corresponding TrackingMesh component to "true" (e.g. by checking the occluder box in the GUI).

For more information on tracking occluders, please see Model Tracking Occluders and explore our Tracking Occluder sample Scene.

RootTransform

Each tracking target comprises several constituent TrackingMeshes. TrackingMeshes themselves cannot stand on their own. They must be linked to a parent tracking target, i.e. a TrackingAnchor. This TrackingAnchor is specified in the TrackingMesh's RootTransform property.

The relative transform between a TrackingMesh and its TrackingAnchor (RootTransform) in Unity defines the transform of the constituent mesh within the tracking target's coordinate system in VisionLib.

Scene Hierarchy in Tracking Anchor

The Scene Hierarchy in the Tracking Geometry section of a Tracking Anchor's editor gives a hierarchical overview of the TrackingMeshes connected to this anchor:

This scene hierarchy lists all GameObjects below the TrackingAnchor. It provides a range of settings and functions for each node. Settings made on a GameObject apply to the object itself and all its children. Mixed states under a GameObject – i.e. if child GameObjects have different values for the same setting – are indicated by a dash. The available settings and functions (from left to right in reach row in the image above) are explained in the following.

Tracking Object

The leftmost column contains the name of the GameObject. The checkbox to the left of the name shows the presence of a TrackingMesh – i.e. whether the GameObject contains tracking geometry. You can add/remove TrackingMesh components by toggling the checkbox. The rows for disabled GameObjects are greyed out entirely.

Tracking Mesh Enabled

The TrackingMesh can be enabled or disabled via this checkbox in the Scene Hierarchy. As explained above, this state decides whether the corresponding MeshFilter is used for tracking.

Mesh Renderer Enabled

In the MeshRenderer checkbox enables/disables the corresponding MeshRenderer. This shows/hides the respective mesh in Unity's scene and game views. It does not affect VisionLib's tracking. For more information about this topic see Using Different Augmentation and Init Pose Guide.

Occluder

The Occluder checkbox marks the TrackingMesh as a tracking occluder. This can only been activated, if a TrackingMesh is present on said GameObject. For more information on this setting, refer to our article on Model Tracking Occluders.

Reveal in Hierarchy

Pressing the Reveal in Hierarchy button highlights the corresponding GameObject in Unity's hierarchy.

Combine Meshes

Complex sub-model structures can be unpleasant to handle in unity. The Combine Meshes button reduces complexity by merging the Mesh on the GameObject whose button you pressed with all Meshes on its children. Children with different occluder/enabled settings are be combined into separate GameObjects, one object for each unique combination of settings.