Level: Advanced

Please note: this is feature is marked experimental. Usage principles might be subject to change.

Introduction

Dynamic Model Tracking enables you to switch reference models used for tracking at runtime (details, see › Dynamic Model Tracking)

In order to gain even more flexibility, model injection (added in Release 19.3.1) now enables you to directly use 3D models from within Unity's hierarchy for tracking, too.

To use these features, the general process is to load an .vl file with no modelURI configuration and attach the VLModelTrackableBehaviour script to the (game) object-to-be-tracked in Unity3D.

Example Scene

The VL Unity SDK includes a sample scene that shows you how to enable and disable tracking models at runtime. Browse to VisionLib/Examples/ModelTracking/Dynamic and open the DynamicModelTracking scene. Enter Play Mode, press the Start button, and select a camera, if prompted. You should be able to see the camera image running, but without a superimposed (reference) model.

Next, enable one of the three objects from the left-handed GUI, e.g car. The 3D model of VL's paper-craft car will appear with a red outline (when not tracking). If you wanted, you could start tracking as usual, or toggle the other models, Cube or Sphere. As you do so, you see the 3D models, and line models respectively, loading/unloading.

By toggling these models through the UI, you are changing both, the reference model for tracking and the overlay model for rendering. To disable the overlay, deactivate the corresponding "mesh renderer" of the respective object.

Loading, managing and tracking multiple models with Dynamic Model Tracking should not be confused with multi model tracking, though: it does not allow to track multiple objects independently. If you had all models active at once, they would simply add up and be treated by VisionLib as one object for tracking.

Structure

Let's have a look what's going on. When the Start button is pressed, we load the noObject.vl file, which doesn't contain a reference model modelURI definition. The file looks like this:

{
    "tracker": {
        "type": "modelTracker",
        "version": 1,
        "parameters": {
            "useColor": true,
            "metric": "m",
            "keyFrameDistance": 5,
            ...
            "showLineModel": true 
        }
    }
}

As you can see, modelURI is missing in the above file if you compare with usual .vl files (below):

{
    "tracker": {
        "parameters": {
            "modelURI": "project_dir:car.obj",
            "useColor": true,
            "metric": "m",
            "keyFrameDistance": 5,
            ...
            "showLineModel": true 
        }
    }
}

This is, because we are setting and loading the model from within our UI in Unity3D. In this example, there are 3 game objects in the Hierarchy under SceneContent you can choose from: Cube,Car, and Sphere. Each of them has a VLModelTrackableBehaviour attached, which itself has the following options:

Use For Tracking - If enabled, the associated meshes and sub-meshes will be streamed into the running instance of VisionLib. An object ID, Name and License features are returned.
Occluder - If enabled, the associated meshes and sub-meshes will occlude other tracking models, but without representing tracking targets themselves. For more information, please refer to the section on occluders.
Model Descriptions - A container of returned information for each associated mesh. It includes license features, internal ids, and names.

Toggling a model from the UI is equivalent to controlling the model's visibility via setActive and switching the useForTracking parameter of the VLModelTrackableBehaviour component.

Updating the Scene

Typically, for most applications you may want to change either transforms or even the (sub-)meshes itself during runtime. In such a case, it is useful calling either UpdateTransformation() or UpdateModel() programmatically on the corresponding VLModelTrackableBehaviour. This updates the model as it is used for tracking.

Flexibility for enhanced AR Use Cases

With dynamic Model tracking, we can not only control which 3D model is used as the reference model for tracking, but also dynamically stream the models into the VisionLib. The possibility to use 3D models directly from the Unity3D scene hierarchy eliminates the need to place models in advance in the project folder StreamingAssets/VisionLib, as opposed to the traditional way of using the VisionLib.

This is, of course, still possible. You can also load models dynamically from the local project directory as well. Loading models in such a way enables you to add/remove models or parts from what VisionLib uses as model reference. Have a look at this article to understand how you can declare multiple models as tracking references inside the .vl file and get more background on Dynamic Model Tracking.

Special considerations for HoloLens