Using Models from Unity Hierarchy for Tracking (Experimental)

Level: Advanced

Please note: this is feature is marked experimental. Usage principles might be subject to change.

Introduction

Dynamic Model Tracking enables you to switch reference models used for tracking at runtime (details, see › Mutable Model Tracking)

In order to gain even more flexibility, model injection (added in Release 19.3.1) now enables you to directly use 3D models from within Unity's hierarchy for tracking, too.

To use these features, the general process is to load a tracking configuration with no modelURI configuration and attach the VLModelTrackableBehaviour script to the (game) object-to-be-tracked in Unity.

Example Scene

The VL Unity SDK includes a sample scene that shows you how to enable and disable tracking models at runtime. Browse to VisionLib/Examples/ModelTracking/Dynamic and open the DynamicModelTracking scene. Enter Play Mode, press the Start button, and select a camera, if prompted. You should be able to see the camera image running, but without a superimposed (reference) model.

Next, enable one of the two objects via their checkbox, e.g Car. The 3D model of VL's paper-craft car will appear with a red outline (when not tracking). If you want, you can start tracking as usual, or toggle the other model, Caravan. As you do so, you see the 3D models and line models respectively loading/unloading.

By toggling these models through the UI, you are changing both, the reference model for tracking and the overlay model for rendering. To disable the overlay, deactivate the corresponding "mesh renderer" of the respective object.

Loading, managing and tracking multiple models with Dynamic Model Tracking should not be confused with Multi Model Tracking, though: it does not allow to track multiple objects independently. If you had all models active at once, they would simply add up and be treated by VisionLib as one object for tracking.

Structure

Let's have a look what's going on. When the Start button is pressed, we load the noObject.vl file, which doesn't contain a reference model modelURI definition. The file looks like this:

{
    "tracker": {
        "type": "modelTracker",
        "version": 1,
        "parameters": {
            "useColor": true,
            "metric": "m",
            "keyFrameDistance": 5,
            ...
            "showLineModel": true 
        }
    }
}

As you can see, modelURI is missing in the above file if you compare with usual tracking configurations (below):

{
    "tracker": {
        "parameters": {
            "modelURI": "project-dir:car.obj",
            "useColor": true,
            "metric": "m",
            "keyFrameDistance": 5,
            ...
            "showLineModel": true 
        }
    }
}

This is, because we are setting and loading the model from within our UI in Unity. In this example, you can find two GameObjects in the hierarchy under SceneContent: VLMiniCar and VLCaravan. Each of them has a VLModelTrackableBehaviour attached, which itself has the following options:

• Use For Tracking - If enabled, the associated meshes and sub-meshes will be streamed into the running instance of VisionLib. An object ID, Name and License features are returned.
• Occluder - If enabled, the associated meshes and sub-meshes will occlude other tracking models, but without representing tracking targets themselves. For more information, please refer to the section on occluders.
• Model Descriptions - A container of returned information for each associated mesh. It includes license features, internal ids, and names.

Toggling a model from the UI is equivalent to controlling the model's visibility via setActive and switching the useForTracking parameter of the VLModelTrackableBehaviour component.

Updating the Scene

For some applications, you may want to change the (sub-)meshes themselves during runtime. In such a case, it is useful to call UpdateModel() programmatically on the corresponding VLModelTrackableBehaviour. This updates the model that is used for tracking.

Flexibility for enhanced AR Use Cases

With Dynamic Model Tracking, we can not only control, which 3D model is used as the reference model for tracking, but also dynamically stream the models into VisionLib. The possibility to use 3D models directly from the Unity scene hierarchy eliminates the need to place models in the project folder StreamingAssets/VisionLib in advance, as opposed to the traditional way of using the VisionLib.

This is, of course, still possible. You can load models dynamically from the local project directory as well. Loading models in such a way enables you to add/remove models or parts from what VisionLib uses as model reference. Have a look at this article to understand how you can declare multiple models as tracking references inside the tracking configuration and get more background on Dynamic Model Tracking.