documentation
Since Release 18.9.1, VisionLib's Model Tracking does not only support to track one static 3D model, but also to set & change the 3D reference model used for tracking during runtime. This is what we refer to when we talk about tracking and changing (reference) models dynamically.
This enables you basically to change the reference model used for tracking, and to switch between multiple reference models to be tracked at runtime. Before, VisionLib only tracked the single referenced model, which was defined in the tracking configuration upfront. And, in case of Unity, this reference model had to be stored in the StreamingAssets folder.
Loading, managing and tracking multiple models with Mutable Model Tracking should not be confused with multi model tracking, though: it doesn't enable you to track multiple objects simultaneously. If you have several models active at once, they simply add up and are treated by VisionLib as one reference object during tracking.
With Dynamic Model Tracking and what we call Model Injection, you can now also take 3D models from Unity's hierarchy and use them for tracking (For more on this, see › Using Models from Unity Hierarchy for Tracking (Experimental)).
That said, you can still use models stored in your project directory, of course. Leveraging Mutable Model Tracking with dedicated 3D models is exactly the scope of this article. We show the general way for defining and altering the behavior of what VisionLib is likely trying to track, and how to access it via Unity using tracking configurations. Even if you are not using Unity as primary interface, this article can help you understand the general usage. We assume that you have already defined your own tracker, and that you are familiar with the general structure of a tracking configuration (.vl file) (otherwise, see › Configuration File Reference).
Such functionality is handy if your physical tracking target changes quite much. Say, you wanted to create a step-by-step manual in AR: you can add/remove parts in the reference model, as parts from the real object do during assemblies. Or, if you had different states you would want to reflect, like a door either opened or closed, changing model references accordingly (to changes in reality) improves tracking results, because you as a developer can reflect such changes.
Typically, you can use the modelURI parameter to define one 3D model as a reference model. In order to define multiple models, you can use the models parameter instead in order to switch through them. When both are declared, VisionLib will prefer models over modelURI.
Here is an example use of models:
In the case stated above, we basically load 4 different obj files. All are enabled, causing VisionLib to add them up and treat them like one reference object. By disabling some of them, they subsequently are removed from the line model.
The following parameters are allowed to be defined for each model definition:
| Parameter | Type | Default value | Runtime set-/getable | Example |
|---|---|---|---|---|
| name | string | mandatory | NO | "name": "MyModel1" |
|
A unique model identifier Can be an arbitrary string. The object can later be referenced through the | ||||
| uri | string | mandatory | YES | "uri": "http://mysuperserver/mymodel.ply" |
|
URI to the mandatory model to be tracked. The following file formats are generally supported: 3D, 3DS, AC, ASE, B3D, BLEND, BVH, COB, CSM, DAE, DXF, FBX, GLB, GLTF, HMP, IFC, IRR, IRRMESH, LWO, LWS, LXO, MD2, MD3, MD5, MDC, MDL, MDL, MS3D, NDO, NFF, NFF, OBJ, OFF, OGEX, PK3, PLY, Q3D, Q3S, RAW, SCN, SMD, STL, TER, VTA, X, XGL, XML and ZGL. Using FBX files can lead to problems, if the file uses an internal scaling factor. Depending on the model loader you are using, the resulting 3D model may be scaled by such a factor. Especially, the scaling factor has a different interpretation in VisionLib than in Unity. Usage of OBJ and PLY files is recommended, because these formats are well tested and they can be easily produced using MeshLab (http://meshlab.sourceforge.net/) and the data is stored binary (with PLY). The usage of http URIs is currently not possible on UWP, which includes HoloLens. If you require this feature, please contact us. | ||||
| enabled | boolean | optional (TRUE) | YES | "enabled": false |
|
Enabling the object for tracking Will define if the object will be used for tracking. | ||||
| occluder | boolean | optional (FALSE) | YES | "occluder": true |
|
Use this model for occluding 'un-trackable' objects Will define if the object will be used to set parts as occluding parts in the reference model. | ||||
| transform.t | array of 3 floats | optional ([0,0,0]) | YES | "transform": {"t":[3,0,0]} |
|
Translation of the object. Will define the translation of the object at runtime in model units. | ||||
| transform.s | array of 3 floats | optional ([1,1,1]) | YES | "transform": {"s":[0.44,0.44,0.44]} |
|
Scale of the object. Will define the scale of the object at runtime. | ||||
| transform.q | array of 4 floats | optional (Identity) | YES | "transform": {"q":[0,1,0,0]} |
|
Rotation of the object as quaternion. Will define the rotation of the object at runtime. | ||||
You can alter the behavior for each reference model by using the setModelProperty method. It works with 3 parameters:
name: followed by name (e.g. name:Piece01)enabledenabled it is true or falseYou're now set up to use models from a given models list as tracking reference.
Thinking of assemblies again, for instance dismantling a power-train engine, you can now separate those steps and represent removable parts in a single 3D file each (just like above), containing the appropriate state during dismantling. Summing them up, you would get the full model again.
Or, say you had optional product components, e.g. a car trailer or a roof rack: keeping the 3D models in separate files enables you to enable/disable such components regarding to the tracking target's state.
Within the example scenes that come with the Unity3D SDK, we provide an example for this feature. Take a look at the MutableModelTracking.unity scene under VisionLib/Examples/ModelTracking/Mutable/Scenes.