documentation
Since Release 18.9.1, VisionLib's Model Tracking does not only support to track one static 3D model, but also to set & change the 3D reference model used for tracking during runtime. This is what we refer to when we talk about tracking and changing (reference) models dynamically.
This enables you basically to change the reference model used for tracking, and to switch between multiple reference models to be tracked at runtime. Before, VisionLib only tracked the single referenced model, which was defined in the tracking configuration upfront. And, in case of Unity, this reference model had to be stored in the StreamingAssets folder.
Loading, managing and tracking multiple models parts should not be confused with multi model tracking, though: it doesn't enable you to track multiple objects simultaneously. If you have several models active at once, they simply add up and are treated by VisionLib as one reference object during tracking.
With Dynamic Model Tracking and what we call Model Injection, you can now also take 3D models from Unity's hierarchy and use them for tracking (For more on this, see › Using Models from Unity Hierarchy for Tracking (Beta)).
That said, you can still use models stored in your project directory, of course. Leveraging Model Parts Tracking with dedicated 3D models is exactly the scope of this article. We show the general way for defining and altering the behavior of what VisionLib is likely trying to track, and how to access it via Unity using tracking configurations. Even if you are not using Unity as primary interface, this article can help you understand the general usage. We assume that you have already defined your own tracker, and that you are familiar with the general structure of a tracking configuration (.vl file) (otherwise, see › Configuration File Reference).
Such functionality is handy if your physical tracking target changes quite much. Say, you wanted to create a step-by-step manual in AR: you can add/remove parts in the reference model, as parts from the real object do during assemblies. Or, if you had different states you would want to reflect, like a door either opened or closed, changing model references accordingly (to changes in reality) improves tracking results, because you as a developer can reflect such changes.
As you probably already know, you can use the modelURI parameter to define one 3D model as a reference model. In order to define multiple models, you can use the models parameter instead in order to switch through them. When both are declared, VisionLib will add both the models and the modelURI.
Here is an example use of models:
In the case stated above, we basically load 4 different obj files. All are enabled, causing VisionLib to add them up and treat them like one reference object. By disabling some of them, they subsequently are removed from the line model.
Some of the models properties can be changed during runtime, using the setModelProperties command. It receives a model definition as its parameter, that can be only partially filled and will update the fields that are present.
Instead of changing a single Model, you can also send a setModelPropertiesForAll command, to update the properties of all models at once.
If you want to set the options of several models simultanuously but different for every model, you can use the setMultipleModelProperties command. It receives an array of (partial) model definitions:
The following parameters can be used inside the model definition for addModel, setModelProperties, setModelPropertiesForAll and setMultipleModelProperties:
| Parameter | Type | Default value | Runtime set-/getable | Example in model definition |
|---|---|---|---|---|
| name | string | optional (null) | NO | "name": "MyModel1" |
|
A unique model identifier Can be an arbitrary string, which is used to reference the model later on. If it is not provided, the name will be set to "Model_[n]", with incrementing numbers [n]. | ||||
| uri | string | optional ("") | YES | "uri": "http://mysuperserver/mymodel.ply" |
|
URI to the cad file of the model to be tracked. The following file formats are generally supported: 3D, 3DS, AC, ASE, B3D, BLEND, BVH, COB, CSM, DAE, DXF, FBX, GLB, GLTF, HMP, IFC, IRR, IRRMESH, LWO, LWS, LXO, MD2, MD3, MD5, MDC, MDL, MDL, MS3D, NDO, NFF, NFF, OBJ, OFF, OGEX, PK3, PLY, Q3D, Q3S, RAW, SCN, SMD, STL, TER, VTA, X, XGL, XML and ZGL. Using FBX files can lead to problems, if the file uses an internal scaling factor. Depending on the model loader you are using, the resulting 3D model may be scaled by such a factor. Especially, the scaling factor has a different interpretation in VisionLib than in Unity. Usage of OBJ and PLY files is recommended, because these formats are well tested and they can be easily produced using MeshLab (http://meshlab.sourceforge.net/) and the data is stored binary (with PLY). The usage of http URIs is currently not possible on UWP, which includes HoloLens. If you require this feature, please contact us. | ||||
| enabled | boolean | optional (true) | YES | "enabled": false |
|
Enabling the object for tracking Will define if the object will be used for tracking. | ||||
| occluder | boolean | optional (false) | YES | "occluder": true |
|
Use this model for occluding 'un-trackable' objects Will define if the object will be used to set parts as occluding parts in the reference model. | ||||
| transform.t | array of 3 floats | optional ([0,0,0]) | YES | "transform": {"t":[3,0,0]} |
|
Translation of the object. Will define the translation of the object at runtime in model units. | ||||
| transform.s | array of 3 floats | optional ([1,1,1]) | YES | "transform": {"s":[0.44,0.44,0.44]} |
|
Scale of the object. Will define the scale of the object at runtime. | ||||
| transform.q | array of 4 floats | optional ([0,0,0,1]) | YES | "transform": {"r":[0,1,0,0]} |
|
Rotation of the object as quaternion. Will define the rotation of the object at runtime. | ||||
Models can be added at runtime by sending a addModel command. This command receives a model definition as its parameter.
The command will return {"addedModel": "Piece000"} where in this example Piece000 is the name of the added model. Note: this is especially useful if the model name was not supplied in the addModel command.
If the name of the model was already used, the following message will be returned in the error parameter of the command callback:
Models can be removed at runtime by sending a removeModel command. This command receives a modelName as its parameter.
This command will return nothing on success.
Instead of removing a single Model, you can also send a removeAllModels command, to remove all models at once. This command has no parameters.
You're now set up to use models from a given models list as tracking reference.
Thinking of assemblies again, for instance dismantling a power-train engine, you can now separate those steps and represent removable parts in a single 3D file each (just like above), containing the appropriate state during dismantling. Summing them up, you would get the full model again.
Or, say you had optional product components, e.g. a car trailer or a roof rack: keeping the 3D models in separate files enables you to enable/disable such components regarding to the tracking target's state.
Within the VisionLib.SDK.Examples-Unity.unitypackage, we provide an example for this feature. After importing the package, have a look at the ModelPartsTracking.unity scene under VisionLib Examples/ModelTracking/ModelLoading/ModelPartsTracking/Scenes.