documentation
When using VisionLib's Model tracking, so-called templates are captured during the tracking process and will be temporarily saved into a local init data storage in the memory of your device. This initialization data is linked to previously visited view-points along the traveled camera path. Once the tracking is lost, VisionLib will automatically and quickly reinitialize from similar view-points without the user having to align the line model with the real object.
You can observe this behavior while tracking VisionLib's paper craft car in this quick-start tutorial. If you move the camera away from the car, the tracking state will change to lost, and if you point it back at the object from the same angle, it will re-initialize, changing the tracking state to tracked. You can only re-initialize from a pose that is close enough to one of the stored templates.
Based on the captured camera images, VisionLib constantly learns what your model looks like from various perspectives and stores these templates. The saved templates are cleared upon a Reset Hard. This command allows you to reset your application to the beginning state of the session.
You can save these templates for later use. Say, you wanted to track a model on a turn-table-like rotating plate. A single init-pose would hardly allow your user to access the AR experience, as they would need to match the given (single) init-pose with the moving/rotating object.
So what if you had recorded hundreds of (good) templates from an object? Fine! Because it helps you overcoming the typical align-and-snap problems. Especially if you expect users to have common points of view when initializing your object. By storing collected templates, you can use them to have more init-poses that allow your user to init from different perspectives.
Once you have collected enough poses of your model, you might want to save this data. To achieve this, you can call the writeInitData function along with an URI. Passing local-storage-dir:/myInitData.binz as the URI will save the data to your device. The filename ending .binz is important in this case and should not be forgotten.
Once this data has been written, it can be recalled using the readInitData function or directly on startup of the tracking pipeline passing the URI using the initDataURI parameter in the tracking configuration.
Templates are usually held in two different containers, allowing more flexibility in the user experience:
The dynamic container is used during runtime. It collects data while learning and re-initializing during tracking. It is extended dynamically and cleared on a hard reset. When the application is closed, all data inside this container is lost.
This container persistently holds stored init data that is loaded on startup. So, when you load a .binz data set, it will be held in the static container. Dynamic templates are merged with existing ones from the static container, once you decide to store them (via writeInitData). A joined container will be written then, eventually extending the existing template data set. At the same time the dynamic container will be cleared. Loading templates in return will consequently only fill up the static container.
The static container can be reset by calling the resetInitData or resetStaticInitData command, which will empty the static container at runtime.
Having two separate containers enables developers to deploy pre-learned containers with the application, while the application may continue to learn during runtime as well. In case the init data is "over learned" or wrongly tracked (resulting in misleading templates), the user can initiate a hard reset. This resets the dynamically captured init data at runtime, but leaves stored data untouched, allowing to start over the pipeline.
In order to test this behavior, import the VisionLib.SDK.Examples-Unity.unitypackage and open the ModelTrackingSetup scene from VisionLib Examples/ModelTracking/TrackingSetup. Reference your license file in the TrackingConfiguration component of the VLTracking object and press the play button. It is useful to have a calibrated camera, so you might also want to reference your calibration file in the same component.
Please start tracking by pressing the Start button and align the car model to the real one. Open the debug view (Debug On) and observe the _NumberOfTemplates parameters. You will see how dynamic template data is collected while moving the camera. When pressing the Write init data button, you will notice how data is merged from Dynamic to Static Templates. If you Stop tracking and Start it again, you can press ReadInitData to restore the saved data. This way, VisionLib will reinitialize using previously stored positions.
The init data templates do not always work perfectly. They come with some disadvantages, which the developer should keep in mind:
The advantages of collecting, storing, and using templates are:
Since the templates are dependent on the rotation and distance of camera to the object, you typically need to be pre-train a large number of templates, so that the tracking can automatically initialize from a reasonable volume of different view-points.
You can influence the frequency of the init data template creation by adjusting the keyFrameDistance parameter. For example by setting it to 50, every 5 cm a new template will be generated.
The number of dynamic and static templates can be accessed through the following variables in the TrackingState:
_NumberOfTemplates - shows the sum of all learned and used templates._NumberOfTemplatesDynamic - shows the number of dynamic templates collected during runtime._NumberOfTemplatesStatic - shows the number of static templates.