Release Notes 3.0

Version 3.0.0

Version Info

Release date: 18.04.2023
Release info: Major Release

Summary

This is a major update for VisionLib which adds:

• Significantly simplified tracking setups in Unity.
  • The reworked TrackingAnchor is now universal to all features.
  • Model Injection from Unity's hierarchy, instead of loading models via configuration files.
• Compatibility between Multi-Model Tracking and Model Injection on all platforms and devices.
• Automated scene Validation to help identify and avoid mistakes in the tracking setup.
• Support for VisionLib on Magic Leap 2 (see VisionLib Tracking on Magic Leap 2 for more details).
• Runtime optimizations for a better tracking experience.

This release includes major interface changes. To update existing scenes, please consult the Migrating From 2.x to 3.0.

Unity versions below 2020.3 LTS are no longer supported as of this release.

Thanks to all our users for your valuable feedback!

Features

Unity

• Added new TrackingAnchor component. This component is now the default method to add models for tracking. TrackingAnchors support all features, including MultiModelTracking and AutoInit. For details see the corresponding documentation page.
• Added new GameObjectPoseInteraction component for manipulation of GameObject poses via mouse/touch inputs during runtime.
• Added new InitPoseInteraction component. This component is added to a GameObject with a TrackingAnchor. It interprets interactions via the GameObjectPoseInteraction as changes to the init pose of the tracking model represented by the TrackingAnchor.
• It is now possible to combine MultiModelTracking with all features and platforms, including ARFoundation or ImageInjection.
• Added texture streaming to Dynamic Model Tracking. Textures are automatically streamed if the respective model has a UV and if the material assigned to the model in the hierarchy has a texture. Set textureColorSensitivity > 0 to use the texture for tracking. Use the ModelTrackingSetup example scene to find a suitable textureColorSensitivity value.
• Added button in the TrackingAnchor to copy the license feature of all its tracking models.
• Added possibility to set debug level in Model Tracker and Poster Tracker at runtime.
• Added possibility to write tracking configuration of a Multi-Model Tracker.
• Added Example Scenes:
  • Tracking Occluder: Demonstrates usage & configuration of tracking occluders (see Model Tracking Occluders)
  • Different Augmentation: Demonstrates usage of different models for tracking geometry, init pose guide and augmentation (see Using Different Augmentation and Init Pose Guide)

Magic Leap 2

• Added support for deploying VisionLib.SDK to Magic Leap 2 devices. See VisionLib Tracking on Magic Leap 2.

Android

• Added support for Android x64_86 architecture (Magic Leap 2). This requires a separate license feature.

Enhancements / Bugfixes

General

• Internal optimizations reduced the overall run time of the tracking algorithm.
• Added enabled property to the anchor definition, which is supplied via the tracking configuration file or the createAnchor command. This is set to true by default. You can set it to false to create an anchor in a disabled state -disabled anchors don't count toward the limit of anchors.
• Added perCorrespondency property to the showLineModel parameter, which allows color-coded visualization of the tracking state of every single correspondence used.

Unity

• Simplified tracking setup.
  • It is no longer necessary to adjust the tracking configuration file to change the tracked model. Instead, you simply replace the mesh underneath the TrackingAnchor in your scene's hierarchy.
  • Added several helper functions and buttons to automate common set-up steps.

HoloLens

• Fixed a bug that prevented the camera from being released when tracking is paused.
• Unified the transform adjustment so all platforms use the same coordinate systems. This reduces the places where specific HoloLens code has to be used.

iOS

• Adjusted the SLAM coordinate system up direction (Y-axis) on Android & iOS. Rotated the SLAM coordinate system 180° around the X-axis to align the SLAM up direction with Unity's world up direction.
• Changed file naming for stored world map

old	new
myInitData.binz.arkitwd	myInitData_worldMap.arkitwd
myInitData.binz.arkitwd.simPose	myInitData_worldMapFromModelTransform.xml

Android

• Adjusted the SLAM coordinate system up direction (Y-axis) on Android & iOS. Rotated the SLAM coordinate system 180° around the X-axis to align the SLAM up direction with Unity's world up direction.
• VisionLib SDK for android is now compatible with gradle minification.
  • This happens for example when Minify > Release is enabled in Unity player settings.
  • Previously, enabling this option led to crashes with the message JNI DETECTED ERROR IN APPLICATION: JNI RegisterNatives called with pending exception java.lang.ClassNotFoundException: org.instantreality.mini.Platform.

Interface Changes

General

• Renamed the feature DynamicModelTracking to ModelInjection.
• The model tracker parameter allowedNumberOfFramesSLAMPredictionObjectVisible now has the default value of 180.
  When tracking is lost and extendibleTracking is activated, VisionLib predicts the pose of the object in the world via the SLAM transform. This keeps tracking "alive" – i.e. prevents the transition from status "critical" to "lost" while the SLAM-predicted object location is inside the camera's field of view. The allowedNumberOfFramesSLAMPredictionObjectVisible limits the number of frames for which purely SLAM pose predictions may keep tracking alive. To deactivate this feature – i.e. to keep tracking alive indefinitely based on SLAM only – set allowedNumberOfFramesSLAMPredictionObjectVisible to -1 in the corresponding tracking configuration file.
• allowedNumberOfFramesSLAMPredictionObjectVisible was set to 45 in all HoloLens example scenes.
• Changed the default value of the tracker parameter poseInverseConditionNumberMin from 0.007 to -1. This change reduces the number of falsely rejected good pose estimations. It does, however, increase the likelihood of acceptance for ambiguous pose estimations. In some cases, this can cause tracking glitches such as the model drifting along parallel edges. To restore the old behavior – e.g. if you encounter said glitches after the update – set the value to 0.007 manually.

Removed

• Removed support for parameter metaioLineModelURI.
  • Please use lineModelURI instead.
  • Metaio line models can be rotated 180 degrees around the y-axis to create VisionLib line models.
• Removed support for init poses of type metaio, including the XML-based format. Please use type visionlib instead.
  • To use a metaio init pose as a VisionLib init pose, you have to apply a 180° rotation around the y-axis from the right and a 180° rotation around the x-axis from the left.
• Removed support for deprecated scheme names with underscores (like project_dir, local_storage_dir, etc.). Update the schemes in your projects by replacing underscores with hyphens.
• Removed setter for internal parameters of IntrinsicData.
• Removed vlWorker_GetImageByNameSync without replacement.
• Removed support for parameter usePoseFiltering.
  • Please use poseFilteringSmoothness instead. This parameter defines the smoothness of the pose filter. Setting the value greater than 0 turns on the filter.
• Removed support for parameters:
  • showLineModelTracked and showLineModelTrackedColor.
  • showLineModelCritical and showLineModelCriticalColor.
  • showLineModelLost and showLineModelLostColor.
  • showLineModelTrackedInvalid.
  • For more information please see the Migrating From 2.x to 3.0.
• Removed support for setting laplaceThreshold and normalThreshold separately for the AutoInit configuration. The values are now always extracted from the general configuration of the tracker.

Unity

• VLCamera.prefab now uses Multi Model Tracking as default Coordinate System Adjustment in the TrackingCamera. If you are using a ModelTracker instead of a MultiModelTracker change this to Single Model Tracking.
• The PlaneConstrainedMode component must now be placed on the same GameObject as a TrackingAnchor.
  • PlaneConstrainedMode can now only be used with Multi Model Tracking setups.
• Renamed TrackingState.TrackingObject to TrackingState.Anchor to better represent it as the counterpart to the TrackingAnchor.

Removed

• SynchronousTrackingManager no longer works with trackers of the type ModelTracker. Please adapt your existing tracking configurations to use a tracker of the type MultiModelTracker instead. The examples that use the SynchronousTrackingManager (i.e. ImageInjection, ARFoundation and MagicLeap Example) have been adapted accordingly.
• The WorkSpaceManager is no longer responsible for displaying the AutoInit learning progress. Add the new AutoInitProgressBar component instead.
• The WorkSpaceManager was removed. You no longer need a WorkSpaceManager with a TrackingAnchor.
• Removed obsolete field id of ModelDeserializationResult
• Removed obsolete field updateCount of BinaryDataDescriptor
• Removed obsolete field type of ModelDataDescriptor
• Removed obsolete function ModelTransform.MultiplyLeft. Please use the * operator instead.
• Added the anchorExists command which checks whether an anchor with the given name exists.
• Removed ModelPartsTracking scene from the examples.
• Removed support for ARFoundation 2. You no longer have to set ENABLE_ARFOUNDATION_4 - this is now set by default.
• Removed all fields except the availableCameras from Visometry.VisionLib.SDK.Core.API.DeviceInfo.

Native

• Adjusted the constructor of IntrinsicData. All mandatory variables are now required immediately.
• Added new function to get the model hash of a serialized model without starting a tracker.
• Adjusted the coordinate system transformation for WindowsMR on HoloLens. This is only relevant if you are using HoloLens without our Unity SDK. In this case, you have to flip the Y and Z axis of all model poses you are setting in VisionLib.
• Removed all fields except the availableCameras from the result of vlWorker_GetDeviceInfo().
• Changed file naming of stored line models

old	new
myInitData.binz.lineModels.binz	myInitData_lineModels.binz

• Changed file naming of init data for multi-model tracker case

old	new
myInitData.binz_anchorName	myInitData_anchorName.binz
myInitData.binz_anchorName.lineModels.binz	myInitData_anchorName_lineModels.binz

• Changed the way to store and read uEye camera parameters to and from a file. This was previously done via the attributes imageSource.loadParametersFromFile and imageSource.saveParametersToFile. These attributes were replaced by json-commands with the same names. See UEye Device Command Reference.

Others

Licensing Changes

• The maximum anchor count now only considers anchors that are currently enabled. It is therefore also checked when enabling anchors - and the corresponding error is raised when the number of concurrently enabled anchors is too high.

Documentation Changes

• Added documentation page for using different augmentation and init pose guidance than the ones used for tracking (see Using Different Augmentation and Init Pose Guide).