Release Notes 2.3

Version 2.3.0

Version Info

Release date: 25.07.2022
Release info: Feature Release

General

This is a minor feature release for VisionLib which adds:

• Enhancements of the combination of model tracking with SLAM, which increase the update frequency of the model transform update and improve the reinitialization after tracking becomes "lost" or "critical".
• Support for the universal render pipeline in Unity.
• Improvement of tracking setup workflow in the AdvancedModelTracking example scene, which has therefore been renamed to ModelTrackingSetup.
• A separate scene for modifying tracking parameters on HoloLens 2 using the Mixed Reality Toolkit.
• Several bug fixes and enhancements.

Special thanks to all our users for your valuable feedback!

Features

General

• Enhanced staticScene support by decoupling SLAM from ModelTracking.
• "staticScene": true formerly estimated the scale of the SLAM coordinate system. This is now achieved by setting estimateWorldScale to true.
• Enhanced options for the recognition of "lost" tracking if extendibleTracking is activated. Previously, the model tracking would remain stuck in the "critical" state as long as SLAM pose predictions were still available. This prevented reinitialization. Two new parameters are now available to manage the influence of SLAM on the tracking state:
  • allowedNumberOfFramesSLAMPrediction: Limits the number of frames for which purely SLAM pose predictions may keep tracking alive, i.e. prevent the transition from state "critical" to "lost".
  • allowedNumberOfFramesSLAMPredictionObjectVisible: This has the same effect as allowedNumberOfFramesSLAMPrediction. However, the frame counter only increases while the SLAM predicted object location is inside the camera's field ov view.
• Added recovery paths from "critical" tracking while extendibleTracking is used. Previously, while in state "critical", the tracked object was only searched for at the location predicted by SLAM. This search is now additionally performed using every available form of initialization data (e.g. AutoInit, init pose, learned poses). This significantly increases the likelihood of tracking recovery despite SLAM pose prediction drift or possible object movements.
• Enhanced handling of invalid or implausible SLAM data.

Unity

• Added a separate package and example scene for Universal Render Pipeline support. Please read our URP article for more details.
• Improved the input source selection on the TrackingConfiguration component. There is now an option to start the tracking using an image sequence (located at a specific URI). Testing VisionLib scenes with image sequences no longer requires manual changes to the input section of the tracking configuration file.
• Added the ModelTrackerParameters component to the VLModelTracker prefab. Set an URI and call SaveCurrentConfiguration on this component to save the current model tracker parameter configuration to a file. This allows you to permanently save volatile configuration parameter changes made at runtime.
• Added the possibility to save the current parameter configuration via a button in the ModelTrackingSetup example scene.
• Added the example package VisionLib.SDK.MRTK.Examples including the new HoloLens2ModelTrackingSetup scene. This scene serves two purposes:
  • Example for the usage of the VisionLib SDK together with the MRTK
  • Sandbox for live configuration parameter adjustment directly on the HoloLens 2 while deployed. The modified tracking parameters can be saved in a custom configuration. The saved configuration is retrievable via the file explorer.
• Added a VLImageSourceParameters prefab variant for HoloLens scenes that contains the FieldOfView parameter.

macOS

• Added native compatibility with ARM (Apple silicon).

Enhancements / Bugfixes

General

• The PosterTracker now shows the state "critical" if the current pose was only predicted using SLAM. Previously, it would report the "tracked" state in this situation, obscuring the true state.
• Fixed a bug in the ImageSequence when using negative step size.
• Fixed a bug in the ImageRecorder where the frameIndex would increase despite no frame having been recorded.
• Assigned names to VisionLib related background threads.
• Fixed a bug that caused dropped frames when executing the MultiModelTracker in parallel.
• Reduced the file size of the libraries by between 6 % and 11 % (depending on the platform).
• Unified the handling of tracking in single- and multi-camera use-cases. The available commands and attributes of the ModelTrackerMultiView have been expanded to match the one of ModelTracker in most cases.
• Added recording of timestamps in image sequences if extendibleTracking is active. Timestamps are read and used when replaying such image sequences.
• Improved pose smoothing (especially with static scenes):
  • The smoothing does not overshoot anymore.
  • Pose smoothing is now done in the world coordinate system if SLAM is available.
  • If staticScene is enabled, pose smoothing is much faster due to the decoupling of SLAM and ModelTracking.

Unity

• Encapsulated all scripts in examples package (VisionLib.SDK.Examples-Unity) into new assembly VisionLib.SDK.Examples.
• Added toggle for ShowLineModel in the TrackingSetupScene.
• Fixed a bug that accesses MonoBehaviours after they have been destroyed.
• Fixed a bug in the shutdown of the AR Foundation example scene.
• Fixed a bug in the init issue generation process which caused NullReferenceExceptions.
• Fixed a bug in setting the init pose while using AR Foundation.
• Fixed a bug in the TrackingModel which prevented some meshes from being sent to VisionLib.

HoloLens

• Fixed a bug in the initialization order of the global coordinate system.

iOS

• Fixed a bug which caused slightly different tracking results on iOS compared to all other platforms.

Windows

• Fixed a memory alignment bug under Windows x86.

Interface Changes

• Renamed parameter minInlierRatioInit to minInitQuality and minInlierRatioTracking to minTrackingQuality.
  Note: The old parameter names are now deprecated and will be removed in a coming release.
• Rotation parameters were previously denoted "q" and "r" respectively in different contexts. This ambiguity has been removed. In vl-files and commands, rotations are now always named "r".
  Note: You may still use "q" where it was previously used, but this is now deprecated and support will be removed in the next release.

General

• Removed disused tracking configuration parameters from documentation:
  • minFeatureDistance
  • minCornerness
  • minTriangulationAngle
• Changed minimum value of lineSearchLengthTrackingRelative and lineSearchLengthInitRelative to 0 in order to accommodate higher image resolutions.
• Added new command for receiving all parameters from the current ModelTracker as a Json string. Available as:
  • getNonDefaultAttributes in VisionLib.SDK.Native
  • ModelTrackerCommands.GetNonDefaultAttributesAsync in VisionLib.SDK.Unity
• Deprecated the Objective-C interface.
  Note: The support for this interface is discontinued and it will be removed in the next major release.
• Unsuccessful parsing of a poster's Transform will result in an error now.

Unity

• Cleaned up separation between synchronous and asynchronous TrackingManager: Split TrackingManager into base TrackingManager and AsynchronousTrackingManager. SynchronousTrackingManager now extends the new abstract base class TrackingManager.
• Renamed the AdvancedModelTracking example scene to ModelTrackingSetup. This better reflects its purpose, namely: Setting up, debugging and saving a custom tracking configuration.
• Added the possibility to call a tracking reset on a specific anchor.
• Replaced ModelTransform.operator == with more explicitly named function ModelTransform.IsSimilarTo in order to avoid misuse.
• Deprecated TrackingManagerReference class. The need to inherit from TrackingManagerReference for access to the TrackingManager has been removed. To access the TrackingManager please use the new static property TrackingManager.Instance.
  Note: The TrackingManagerReference will be removed in the next major release.
• Deprecated SynchronousTrackingManagerReference class. The need to inherit from SynchronousTrackingManagerReference for access to the SynchronousTrackingManager has been removed. To access the SynchronousTrackingManager please use the new static property SynchronousTrackingManager.Instance.
  Note: The SynchronousTrackingManager will be removed in the next major release.
• Fixed some coordinate system problems when using TrackingAnchors in combination with AR Foundation.
• Deprecated the parameter useAllChildNodes in TrackingModel.UpdateModelProperties and TrackingModel.UpdateModelPropertiesAsync.
  Note: This parameter no longer has any function and will be removed in a future major release.

Native

• Updated external dependencies:
  • angle
  • assimp
  • boost
  • curl
• The ModelTrackerMultiView will no longer accept any setter via its node:
  • vlWorker_SetNodeImageSync will no longer work with indexed key "image" for ModelTrackerMultiView. Please use the InjectMultiView device instead.
  • vlWorker_SetNodeIntrinsicDataSync will no longer work with indexed key "intrinsic" for ModelTrackerMultiView. Please use the InjectMultiView device instead.
  • vlWorker_SetNodeExtrinsicDataSync will no longer work with indexed key "relativeExtrinsic" for ModelTrackerMultiView. Please use the InjectMultiView device instead.
  • vlWorker_SetNodeExtrinsicDataSync will no longer work with key "initPose". Please use jsonCommand with setInitPose instead.
  • Those keys can also no longer be used in a manual graph definition to connect with other trackers or devices. Please use the key "multiViewFrame" instead.
• Previously, the ModelTrackerMultiView used the same indices for accessing the camera images as specified for the device. Due to internal refactoring, the indices of the internally used data will now be sequential starting from 0: E.g. if trackingCameras is set to [1, 5], ModelTrackerMultiView previously provided the debug image keys debugImage_1 and debugImage_5. Now the tracker will provide the keys debugImage_0 and debugImage_1 instead.
• To improve type safety, a warning will be triggered if setting a json-parameter (via command or vl-file) to an incorrect type, as long as the incorrect type can be converted to the correct one. This is the case e.g. when passing an integer instead of a bool passing a double value as text in a string.
  Note: This warning will be converted into an error in the next major release.
• When passing a string parameter in a query string, the parameter must be enclosed in quotation marks. Otherwise a warning will be triggered.
  Note: This warning will be converted into an error in the next major release.

macOS

• The minimal required operating system version has been increased to macOS 10.13 (High Sierra).

Others

Licensing Changes

• Now using a new method for hostID generation on Windows which avoids changes to hostID when the network adapter changes. Old licenses will still be valid.
• The hostID on HoloLens now only depends on the device and the certificate of the application (and no longer on the application itself). This enables the development of several applications on the same HoloLens with just one engineering license.

Documentation Changes

• Added a section to the [Record and Replay Image Sequences](vlUnitySDK_imageSources2Record) documentation that explains how to replay image sequences in a Unity scene without the need to change the tracking configuration file manually.
• Moved the [Release Notes](vlGeneral_ReleaseNotes) page to the top level to improve visibility.
• Added list of possible values for setting the camera resolution into the schema.json file.
• Added support for the CameraCalibration tracker in the schema.json.
• Added documentation pages explaining the use of the ModelTrackingSetup example scene
  • Generally in the unity editor
  • Specifically with the HoloLens