Using the Model Tracking Setup Scene on HoloLens 2

Level: Basic

Like the ModelTrackingSetup scene for all other platforms (see Using the Model Tracking Setup Scene), the corresponding scene on HoloLens 2 can be used to adjust parameters, observe the effect of the changes and save the adjusted configuration into a vl-file.

The scene can be found in our Mixed Reality Toolkit example. This has to be included in your Unity project manually. For more details see our corresponding Mixed Reality Toolkit page.

Note: Since this scene uses components which are only available on HoloLens 2, this example will not be usable on HoloLens 1.

Workflow for Setting Up Model Tracking on HoloLens

This section outlines a workflow allowing you to customize the tracking parameters for a new model on HoloLens and consequently use these parameters in your own application. Each step in the workflow is described in its own subsection in the following.

Replacing the Model in the Tracking Setup Scene and the Corresponding vl-File

As a first step, you must add the model you intend to track to the HoloLens2ModelTrackingSetup scene and make it usable as a tracking target:

1. Add your model data to your Assets folder under StreamingAssets/VisionLib/Examples/HoloLens/ModelTracking/Model.
2. Open the HoloLensModelTrackingConfiguration.vl located at StreamingAssets/VisionLib/Examples/HoloLens/ModelTracking and modify the the modelURI parameter: Replace the current parameter value (project-dir:Models/VLMiniCar.obj) with project-dir:Models/YOUR_MODEL_FILE_NAME.
3. Open the HoloLens2ModelTrackingSetup scene and import your object.
4. Replace the GameObject VLMiniCar (under MixedRealitySceneContent > VLSceneContent in the scene hierarchy) with your model.
5. Ensure the GameObject with your model is enabled at all times, keeping it visible and available as a guide, both for tracking as well as for adjustment of the initial pose (see the following section).

Adjusting the Init Pose

In general, the default InitPose does not work for all objects and requires adjustment:

1. Ensure your model is located under MixedRealitySceneContent > VLSceneContent in the scene's hierarchy. Also ensure the GameObject containing your model is enabled and visible.
2. Switch to the Scene View in the editor, make sure you are not in 2D mode.
3. Double click the GameObject containing your model in the hierarchy. This snaps the scene view to the model regardless of its location and size.

1. From there, move the scene view to a pose you would like to initialize from.
2. Select the VLHoloLensInitCamera under VLTracking in the hierarchy by single clicking it. An inset camera preview window becomes visible in the bottom right of the scene view.
3. Press the Align with view button on the VLHoloLensInitCamera component. This snaps the selected VLHoloLensInitCamera to the view you are seeing.
4. Check the camera preview inset and ensure the initial pose's point of view is as desired. Should this not yet be the case, refine your view and align the VLHoloLensInitCamera again.
5. Repeat this until the result is satisfactory.

Deploying and Starting the Scene on HoloLens

Deployment and starting of a scene on HoloLens is described in the general tutorial (see HoloLens Model and Poster Tracker Tutorial).

Adjusting the Parameters

Having started the scene, look at the palm of your hand. In the menu, press the Parameters On button. Adjust the parameters so the tracking of your object works well. Our Optional Tracking Parameters page gives an overview over these parameters and their function.

Saving the Current Configuration

After you finished adjusting your parameters, look at the palm of your hand again. In the menu, press the Save Configuration button. A notification appears after the configuration has been saved.

Copying the Saved vl-File to a New Scene

To make use of the saved vl-file in your own scene you have to do the following steps:

1. Open the HoloLens file system via the Windows explorer.
2. Copy the file Internal Storage/Videos/Captures/VisionLib/savedTrackingParameterConfiguration.vl into the StreamingAssets folder of your Unity project and give it an adequate name.
3. Copy your model file into the StreamingAssetFolder of your Unity project. Ensure the modelURI parameter in the configuration file is correct. In the simplest case, place the model in the same folder as the configuration file and use the project-dir: scheme (see File Access).
4. For visualization purpose, you should also place the model file somewhere outside the StreamingAssetFolder and drag it into the hierarchy of your scene as a child of the SceneContent GameObject.
5. Reference the .vl file in the VLTracking > TrackingConfiguration script under the Tracking Configuration File parameter.

For details on setting up tracking for your own model on HoloLens see HoloLens Model and Poster Tracker Tutorial.

Scene Controls

This section covers the purpose of the controls available in the HoloLens2ModelTrackingSetup scene. The controls are linked with the hand menu, so you have to look at the palm of your hand in order to show this menu. The following image shows an overview of the available controls:

Legend:

1. Show Debug Image/Hide Debug Image
2. Pause Tracking/Resume Tracking
3. Parameters On/Off
4. Reset Tracking
5. Save Configuration

Showing/Hiding Debug Image

The button in the top left of the menu (see (1) in the overview image above) toggles a window in the background showing the debug image. This greyscale image is the one on which tracking is performed. The line model is overlaid on top.

If you enable the parameters, the application will automatically enable the debug image. After finishing parameter adjustment and closing the parameter panel, the debug image will be disabled.

Pausing/Resuming Tracking

The Pause Tracking button in the top right of the menu (see (2) in the overview image above) halts tracking. The object will then remain at the current position until you Resume the tracking again. This feature might be useful, if you only want to position the object once and save processing power on HoloLens.

Parameters On/Off

The Parameters On button (see (3) in the overview image above) enables the parameter panel and the debug image. In this panel, you can see buttons for different parameters, which can be selected to adjust the corresponding parameter. An overview over these parameters and their meaning can be found at the Optional Tracking Parameters page. The debug image in the background should help to see the effect of the parameters on the tracking.

Selecting a parameter displays a slider for its adjustment. The line model will only be update after you finished setting the parameter.

The only parameter without a slider is the field of view. It is only used on HoloLens and not available on other platforms. In the following graphic, you can see the effect of this parameter: It adjusts the area covered by the camera image. A narrow field of view results in better tracking results, given that the user is looking directly at the target object and that the object fits entirely inside the camera image. A wide field of view helps stabilize the tracking in cases where the user is unable to see the object in its entirety. In other words: If the object is either very close or very large.

Reset

The Reset button (see (4) in the overview image above) resets tracking and returns to initialization mode. In other words, the currently tracked pose is dropped and the tracking state switches to lost. Note for advanced users: This internally performs a Reset Hard command.

Save Configuration

The Save Configuration button (see (5) in the overview image above) saves the current tracking parameters and init pose for the tracker into a .vl configuration file.

By default this file's URI is capture-dir:VisionLib/savedTrackingParameterConfiguration.vl. This folder is accessible via the Windows File Explorer on the HoloLens device under the path Internal Storage/Videos/Captures/VisionLib/savedTrackingParameterConfiguration.vl A custom URI can be set via Save Configuration URI in the hierarchy on Model Tracker Parameters on VLTracking > VLModelTracker. File schemes (see File Access) are supported. On the HoloLens, we recommended using the capture-dir scheme to be able to access the file from the Windows File Explorer.