Mixed Reality Toolkit Integration

Level: Basic

VisionLib can be used together with Microsoft's Mixed Reality Toolkit (MRTK). The steps for integrating the MRTK and VisionLib in a Unity project are described below. Consider reading the HoloLens Tutorial for more details regarding HoloLens applications with VisionLib.

The Mixed Reality Toolkit package also contains the ModelTracking setup scene for HoloLens 2. This scene can serve as a starting point for development of your own application using VisionLib in combination with the MRTK. For more details on this, see the Tutorial page for the HoloLens2 ModelTracking setup scene.

Note: It is not possible to remote control the HoloLens from Unity when using VisionLib at the moment. You have to deploy your application using Visual Studio to test the tracking properly.

Preparation

Please ensure you have set up your system for HoloLens Development with Unity as per the following requirements:

• Windows 10, Windows 10 SDK, Developer Mode activated
• HoloLens with activated Developer Mode
• Visual Studio 2019 (with UWP Tools installed) or higher
• Unity with .NET/IL2CPP scripting backend

Mixed Reality Toolkit

This guide has been tested with MRTK v2.8.3 and Unity 2021.3 LTS.

1. In the build settings of your project, select UWP with HoloLens as target device and click Switch Platform.

VisionLib

1. Import VisionLib.SDK-Unity.tgz as well as VisionLib.SDK.HoloLens-Unity.tgz via the package manager.
2. Copy your license file to Assets/StreamingAssets/VisionLib.

Mixed Reality Toolkit

1. Download the Mixed Reality-Featuretool.
2. Start MixedRealityFeatureTool.exe
   1. Wait for the tool to update itself, then press Start.
   2. Select the root folder of your Unity project as Project Path and click Discover Features.
   3. Select the Mixed Reality Toolkit Foundation and Mixed Reality Toolkit Standard Assets and click GetFeatures.
      
   4. Click Import and then Approve to include the Mixed Reality Toolkit.
   5. Close the Microsoft Mixed Reality Feature Tool by pressing Exit.
3. Open your Unity project. The MRTK Project Configurator will be opened. Apply the requested settings changes.
4. For testing on HoloLens 2, you can add the VisionLib.SDK.MRTK.Examples-Unity.unitypackage via Assets > Import Package > Custom Package and use the HoloLens2ModelTrackingSetup scene from VisionLib MRTK Examples/TrackingSetup/Scenes. More details on this scene can be found in the Using the Model Tracking Setup Scene on HoloLens 2
5. Add your scene to the build settings and follow the build instructions given in the Deploying to HoloLens