Mixed Reality Toolkit Integration

Level: Basic

VisionLib can be used together with Microsoft's Mixed Reality Toolkit (MRTK). The steps for integrating the MRTK and VisionLib in a Unity project are described below.
Consider reading the HoloLens Tutorial for more details regarding HoloLens applications with VisionLib.

The Mixed Reality Toolkit package also contains the ModelTracking setup scene for HoloLens 2. This scene can serve as a starting point for development of your own application using VisionLib in combination with the MRTK. For more details on this, see the Tutorial page for the HoloLens2 ModelTracking setup scene.

Note: It is not possible to remote control the HoloLens from Unity when using VisionLib at the moment. You have to deploy your application using Visual Studio to test the tracking properly.

Preparation

Please make sure that you have installed all required tools and adjusted the settings for HoloLens and Unity Development.

• Windows 10, Windows 10 SDK, Developer Mode activated
• HoloLens with activated Developer Mode
• Visual Studio 2019 (with UWP Tools installed) or higher
• Unity with .NET/ IL2CPP Scripting Backend

Mixed Reality Toolkit

The Mixed Reality Toolkit is compatible with Unity 2018.4 or higher. This guide has been tested with MRTK v2.5.4 and Unity 2019.4.

1. In the Build Settings of your project, choose UWP with HoloLens as target device and click Switch Platform.

VisionLib

1. Import the VisionLib.SDK-Unity.tgz as well as the VisionLib.SDK.HoloLens-Unity.tgz via the package manager.
2. Copy your license file to Assets/StreamingAssets/VisionLib.

Mixed Reality Toolkit

1. Download the Mixed Reality-Featuretool.
2. Start MixedRealityFeatureTool.exe
   1. After the tool has updated itself press Start.
   2. Select the root folder of your Unity project as Project Path and click Discover Features.
   3. Select Mixed Reality Toolkit Foundation and Mixed Reality Toolkit Standard Assets and click GetFeatures.
      
   4. Click Import and then Approve to include the Mixed Reality Toolkit.
   5. Close the Microsoft mixed Reality Feature Tool by pressing Exit.
3. Open your Unity project. The MRTK Project Configurator will be opened. Apply the requested settings changes.
4. For testing on HoloLens 2, you can add the VisionLib.SDK.MRTK.Examples-Unity.unitypackage via Assets > Import Package > Custom Package and use the HoloLens2ModelTrackingSetup scene from VisionLib MRTK Examples/TrackingSetup/Scenes. More details on this scene can be found in the Using the Model Tracking Setup Scene on HoloLens 2
5. Add your scene to the build settings and follow the build instructions of the HoloLens Model and Poster Tracker Tutorial