UnitySDK Quick Start

Level: Basic

Setting up a quick AR Demo using the 3D Model Tracker in Unity

This article will guide you through the first steps with the VisionLib SDK for Unity. You will learn how to track our paper car and how to replace the car with your own model.

Prerequisites

Before you start, be sure to have the following installed or at hand:

• Unity 2021.3 LTS or higher
• A webcam connected to your computer
• The latest VisionLib.SDK-Unity package, downloaded from the customer area
• A valid license file, downloaded from the customer area
• A physical paper mini car as a test object. You can download the template here. Then print the PDF and follow the instructions to craft the car.

Unity Example Project

• Start Unity and create a new project.
• Import the VisionLib.SDK-Unity.tgz via the package manager.
• Import the VisionLib.SDK.Examples-Unity.unitypackage via Assets > Import Package > Custom Package.
• Browse to VisionLib Examples/ModelTracking/Simple in the project navigator and open SimpleModelTracking.unity.

License File

• Copy your license file into the StreamingAssets/VisionLib folder of your Unity project.

Select the VLTracking GameObject in the hierarchy and drag the license file from the project view into the public License File field in the inspector of the TrackingConfiguration component.

Run the Application

• Ensure your external webcam is connected to your system.
• Start the application in the editor by pressing the play button.
• Start the tracking engine by pressing the Start button.

The example scene will ask you to select your camera now.

• Select the desired camera and proceed.

You should see the live camera stream and a rendered view of the 3D mini car model in Unity's game view now.

• Next, move your camera to align the rendered model with the real paper car.
• Once recognized, the 3D model should snap to the physical car in the live video – and be tracked.

If everything worked correctly, your game view should look like in the picture above.

In this simple demonstration, the same 3D model that is used for tracking is also used as augmentation (i.e. superimposed over the camera stream). You can, of course, replace this simple augmentation with an arbitrary and more complex model using Unity's full range of features. You can also see the new setup for this feature in the corresponding documentation page Using Different Augmentation and Init Pose Guide. Unlike other tracking techniques, model tracking will always use a physical object as reference. One of the major advantages of this technique is that the spatial relationship between the camera and the model in real world units becomes known as soon as the real object is recognized and tracked in the images. This makes placement of augmented content with respect to the tracked object very straight-forward.

Changing the Model

To track a different model, you need:

• A digital 3D model of the physical object you want to track. This model has to be loaded into Unity. To do this, you either should use a file format supported by Unity or a third party library to include your desired format.
• The real physical object for live tracking.

In this example, we will replace the mini car with a sports car model. To do so:

• Stop the current demo by leaving the play mode.
• Copy the new digital 3D model into the Models folder of your project.
• In the hierarchy under SceneContent/VLTrackingAnchor, disable the VLMiniCar GameObject.
• In the project panel, open the Models folder.
• Drag your new model into the hierarchy under SceneContent/VLTrackingAnchor.

• In the hierarchy, select SceneContent/VLTrackingAnchor. There should be a warning telling you that no TrackingMesh is found in the subtree. This means that the TrackingAnchor could not find any models designated as tracking geometry. To solve this issue, press the + button below the HelpBox stating Add TrackingMesh components to all models. This marks all Meshes under the TrackingAnchor as tracking geometry.

• In the final step, the initial tracking pose (init pose) has to be adjusted. This is the relative pose between the camera and the real tracked object at which tracking will be initialized. The camera will later have to be moved to this pose to engage tracking. To adjust this pose for the new model:
  • Switch to the game view.
  • To obtain a good initial guess for the init pose, press the Center in Slam Camera button on the TrackingAnchor. This will move the TrackingAnchor to a position where it is fully visible in the Game view window. Pressing the button will also adjust the near and far plane of the selected camera if this is necessary to keep the model entirely visible.
  • Continue adjusting the TrackingAnchor's pose until you have found a satisfactory initial pose.

Check for Issues in the scene

VisionLib checks for a range of issues with the scene setup. To see if there are any such issues in your scene (and resolve them), select the VLTracking GameObject in the scene hierarchy. If there are any issues, they will be displayed in the Tracking Configuration component. Most of these issues come with one (or more) easy fixes. A common problem at this point in the configuration may be that your TrackingMesh cannot be serialized. This happens if the Read/Write Enabled option in your model's import settings isn't enabled. To solve this, just press the Fix button on the specific issue.

Run the Application again

• In Unity, press play to start the scene again.
• Next, press the Start button and select a camera. The new model should now be rendered as you've defined its init pose in the scene.
• Again, match the real object and the virtual model by moving the camera. If everything is fine, tracking should work immediately. The virtual model and the physical object should overlap precisely:

Conclusion

Congratulations! You just tested VisionLib's 3D model tracking and exchanged the demo model.

If this did not work with your model, we recommend loading the model into the ModelTrackingSetup example and troubleshooting the tracking there. Refer to this article for guidance on finding the correct tracking parameters for your model: Using the Model Tracking Setup Scene.

Next Steps

You might have noticed a warning about your camera not having been calibrated. Calibration is a crucial step, which generally improves tracking performance by a great deal. We therefore recommend calibrating your camera as the next step. The calibration process is described in this tutorial: Camera Calibration.