AR Foundation Support (Beta)

Level: Advanced

This tutorial explains how to set up VisionLib object tracking when using AR Foundation in a Unity scene. You should already be familiar with setting up VisionLib in a Unity scene without AR Foundation, as described in the Model Tracker Tutorial.

Note: The provided code for using AR Foundation together with VisionLib was developed for Unity 2019.4 LTS.

Note: The ARFoundation support is only included in the experimental vlUnitySDK package, which can be downloaded here: https://visionlib.com/customer-area/.

Note: The ARFoundation support requires a license file containing the AllowInjectingImage feature.

Example Scene

• Start by creating an empty Unity 2019.4 LTS project.
• Import the experimental vlUnitySDK package into the project.
• Copy the license file to StreamingAssets/VisionLib/ reference it in the VLTrackingConfiguration component of the VLTracking GameObject.
• The example scene for AR Foundation support can be found in the folder Assets/VisionLib/Examples/Experimental/ARFoundation/Scenes/ of the experimental vlUnitySDK package.
• This scene already contains all necessary elements from both ARFoundation and vlUnitySDK, to track the VisionLib paper car. Details on how to setup the tracking for your own models can be found at the bottom of this page.

Requirements

AR Foundation is currently only supported on Android and iOS. Our implementation has been tested on Android 10, Android 11, iOS 13 and iOS 14.

It is not possible to test scenes in the editor when using AR Foundation. Please deploy them to a device in order to test them.

Package Dependencies

• Use the Unity Package Manager (Window > Package Manager) to install the AR Foundation package in the recommended version. (We tested it with 2.1.8.)

iOS

• If you want to deploy to iOS, you have to install the ARKit XR Plugin in the recommended version. (We tested it with 2.1.9.)

Android

• If you want to deploy to Android, you have to install the ARCore XR Plugin in the recommended version. (We tested it with 2.1.8.)

Build Settings

• At File > Build Settings select iOS or Android and click on Switch Platform if it is not already selected.

In the following sections, Player Settings will refer to Edit > Project Settings > Player > Settings For (Android / iOS).

• To enable the vlUnitySDK code for AR Foundation support, you have to set the Scripting Define Symbol ENABLE_ARFOUNDATION. You can set this in Player Settings > Other Settings > Scripting Define Symbols.
• In the same menu you have to enable the Allow unsafe Code checkbox. Acquiring images from AR Foundation uses direct access to memory that is marked as unsafe by the C# language specification.

iOS

• At Player Settings > Other Settings
  • Set Camera Usage Description* to any value, e.G. Augmented Reality
  • Set Target minimum iOS Version to at least 11.0
  • Set Architecture to ARM64

Android

• At Player Settings > Other Settings
  • Make sure that your Minimum API Level is at least set to Android 7.0 Nougat (API level 24). Otherwise you will get the following error message:

    BuildFailedException: ARCore Required apps require a minimum SDK version of 24. Currently set to AndroidApiLevel21

  • Enable Auto Graphics API. If you want to disable Auto Graphics API, make sure Vulkan graphics API is not on top of the list of Graphic APIs. Otherwise you will get the following error message when trying to build:

    BuildFailedException: You have enabled the Vulkan graphics API, which is not supported by ARCore.

  • As of now, Vulkan is not supported by ARCore.
• At Player Settings > XR Settings, make sure that ARCore Supported and Virtual Reality Supported are unchecked. Otherwise you will get the following error message:

• At XR > ARCore, create the configuration asset for AR Core by clicking Create and store it in the Assets directory of your project. (Unity will suggest to store it there).
  
  • After creating that file, you have to set the requirement mode of ARCore to Optional.
    
  • Otherwise you will get the following error message:
    
• At this point the example scene can be deployed and tested.

Custom Tracking Configuration

In the AR Foundation Example, we use the tracking configuration in StreamingAssets/VisionLib/Examples/Experimental/ARFoundation/ARFoundation.vl. You can use that as a starting point for your own configuration file as described in the Model Tracker Tutorial. Please copy the file instead of modifying the existing file to prevent issues when updating to new versions of VisionLib.

The VLTracking GameObject itself contains the VL Tracking Configuration component, where you can set the path to your tracking configuration file.

Since Unity sets the position of the AR Foundation camera in meters, your model also has to be scaled accordingly. Please also set the metric to "m" in your tracking configuration file.

Note: The AR Foundation setup has to use the HoloLensModelTracker in its tracking configuration file, since we track objects relative to the AR Session Origin.

VisionLib XR Tracking Settings

The components that connect AR Foundation with VisionLib are placed on the GameObject called VLXRTracker.

The parameters have the following effects:

• If you want to reduce the workload that is created by VisionLib object tracking, you can set the Max Tracking FPS to a lower value - or increase it to trigger object tracking more frequently.
• If you select Use Pose From Tracking Config, the initial position will be read from the tracking configuration file instead of the VLInitCamera.
• If you deselect Keep Upright, the initial position of the tracking content will not be geared to the world up vector.
• The value in Smooth Time is the time in seconds that the tracking content will take to go to the position of the object tracking result. Higher values will make the transitions smoother but the updates will be less immediate.

Add VL Tracking to an existing AR Foundation Scene

To enable VisionLib object tracking in an existing AR Foundation scene, select Game Object > VisionLib > VL XR Tracking. This will add all necessary GameObjects and Components under a parent object called VLTracking to your scene. It will also try to connect them automatically to the AR Foundation GameObjects.

Note: The menu entry is only available, once you added the ENABLE_ARFOUNDATION scripting define.

Once the VLTracking GameObject is created, set your tracking configuration in the public parameter of the object's VL Tracking Configuration component.

Adjust the Transform of the VLInitCamera, so that the camera is pointing to the tracking target from the point that should be used to start tracking from.

Finally, you have to set the Content property of the VLXR Tracker Behaviour component of the VLXRTracker to the GameObject that should be placed as an augmentation on the tracked object. The example uses the Scene Content GameObject for that purpose.