Level: Advanced

VisionLib also supports IDS uEye devices as one type of high-end industrial cameras, in order to meet quality and accuracy demands such as in automated manufacturing, inspection, and robotics. Using VisionLib together with uEye may be particularly useful to you if your use case is characterized by some of the following requirements:

high accuracy demands and the need for high resolution tracking with global shutter images
more fine-grained control over the image capturing and image generation parameters
wired connection over large distances (e.g. via Ethernet)
static camera setup and a moving object
special lenses for wide or narrow field of view

Prerequisites

In order to use uEye cameras, it is necessary to install the uEye driver package first. It can be obtained from the IDS download section. You may need to create an account on the IDS web page.

In general, uEye cameras can be accessed with VisionLib in two ways:

ordinary access via the operating system, e.g. DirectShow on Windows
access via a direct uEye interface

The DirectShow interface should work immediately after uEye driver installation. Access your uEye camera with VisionLib like any other camera or webcam by using the camera input source type as described in Handling Input Sources. It works with any version of uEye drivers.

The VisionLib uEye interface provides you with additional possibilities for camera control and is the main focus of this article. To make it work, the version of the driver package must match the uEye library version with which the VisionLib was built, which is IDS Software Suite 4.94 WHQL. If it is not the most recent version anymore, you will find it in the archive section of the IDS web page.

The VisionLib uEye Interface

The uEye Example Scene in Unity

In Unity, import the VisionLib.SDK.Examples-Unity.unitypackage and open the UEyeCameraTracking example scene from VisionLib Examples/ModelTracking/UEye to see how the VisionLib uEye interface can be used. This scene also enables you to calibrate the uEye camera and configure some of its parameters with UI elements. Furthermore, you can save and load the final settings to and from the internal camera memory (EEPROM) or a specified file.

In order to begin the configuration of your camera, press the Calibrate button and select the desired uEye camera by pressing the <serialNumber> VisionLib uEye-Interface button. If this option is not available, the camera is not properly connected or the plugin has not been loaded. In the latter case, check the console messages and ensure the drivers are installed.

Afterwards, you can use the Parameters on button to show the parameter sliders for configuration. There are options to set exposure, gain, black level, gamma and color mode.
Modify them until you get a sharp image. Once you are satisfied with your settings, you can save them to a file or the EEPROM of the camera with the corresponding buttons. The file used for saving or loading must be specified by a file path in the text field above the buttons and the filename extension must be .ini.

Now start the actual calibration of the camera by hitting the Run button. When you are done, drop the resulting calibration file in the TrackingConfiguration component of the VLTracking GameObject. Finally, press Start and test your configuration by tracking the handcrafted paper car.

Loading the Plugin

Manual loading of the plugin is required if you want uEye cameras to be listed in the available devices list of the device info, for example to show uEye cameras in the input selection. In the example scene, the plugin is loaded automatically by the UEyePluginLoader component of the VLUEyeCameraParameters GameObject. When using a custom scene, you can achieve the same behaviour by adding the VLUEyeCameraParameters prefab to your hierarchy. In the C-interface, load the plugin by calling vlWorker_LoadPlugin("VideoUEye").

Note: The plugin will be loaded automatically when using a tracking configuration file with a device of type uEyeCameraMono, like described below.

Note: The VPVideoUEye.dll always has to be located next to vlSDK.dll.

If something is not working correctly, make sure to have a look at the log messages for guidance.

Accessing the uEye Camera

In Unity, it is possible to select the VisionLib interface by choosing <serialNumber> VisionLib uEye-Interface in the input source selection dialogue.

Alternatively, the uEye camera can be accessed through the tracking configuration file. Therefore, one has to specify the type of the image source in the input section to be uEyeCameraMono. The uEye plugin will be loaded automatically in this case. Moreover, one can specify a deviceID in the data section of that image source to choose a specific device if more than one uEye camera is connected to your system. You can copy the specific deviceID from the console log in Unity after selecting the desired device in the input selection of the example scene. If nothing is specified, the first uEye device detected by the system will be used.

Please have a look at the exemplary input section of a tracking configuration file below:

"input": {
    "useImageSource": "uEyeCamera",
    "imageSources": [{
        "name": "uEyeCamera",
        "type": "uEyeCameraMono",
        "data": {
            "undistort": false,
            "deviceID": "DeviceUEyeCameraMono<serialNumber>",
            "useColor": true,
            "loadParametersFromEEPROM": true
        }
    }]
}

Especially, notice the option to undistort the image for more accurate tracking after calibrating your camera. As uEye cameras support a wide variety of lenses, it is recommended to always calibrate them in order to receive the best results. For more details on the calibration process, see Camera Calibration.

While uEye cameras support many attributes that are available on other devices (see Configuration File Reference), there are some uEye specific optional parameters that can be set in the data section of a uEyeCameraMono device. These parameters are ignored by other device types:

Parameter	Type	Default Value
loadParametersFromEEPROM	bool	false
	If set to `true`, the program will attempt to load the camera parameters from the camera's internal memory after connecting with the camera. You can save a camera configuration to the internal memory by using the `saveParametersToEEPROM` command or by using `uEyeCockpit`, a software published by IDS and shipped with the uEye drivers. If both `loadParametersFromFile` and `loadParametersFromEEPROM` are set, the parameters are loaded from file.
loadParametersFromFile	string
	If specified, the program will attempt to load the camera parameters from the corresponding uEye parameter file (`.ini`) after connecting with the camera. You can generate such files using the `saveParametersToFile` command or by using `uEye Cockpit`, a software published by IDS and shipped with the uEye drivers. If both `loadParametersFromFile` and `loadParametersFromEEPROM` are specified, the parameters are loaded from file.
serialNumber	string
	Serial number of the camera to be used (e.g. "4002736430"). Guarantees that only the camera with this specific serial number is used. If the camera is not connected to the system, an error will be thrown. It is completely redundant to the `deviceID` and we recommend always using the `deviceID`instead, as the `serialNumber` may be deprecated in a future release. The `deviceID` corresponding to a given serial number is "DeviceUEyeCameraMono<serialNumber>" (e.g. "DeviceUEyeCameraMono4002736430").

Accessing uEye Parameters

Unity

To configure uEye parameters in a custom Unity scene, use the VLUEyeCameraParameters prefab from VisionLib SDK/Core/Prefabs/TrackingParameters/. This prefab provides all accessible parameters. Use the events emitted by the individual parameters to read their values and the SetValue function of each parameter to set them to your own desires. Additionally, the UEyeCameraParameters component attached to the prefab provides functionality to use the internal camera memory.

C-Interface

In VisionLib's C-interface, camera parameters are controlled via json-commands. The attributes listed below are accessed via SetAttribute and GetAttribute commands or the special commands respectively. For example, you could load the camera settings from internal memory or change the color mode to monochrome by using the json-commands

{
  "name": "imageSource.loadParametersFromEEPROM"
}

{
  "name": "setAttribute",
  "param":{
      "att": "imageSource.colorMode",
      "val": "6"
  }
}

in vlWorker_PushJsonCommand. Remember to escape the necessary characters in the C-string.

Attributes and Commands

Attributes

The following attributes are accessible by the interface:

Attribute	Type	Writeable
imageSource.exposure	float [exposureMin..exposureMax]	yes
	The exposure/shutter value of the camera in milliseconds.
imageSource.exposureMax	float	no
	The maximal exposure value for the current camera settings.
imageSource.exposureMin	float	no
	The minimal exposure value for the current camera settings.
imageSource.gain	integer [0..100]	yes
	The gain value of the camera. A value of 0 corresponds to the minimum possible gain while 100 corresponds to the maximum possible gain.
imageSource.blackLevel	integer [0..255]	yes
	The black level value of the camera.
imageSource.gamma	integer [0..1000]	yes
	The gamma value of the camera. The `gamma` parameter cannot be used in combination with colorMode 11 (raw).
imageSource.colorMode	integer [0, 1, 6, 11, 128, 129]	yes
	Read / Write the color mode, i.e. the pixel format of the images as transmitted from the camera to VisionLib. Can be set to: 0 (BGRA), 1 (BGR), 6 (monochrome), 11 (raw sensor / Bayer format), 128 (RGBA), or 129 (RGB) The parameter `colorMode` controls the format in which images are transferred from the camera to VisionLib. The arguments and return types of the command represent the encoding of color modes in the uEye SDK and their meanings are also described in the uEye documentation. Only the above color modes are supported by VisionLib, any other value is not set and results in a warning. It is possible that a camera is set to an unsupported color by default or by loading a configuration from EEPROM or from an `.ini` file. In this case the program will abort with an error. You can accelerate tracking by selecting mode 6 (monochrome) together with the option `"useColor": "false"` in both, the `input`-section and the `tracker`-section of the tracking configuration (vl-file) at the expense of a small loss in robustness of auto initialization of the tracker. The mode 11 (sensor raw) can be used to shorten the transmission time between camera and VisionLib and to reduce the latency.

Commands

For a list of the uEyeCamera specific commands see UEye Device Command Reference.

uEye Cockpit

If the set of parameters currently available is not enough to configure the camera for your needs, you can also use uEye Cockpit, which is shipped with the uEye drivers. With `uEye Cockpit you can adjust any other camera parameter. You can save the whole configuration either on your hard drive (.ini file) or on the camera memory (EEPROM). From there you can load it with the VisionLib uEye interface to use your custom configuration.

Note: uEye Cockpit and VisionLib cannot access the same camera at the same time.

documentation