ToF (Time of Flight) AR

ToF AR components

Here is an overview of the components provided by ToF AR. More in-depth descriptions of the components and data formats can be found in the ToF AR reference articles in the development guide.

Color

The color component provides access to the RGB camera data. The features provided are:

• Data retrieval
• Data visualization
• Controlling the RGB camera (resolution, auto-focus, auto-exposure, white-balance, and so on)

Plane

The plane component recognizes a plane in real time for any given point inside the Depth camera image. The features provided are:

• Plane data retrieval
• Dynamic updating of an object inside a Unity scene.

Can get/set up to eight detection configs at once. Plane detection stops executing if null or an empty config list is set.

Mesh

The mesh component generates a 3D mesh from the Depth camera image in real time. The features provided are:

• Vertex or triangle data retrieval
• Dynamic updating of a mesh object inside a Unity scene.
• Mesh reduction control
• Mesh generation excluding the mask part generated from the Segmentation component

Hand

The hand component recognizes in real time the location of the joints, pose and gesture of the hand and fingers by utilizing the depth camera data.

MarkRecog

The MarkRecog component recognizes optional marks located inside binary images. MarkRecog primarily supports the following function:

• Recognition of mark located inside image.

ARFoundationConnector

By using ARFoundationConnector, it's possible to use ARFoundation and ToF AR in the same UnityProject.

As ToF AR currently does not control the lower layer of the camera directly it instead uses the output data frames of the RGB or ToF camera obtained by ARFoundation to operate the Hand and Body recognition processes. Therefore, app developers can receive data frames from ToF AR with the same interface whether or not ARFoundation is present.

Body

The Body component performs Body recognition related processing and provides recognition result data to the application. It provides the following functions:

• Body recognition from depth camera image
• Display of Body recognition result
• Body gesture recognition

AR Foundation Human Stencil / Human Depth

Gets Human Stencil data and Human Depth data from AR Foundation and create mask texture for iOS.

HumanDetector

Creates a mask texture by estimating the human parts in the Color camera image.

Face

The Face component performs Face recognition related processing and provides recognition result data to the application. It mainly provides the following functions:

• Output Face recognition data and BlendShape data in an AR Foundation compliant format
• Display Face recognition results
• Display line of sight recognition results

Segmentation

The Segmentation component performs Segmentation recognition related processing and provides recognition result data to the application. It mainly provides the following functions:

• Create a mask texture by estimating the sky section from the Color camera image (When using SkyDetector).
• Create a mask texture by estimating the person section from the Color camera image (When using HumanDetector).

SkyDetector

Creates a mask texture by estimating the sky section from the Color camera image.