Components | Developer World

ToF AR components

Here is an overview of the components provided by ToF AR. More in-depth descriptions of the components and data formats can be found in the ToF AR reference articles in the development guide.

ToF component output screen. (low resolution)

ToF

The ToF component provides access to the depth camera’s Depth data, Confidence data and PointCloud data. The features provided are:

Data retrieval
Data visualization
Controlling the Depth camera (frame rate, shutter speed, and so on)

Color

The color component provides access to the RGB camera data. The features provided are:

Data retrieval
Data visualization
Controlling the RGB camera (resolution, auto-focus, auto-exposure, white-balance, and so on)

Plane

The plane component recognizes a plane in real time for any given point inside the Depth camera image. The features provided are:

Plane data retrieval
Dynamic updating of an object inside a Unity scene.

Can get/set up to eight detection configs at once. Plane detection stops executing if null or an empty config list is set.

Mesh

The mesh component generates a 3D mesh from the Depth camera image in real time. The features provided are:

Vertex or triangle data retrieval
Dynamic updating of a mesh object inside a Unity scene.
Mesh reduction control
Mesh generation excluding the mask part generated from the Segmentation component

Coordinate

The coordinate component provides a means to convert between the different coordinate systems of the Depth and RGB cameras. It is mainly used for retrieving converted coordinate data through getting component properties.

Hand

The hand component recognizes in real time the location of the joints, pose and gesture of the hand and fingers by utilizing the depth camera data.

MarkRecog

The MarkRecog component recognizes optional marks located inside binary images. MarkRecog primarily supports the following function:

Recognition of mark located inside image.

Modeling

The Modeling component accumulates depth data of depth cameras over multiple frames and generates 3D Mesh data. It mainly provides the following functions:

Start/end 3D modeling process
Set 3D modeling parameters
Output data
Mesh generation excluding the mask part generated from the Segmentation component

Modeling component output screen. (low resolution)

ARFoundationConnector

By using ARFoundationConnector, it's possible to use ARFoundation and ToF AR in the same UnityProject.

As ToF AR currently does not control the lower layer of the camera directly it instead uses the output data frames of the RGB or ToF camera obtained by ARFoundation to operate the Hand and Body recognition processes. Therefore, app developers can receive data frames from ToF AR with the same interface whether or not ARFoundation is present.

Body

The Body component performs Body recognition related processing and provides recognition result data to the application. It provides the following functions:

Body recognition from depth camera image
Display of Body recognition result
Body gesture recognition

AR Foundation Human Stencil / Human Depth

Gets Human Stencil data and Human Depth data from AR Foundation and create mask texture for iOS.

HumanDetector

Creates a mask texture by estimating the human parts in the Color camera image.

Face

The Face component performs Face recognition related processing and provides recognition result data to the application. It mainly provides the following functions:

Output Face recognition data and BlendShape data in an AR Foundation compliant format
Display Face recognition results
Display line of sight recognition results

Segmentation

The Segmentation component performs Segmentation recognition related processing and provides recognition result data to the application. It mainly provides the following functions:

Create a mask texture by estimating the sky section from the Color camera image (When using SkyDetector).
Create a mask texture by estimating the person section from the Color camera image (When using HumanDetector).

SkyDetector

Creates a mask texture by estimating the sky section from the Color camera image.

How does ToF AR suit your needs?

The ToF AR is built to built for high level of access to platform-specific features to facilitate better performance and user experience. In the chart below you can see the components ToF AR access on other platforms as well as natively.

SDK	ARCore	ARKit	ToF AR
SDK Developer	ARCore Google	ARKit Apple	ToF AR Sony Semiconductor Solutions Group
SDK Platform	ARCore Android	ARKit iOS	ToF AR Android, iOS
SDK SLAM	ARCore ✓	ARKit ✓	ToF AR
SDK Plane Detection	ARCore ✓	ARKit ✓	ToF AR ✓
SDK Light estimation	ARCore ✓	ARKit ✓	ToF AR
SDK Face tracking	ARCore ✓	ARKit ✓	ToF AR
SDK 2D image tracking	ARCore ✓	ARKit ✓	ToF AR
SDK 3D object tracking	ARCore	ARKit ✓	ToF AR
SDK Meshing	ARCore	ARKit ✓	ToF AR ✓
SDK 2D & 3D body tracking	ARCore	ARKit ✓	ToF AR ✓
SDK Human segmentation	ARCore	ARKit ✓	ToF AR ✓
SDK Access depth map	ARCore	ARKit ✓ (Fused)	ToF AR ✓
SDK 3D hand joint tracking	ARCore	ARKit	ToF AR ✓
SDK Hand pose recognition	ARCore	ARKit	ToF AR ✓
SDK Hand gesture detection	ARCore	ARKit	ToF AR ✓
SDK	ARCore	ARKit	ToF AR
SDK ✓ = Verified from each developer's publicly accessible information.	ARCore	ARKit	ToF AR

Get started

To start developing with ToF AR, you only need to install the toolkit and see the guides and you're good to go.

And don't forget to sign up for the Newsletter to get the latest information direct into your inbox.

ToF (Time of Flight) AR