Build the immersive world with ToF AR.Build the immersive world with ToF AR.

ToF (Time of Flight) AR

Build the immersive world with  ToF AR – an advanced toolkit for efficient AR app development

ToF AR components

Here is an overview of the components provided by ToF AR. More in-depth descriptions of the components and data formats can be found in the ToF AR reference articles in the development guide.

Tofar 220315 Tof ImageTofar 220315 Tof Image

ToF

The ToF component provides access to the depth camera’s Depth data, Confidence data and PointCloud data. The features provided are:

  • Data retrieval
  • Data visualization
  • Controlling the Depth camera (frame rate, shutter speed, and so on)
ToF AR color componentToF AR color component

Color

The color component provides access to the RGB camera data. The features provided are:

  • Data retrieval
  • Data visualization
  • Controlling the RGB camera (resolution, auto-focus, auto-exposure, white-balance, and so on)
Tofar 220315 Plane ImageTofar 220315 Plane Image

Plane

The plane component recognizes a plane in real time for any given point inside the Depth camera image. The features provided are:

  • Plane data retrieval
  • Dynamic updating of an object inside a Unity scene.

Can get/set up to eight detection configs at once. Plane detection stops executing if null or an empty config list is set.

Tofar 220315 Mesh ImageTofar 220315 Mesh Image

Mesh

The mesh component generates a 3D mesh from the Depth camera image in real time. The features provided are:

  • Vertex or triangle data retrieval
  • Dynamic updating of a mesh object inside a Unity scene.
  • Mesh reduction control
  • Mesh generation excluding the mask part generated from the Segmentation component
Coordinate componentCoordinate component

Coordinate

The coordinate component provides a means to convert between the different coordinate systems of the Depth and RGB cameras. It is mainly used for retrieving converted coordinate data through getting component properties.

Hand componentHand component

Hand

The hand component recognizes in real time the location of the joints, pose and gesture of the hand and fingers by utilizing the depth camera data.

MarkRecog component.MarkRecog component.

MarkRecog

The MarkRecog component recognizes optional marks located inside binary images. MarkRecog primarily supports the following function:

  • Recognition of mark located inside image.

Modeling

The Modeling component accumulates depth data of depth cameras over multiple frames and generates 3D Mesh data. It mainly provides the following functions:

  • Start/end 3D modeling process
  • Set 3D modeling parameters
  • Output data
  • Mesh generation excluding the mask part generated from the Segmentation component
Modeling componentModeling component

ARFoundationConnector

By using ARFoundationConnector, it's possible to use ARFoundation and ToF AR in the same UnityProject.

As ToF AR currently does not control the lower layer of the camera directly it instead uses the output data frames of the RGB or ToF camera obtained by ARFoundation to operate the Hand and Body recognition processes. Therefore, app developers can receive data frames from ToF AR with the same interface whether or not ARFoundation is present.

ToF AR Body pose skeletonToF AR Body pose skeleton

Body

The Body component performs Body recognition related processing and provides recognition result data to the application. It provides the following functions:

  • Body recognition from depth camera image
  • Display of Body recognition result
  • Body gesture recognition

AR Foundation Human Stencil / Human Depth

Gets Human Stencil data and Human Depth data from AR Foundation and create mask texture for iOS.

HumanDetector

Creates a mask texture by estimating the human parts in the Color camera image.

Face componentFace component

Face

The Face component performs Face recognition related processing and provides recognition result data to the application. It mainly provides the following functions:

  • Output Face recognition data and BlendShape data in an AR Foundation compliant format
  • Display Face recognition results
  • Display line of sight recognition results
Segmentation componentSegmentation component

Segmentation

The Segmentation component performs Segmentation recognition related processing and provides recognition result data to the application. It mainly provides the following functions:

  • Create a mask texture by estimating the sky section from the Color camera image (When using SkyDetector).
  • Create a mask texture by estimating the person section from the Color camera image (When using HumanDetector).

SkyDetector

Creates a mask texture by estimating the sky section from the Color camera image.

How does ToF AR suit your needs?

The ToF AR is built to built for high level of access to platform-specific features to facilitate better performance and user experience. In the chart below you can see the components ToF AR access on other platform as well as natively.

SDK

ARCore*

ARKit*

ToF AR

SDK

Publisher

ARCore*

Google

ARKit*

Apple

ToF AR

Sony Semiconductor Solutions

SDK

Platform

ARCore*

Android

ARKit*

iOS

ToF AR

Android, iOS

SDK

SLAM

ARCore*

ARKit*

ToF AR


SDK

Plane Detection

ARCore*

ARKit*

ToF AR

SDK

Light estimation

ARCore*

ARKit*

ToF AR


SDK

Face tracking

ARCore*

ARKit*

ToF AR


SDK

2D image tracking

ARCore*

ARKit*

ToF AR


SDK

3D object tracking

ARCore*


ARKit*

ToF AR


SDK

Meshing

ARCore*


ARKit*

ToF AR

SDK

2D & 3D body tracking

ARCore*


ARKit*

ToF AR

SDK

Human segmentation

ARCore*


ARKit*

ToF AR

SDK

Access depth map

ARCore*


ARKit*

✓ (Fused)

ToF AR

SDK

3D hand joint tracking

ARCore*


ARKit*


ToF AR

SDK

Hand pose recognition

ARCore*


ARKit*


ToF AR

SDK

Hand gesture detection

ARCore*


ARKit*


ToF AR

* Via Unity's AR Foundation

Data from respective publisher's official information as of 08 February 2023.

Get started

To start developing with ToF AR, you only need to install the toolkit and see the guides and you're good to go.

And don't forget to sign up for the Newsletter to get the latest information direct into your inbox.