SmartEyeglass HeroSmartEyeglass Hero

Develop customized AR solutions

Sony’s augmented reality SmartEyeglass* is developed for increasing business productivity and reducing errors. Transform the way you work by enhancing reality with information and graphics.

*Discontinued model.

User interface


  • Use bitmaps or XML-based layouts to define a UI for your app extension.
  • Define handlers for user-interaction events such as key-press and tap or swipe on the touch sensor.


Key Classes

ControlExtension RegistrationInformation SmartEyeglassControlUtils

The user interface for your SmartEyeglass app consists of a card in the main menu that is the entry-point to your app, and a set of additional screen displays that you define.

The Control API contained in SmartEyeglassControlUtils allows you to access and control the SmartEyeglass display. Instantiate this class in your class that extends ControlExtension. This page describes how to use the Control API to draw bitmaps directly to the display, or to display Android XML-based layouts. Defining the layout for a card or screen in your app is like defining an Activity layout in Android.

For details of the UI structure, see Design guidelines. Be sure to check these guidelines for important UI design recommendations and requirements that are specific to the SmartEyeglass device.

The basics

Set up a project using the sample HelloLayouts as a base; see more about creating SmartEyeglass projects in the How to create your first SmartEyeglass project tutorial.

Extend the Control API

The Control API provides the tools you need for interacting with the user, with methods for rendering images, handling touch events and key presses, and so on. Your application must override methods in the base classes in order to:

  • Show your layout or bitmap on the display.
  • Handle user-input events.

Register a version

For any app that renders bitmaps or layouts on the SmartEyeglass display, you must specify which version of the Control API (defined in SmartEyeglassControlUtils) you are using:

  • Version 1.0 of the Control API allows you to render images on your target accessory’s display using bitmaps.
  • Version 2.0 extends the class with support for rendering an XML-based layout.
  • Version 4.0 extends the class with further support of SmartEyeglass features.

The SmartEyeglass supports all of these versions. In general, it is recommended that you use layouts (version 4) for a SmartEyeglass app, rather than bitmaps; the code is generally cleaner and less complex. However, if your app must be backwards compatible with v1.0, you might want to use only bitmaps, so that you don’t need to handle your UI logic in two different ways.

Your class that extends RegistrationInformation must specify the version of the Control API that supports the specific functionality you are using:

public int getRequiredControlApiVersion() {
// if using only bitmaps, you can use version 1
// if using layouts, require at least version 2
// use version 4 for most SmartEyeglass functionality
     return 4;
The HelloLayouts project does this. If you don’t use this sample as a base, make sure you do the same.

Show your UI on the display

There are two ways to show a UI for accessories that support Control API v2.0 (such as Smart Eyeglass):

  • Use the method ControlExtension.showLayout() to display a layout that you have included in your project.
  • Use the method SmartEyeglassControlUtils.showBitmap() to display a bitmap that you have included in your project. (Note that this version of the method specializes the one defined in ControlExtension to optimize for a monochrome display.)

Define and display a layout

To display a layout that you have defined, pass the layout ID and a Bundle containing the layout data to the showLayout() method. For example:

  1. showLayout(R.layout.sample_control_2, data);

The data bundle can contain things to be set at run time, such as Strings to populate the TextViews:

Bundle b1 = new Bundle();
b1.putString(Control.Intents.EXTRA_TEXT, "1");

Here’s a more complete example:

Bundle b1 = new Bundle();
b1.putString(Control.Intents.EXTRA_TEXT, "1");
Bundle b2 = new Bundle();
b2.putString(Control.Intents.EXTRA_TEXT, "2");
Bundle b3 = new Bundle();
b3.putString(Control.Intents.EXTRA_TEXT, "3");
Bundle b4 = new Bundle();
b4.putString(Control.Intents.EXTRA_TEXT, "4");
Bundle[] data = new Bundle[4];
data[0] = b1;
data[1] = b2;
data[2] = b3;
data[3] = b4;
showLayout(R.layout.sample_control_2, data);

Define an XML layout for the SmartEyeglass

A SmartEyeglass layout is very similar to a standard Android layout, with these limitations:

  • Only a subset of the standard Android ViewGroups and Views are supported. For details see EXTRA_DATA_XML_LAYOUT.
  • All dimensions must be specified as absolute pixels (px); you cannot use density- independent pixels (dp). This is because the screen size is fixed, but the screen size and density of the phone that your app is running on can vary. Since the app itself runs on the host Android device and not on the SmartEyeglass device, if you were to use density-independent pixels, the layout would try to scale based on the screen values of the host device. The resulting SmartEyeglass display would then vary according to the host device.
  • For text sizes, you can use scaled pixels (sp), so that text size can vary according to the user preference settings on the accessory.

Here is a code snippet from a Control extension that shows a layout definition for the SmartEyeglass display.

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="" xmlns:tools="" android:layout_width="@dimen/smarteyeglass_control_width" android:layout_height="@dimen/smarteyeglass_control_height" android:background="@android:color/black" tools:ignore="ContentDescription,PxUsage" >
<ImageView android:id="@+id/image" android:layout_width="48px" android:layout_height="48px" android:layout_alignParentLeft="true" android:layout_alignParentTop="true" android:layout_margin="6px" android:background="@drawable/shape" android:src="@drawable/icon_extension48" />
<TextView android:id="@+id/btn_title_bitmap" android:layout_width="match_parent" android:layout_height="wrap_content" android:layout_toRightOf="@+id/image" android:background="@android:color/black" android:layout_marginBottom="6px" android:layout_marginLeft="60px" android:gravity="center" android:text="@string/text_title_layout" android:textColor="@android:color/white" android:textSize="@dimen/smart_eyeglass_text_size_medium" android:textStyle="bold" />
. . .
This is how the layout appears on the SmartEyeglass emulator:

SmartEyeglass emulator shows a layout you have defined.

Define and display a bitmap

To render a bitmap directly on the accessory display, define the bitmap and pass it to SmartEyeglassControlUtils.showBitmap().

Note: Because the SmartEyeglass has a monochrome display, be sure to use SmartEyeglassControlUtils.showBitmap(), rather than ControlExtension.showBitmap(). The SmartEyeglass version of the method automatically converts your bitmap to monochrome.

Here is an example:

// Create background bitmap for animation.</p>
<p>mBackground = Bitmap.createBitmap(width, height, BITMAP_CONFIG);
// Set default density to avoid scaling.
LinearLayout root = new LinearLayout(mContext);
root.setLayoutParams(new LayoutParams(width, height));
LinearLayout sampleLayout = (LinearLayout)LinearLayout.inflate(mContext, R.layout.sample_control, root);
sampleLayout.measure(width, height);
sampleLayout.layout(0, 0, sampleLayout.getMeasuredWidth(), sampleLayout.getMeasuredHeight());
Canvas canvas = new Canvas(mBackground);

Display images with callback

Sometimes you want to display images as fast as possible, in order to provide animations for your user interface, for example, or to keep the display updated with the latest available information. For cases like this, you can use  these functions that notify your callback when the drawing operation is completed:

SmartEyeglassControlUtils.showImageWithCallback (int resourceId,
int transactionNumber)
SmartEyeglassControlUtils.showBitmapWithCallback (Bitmap bitmap, final int transactionNumber)
SmartEyeglassControlUtils.showBitmapWithCallback (Bitmap bitmap, int x, int y, int transactionNumber) 

When the asynchronous drawing operation is complete, the result is returned to your callback.

SmartEyeglassEventListener.onResultShowImage (int transactionNumber, int result)
SmartEyeglassEventListener.onResultShowBitmap (int transactionNumber, int result)

The transaction number that you supplied in the show-image or show-bitmap call is passed back to the callback, so that your handler can match a specific operation to its result. The result value that is passed to your callback indicates the success or failure status of the operation; see SmartEyeglassControl.Intents.EXTRA_DISPLAY_DATA_RESULT in the API reference.

Handle user-interaction events

You can define handlers for these user-interaction events:

  • Tap or touch on touch sensor
  • Swipe on touch sensor
  • Button press on one of the hard keys

See the API references for specific Intents, actions, and keycode constants.

Tap and touch events

To handle a single tap on the touch sensor, override the onTap() method in your Control extension class. For example:

public void onTap(final int action, final long timeStamp) {
// define behavior for a single tap

SmartEyeglass supports only the TapActions.SINGLE_TAP action.

There is also an onTouch() method that is triggered by various more specific touch actions. Touch actions include:


For example:

public void onTouch(final ControlTouchEvent event) {
if (event.getAction() != Control.Intents.TOUCH_ACTION_LONGPRESS){
//define behavior for long press on touch pad

Swipe events

To handle a swipe on the touch sensor, override the onSwipe() method in your Control extension class. For example:

public void onSwipe(int direction) {
     if (direction == Control.Intents.SWIPE_DIRECTION_LEFT){
     // handle action for left swipe here
     } else if (direction == Control.Intents.SWIPE_DIRECTION_RIGHT) {
     // handle action for right swipe here

Key-press events

To intercept the built-in key-press actions, override the onKey() method, and identify the specific action and key code of interest.

Actions include:


Key codes include:


For example, this defines a response the user pressing and releasing the Back button on the controller:

public void onKey(int action, int keyCode, long timeStamp) {
      if (action == Control.Intents.KEY_ACTION_RELEASE && keyCode == Control.KeyCodes.KEYCODE_BACK) {
     // handle back key action

Learn from the code example

For the full code example showing the topics discussed here, see the HelloLayouts sample project in the SmartEyeglass SDK.

Add layer transitions

The AdvancedLayouts sample shows examples of simple interfaces with three or four layers. The transitions between layers in the samples use default gestures and styles that are handled automatically by the framework; for example, pressing the Back button takes a user up a layer.

The Control API provides utility methods for programmatic transition between layers defined by layouts. These methods use zoom-in animation to indicate a transition to a lower layer, and zoom-out animation for a transition to a higher layer.

Implement layer transitions

The SmartEyeglassControlUtils class provides methods that allow you to move between layers that are defined by layouts:

  • Use moveLowerLayer() to display the next layer down, using a “zoom in” animation for the transition from the current layer:
    1. mSmartEyeglassControlUtils.moveLowerLayer(final int layoutId, final Bundle[] layoutData);
  • Use moveUpperLayer() to display the next layer up, using a “zoom out” animation for the transition from the current layer:
    1. mSmartEyeglassControlUtils.moveUpperLayer(final int layoutId, final Bundle[] layoutData);

Each method takes two arguments, the unique ID of the destination layer, and the actual layout data bundle that defines the destination screen.

Your program must keep track of which layer is currently displayed, and which is next in the navigation order. The AdvancedLayouts sample shows how to create an index/counter that defines the relative positions of each layer, and keep it updated as transitions are performed.

Design for layer transitions

When designing your layer transitions, you must consider how you want your users to navigate through your UI, by using tap and swipe gestures on the touch sensor and by pressing the Back button on the controller.

It’s a good idea to create an outline structure of your implementation. For an app that requires a dynamic list of entries, we recommend using a layered structure as shown here. In this design, the user moves down one layer with each tap event and up one layer with each Back button press.

Layer transition structure.

Learn from the code example

For the full code example showing the topics discussed here, see the AdvancedLayouts sample project in the SmartEyeglass SDK.