How to create your first SmartEyeglass project

In this tutorial, you will learn how to develop your own SmartEyeglass app. It explains what basic steps you need to take for every app, describes the samples that are provided with the SmartEyeglass SDK, and shows how to use a sample as a template that you can modify or expand to add your own functionality.

The SmartEyeglass SDK allows you to create apps for the SmartEyeglass prototype concept. You can learn how to get information about the user’s environment from the built-in sensors, control the built-in camera, and take advantage of the unique display style. For example:

  • You could detect a user’s head movement, and use that information to do things like take a photo when the head tilts forward twice, or share the last photo when you detect a nodding gesture.
  • You could display instant translations as subtitles, rather than the traditional audio translations.
  • You could use the voice-input capability to create an app that allows the user to have both hands free.
  • You could display sports-related information, such as distance to the flag and wind direction for golf, or current time and distance to the finish line for marathon runners.

This early release will give you a chance to get familiar with the features and API usage, so you can plan your apps and get a head start on development.

Creating apps for the SmartEyeglass SDK is similar to creating apps for SmartWatch 2, as both these products build on the SmartExtension API framework. You will use the SmartExtension API in your app, and also the new SmartEyeglass API that extends that framework to support the SmartEyeglass technology. This allows you to develop new apps for SmartEyeglass, or extend your existing SmartWatch2 app to add support for SmartEyeglass.

As for the SmartWatch 2, your app and the control framework run on a host Android device (a smartphone), which communicates with the device itself (the SmartEyeglass eyewear and its wired controller) over a Bluetooth or WLAN connection. For more information on the software architecture of SmartEyeglass, check out the SmartEyeglass API overview within the Develop section.

The basics

The SmartEyeglass SDK comes with full code examples, an emulator for testing, API references, and you’ll find a whole bunch of different developer Guides to aid you in your development. All this is available in the SmartEyeglass develop section.

Now to get started with the SmartEyeglass SDK, you first have to:

  • Read the API overview to get an idea of what you can do with the SmartEyeglass SDK, and how it fits into the Smart Extensions framework for Sony accessories.
  • Follow the instructions in Get started for installing the SmartEyeglass SDK and setting up your development and test environment. Install the SmartEyeglass SDK using the Android SDK Manager. You can use Eclipse or IntelliJ IDEA/Android Studio as your development environment.
  • The SmartEyeglass SDK includes the required libraries for both the SmartExtensions APIs and the SmartEyeglass APIs.  Import the required libraries into your IDE workspace. See specific instructions for each IDE in Get started. These libraries are required for all apps:
    SmartExtensionAPI
    SmartExtensionUtils
    SmartEyeglassAPI

    Important: Import the source code for these libraries into your project. Compiled JAR files are not provided.
  • The SmartEyeglass SDK provides a set of sample projects that demonstrate basic usage for each of the device features.  Import one or more sample projects to use as a base for your app. The sample projects are found in the SDK_root/samples/ folder.

Step 1: Include an Android manifest

A SmartEyeglass project is an Android extension, which requires an Android manifest. Every sample project includes a manifest, which you can use or modify. This file, AndroidManifest.xml, tells the operating system about your app. This is where you:

  • Register an ExtensionReceiver
  • Define Intent filters
  • Assign a PreferenceActivity
  • Request permissions

You must request these permissions in order to use the related SmartEyeglass features:

  • SmartEyeglass Camera  API:
  • SmartEyeglass Voice Text Input API:
  • Framework and all other SmartEyeglass APIs:
    
    

Step 2: Register your app with the host device

Every app must register basic information about its API usage and capabilities with the host application on the host device. In each of the sample projects, you can see how the SampleRegistrationInformation class extends RegistrationInformation in the SmartExtensionsAPI library.

For each API that you use, you must override the getRequired[XX]ApiVersion() method to specify which version of the API you need. The version of each API that supports particular SmartEyeglass functionality is listed in the API references documentation. For example:

@Override
public int getRequiredControlApiVersion() {
return 4;
}

You also use your registration class to associate your app extension with the configuration details that you provide, such as a display name and icon, what display sizes you support, and an optional preference activity.  For more information about this, see the Registration and capabilities guide.

Step 3: Extend the base classes you will need

Each sample extension creates one or more classes that define the logic for a particular feature. You typically extend the ControlExtensionclass in order to define and show layouts for the display, create event listeners, and define handlers for controller events and data returned from the device sensors.  For example, in the SampleCameraExtension project, the methods that handle camera-related events are defined in the class SampleCameraControl.

  1. Each sample that creates a Layout for the SmartEyeglass display creates a class that extends ControlExtension, and instantiates a SmartEyeglassControlUtils member inside that class. For example:
    class SampleDisplaySettingControl extends ControlExtension {
        private final SmartEyeglassControlUtils mSmartEyeglassControlUtils;
        . . .
  2. The constructor initializes the SmartEyeglassControlUtils member. For example:
    public SampleDisplaySettingControl(final Context context, final String hostAppPackageName) {
        super(context, hostAppPackageName);
        this.context = context;
        size = new ScreenSize(context);
        utils = new SmartEyeglassControlUtils(hostAppPackageName, null);
        utils.activate(context);
    }
    

Most of the specific SmartEyeglass feature-control functionality is available through methods in SmartEyeglassControlUtils.

Notifications do not require any extension of the Control API; they are implemented using the Android ContentProvider, and most of the functionality is provided at the system level. The sample project extends the SmartExtensionAPI class ExtensionService in order to configure how you want to present notifications that are received from the sources that you register for. See the Notifications guide for more information.

Step 4: Build a user interface

The SmartEyeglass has unique UI features and requirements because of the binocular see-through display. The user wears it, and the display appears overlaid on the real-world view seen through the transparent screen; the user does not choose to look down at it. Because of this, you must be sure to read the Design guidelines for important safety consideration before you design your app’s UI.

The top level of the SmartEyeglass UI is a card layer, which acts as the main menu. Each card is the entry point for an application. You can use the Widget API to design your own app’s entry point card. See the Widgets guide for more information.

The rest of your API is defined by successive layers of display screens defined by Android XML-based Layouts, or directly by bitmaps. The HelloLayouts sample provides a good starting point for how you can define your first app-specific screen.

  1. Create an XML Layout definition for a screen that fits the display size of the SmartEyeglass device
  2. Create a data Bundle with run-time data to use in the Layout.
  3. In your class that extends the ControlExtension class, override the SmartEyeglassControlUtils.showLayout() method to display your layout.

See the User interface guide for a more complete example, and more information on when to use bitmaps rather than layouts.

Step 5: Add event handlers

Once you have defined the screens you want to display, you need to define when to display them. There are two ways to add this and other behaviors to your app:

  • Override one of the built-in event callbacks in your control class.
  • Create an event listener and define handler callbacks for specific events you are interested in.

Your app can respond to user-interaction events that it receives from user gestures, such as a button press, or a tap or swipe on the controller’s touch sensor.

The wired controller of the SmartEyeglass prototype.

The wired controller of the SmartEyeglass prototype.

Override a built-in event callback

The base ControlExtension class defines default behaviors for various user-interaction events, such as onTap() or onKey(). For example, when your entry-point card is shown, a tap on the touch sensor runs your app, and pressing the Back button goes back up a layer in the UI. You can override these methods in your class that extends ControlExtension.

For examples of how your app can override the default behaviors for these events, see the User interface guide, and the HelloLayout sample project.

Register event handlers with an event listener

When you modify a sample to provide your own functionality, you will typically provide your own event listener and handlers for the events that are of interest in your application. Do this by extending the class SmartEyeglassEventListener. Your class specifies which events to respond to, and registers a handler callback function for each one.

In this example from the Camera sample, mSmartEyeglassEventListener is an instance of a class that extends the base class to implements event call-backs for camera-related events.

private final SmartEyeglassEventListener mSmartEyeglassEventListener = new MySmartEyeglassEventListener();

class MySmartEyeglassEventListener extends SmartEyeglassEventListener {

     @Override
     public void onCameraReceived(CameraEvent event) {
          }

     @Override
     public void onCameraErrorReceived(int error) {
          }

     @Override
     public void onCameraReceivedFile(String filePath) {
          }

     @Override
     public void onRecordingStatus(long timeInMs, long frames) {
          }

}

For you own application, choose which events you want to listen to. For the full list, see the reference documentation for the SmartEyeglassEventListener class.

Step 6: Test your app

Once you’ve created your SmartEyeglass app, the easiest way for you to test is by using an emulator that runs on the host Android device. For details, see Test and verify.  If you are a partner with access to an actual SmartEyeglass prototype device, you can Test using hardware.

Your app is now ready! When publishing your app on Google Play, remember to include the string “SmartEyeglass” in the description, to make sure that it shows up in the search when Google Play looks for apps that are available for this device.

***

Feel free to drop us a comment below with any questions or suggestions about SmartEyeglass development!

More information

Comments 0

Sort by:

Showing 0 of 0 comments. Show all comments