SmartEyeglass HeroSmartEyeglass Hero

Develop customized AR solutions

Sony’s augmented reality SmartEyeglass* is developed for increasing business productivity and reducing errors. Transform the way you work by enhancing reality with information and graphics.

*Discontinued model.


The SmartEyeglass device provides a built-in camera which enables users to take still pictures and record videos from the user’s point of view.

The camera sensor data is available through the SmartEyeglass API. Your app can use the camera raw data for image processing, or to store still pictures or JPEG streams on the host device.

This guide tells you how to:

  • Listen to camera events.
  • Set the camera mode, quality and resolution.
  • Trigger the camera programmatically to capture still pictures or to stream JPEG images.

The basics

To get started working with the camera, set up a project using the SampleCameraExtension as a base; see more about creating SmartEyeglass projects in the How to create your first SmartEyeglass project tutorial.

Register for Camera API usage

You will use the Camera API, defined in the SmartEyeglassControlUtils class, to control the camera. In order to use this API, you must declare this specific permission in your Android manifest. In the AndroidManifest.xml file in your project, make sure you include this element:

You must also register the Control API version with the HostApp. Your class that extends RegistrationInformation must get the version of this API:

  1. @Override
  2. public int getRequiredControlApiVersion() {
  3. return 4;
  4. }

The SampleCameraExtension project does both of these things. If you don’t use this sample as a base, make sure you do the same.

Listen to camera events

Before programmatically triggering the camera shutter you need to start listening to various camera events, which you will receive after a picture has been captured. To do this, you need to instantiate SmartEyeglassControlUtils in your ControlExtension implementation:

  1. class SampleCameraControl extends ControlExtension {
  3. private final Context mContext;
  4. private final SmartEyeglassControlUtils mSmartEyeglassControlUtils;
  6. SampleCameraControl(final String hostAppPackageName, final Context context) {
  7. super(context, hostAppPackageName);
  8. this.mContext = context;
  9. this.mSmartEyeglassControlUtils = new SmartEyeglassControlUtils(
  10. context, hostAppPackageName, mSmartEyeglassEventListener);
  11. }
  12. }

The mSmartEyeglassEventListener is an instance of a class that implements the event callbacks. Choose which events you want to listen to, and define handlers for them. For the full list, see documentation of the SmartEyeglassEventListener class.

  1. private final SmartEyeglassEventListener mSmartEyeglassEventListener =
  2. new MySmartEyeglassEventListener();
  4. class MySmartEyeglassEventListener extends SmartEyeglassEventListener {
  5. @Override
  6. public void onCameraReceived(CameraEvent event) {
  7. }
  8. @Override
  9. public void onCameraErrorReceived(int error) {
  10. }
  11. @Override
  12. public void onCameraReceivedFile(String filePath) {
  13. }
  15. }

Set the camera mode

You must use the SmartEyeglassControlUtils.setCameraMode() method to set the mode before you do any other camera configuration. You must then complete all configuration operations (such as setting a file path) before actually taking a picture.

When setting the camera mode, you also specify the JPEG quality and a resolution that is appropriate to the mode.

  1. mSmartEyeglassControlUtils.setCameraMode(jpegQuality, resolution, mode);

  • One of Standard, Fine, or Superfine
resolutionSmartEyeglassControl.Intents.EXTRA_RESOLUTION constant.

  • For still images, 3M, 1M, VGA or QVGA
  • For image stream modes, QVGA.
modeSmartEyeglassControl.Intents.EXTRA_CAMERA_MODEconstant.These are the possible modes:

  • Still modesStillStill to file
  • Image stream modesJPEG stream low rateJPEG stream high rate

Image stream modes

The  JPEG stream modes are supported only when the High Performance power mode is on—that is, when there is a Wireless LAN connection between the SmartEyeglass device and its host Android device. For details of how to set the power mode, see the Settings guide. If you try to use the streaming JPEG camera modes over a Bluetooth connection, the resulting data is corrupted.

The low rate for image streaming is 7.5 FPS. The high rate is 15 FPS.

Take a still picture

After all setup and configuration is complete, you can trigger the camera shutter programmatically:

  1. mSmartEyeglassControlUtils.requestCameraCapture();

The result of this action depends on the mode you have set and the callback you have registered in your event listener.

Still mode: This code snippet illustrates how a callback can access and process image data sent to onCameraReceived callback when the camera mode is set to CAMERA_MODE_STILL.

  1. @Override
  2. public void onCameraReceived(CameraEvent event) {
  4. Bitmap bitmap = null;
  6. byte[] data = null;
  8. // Index indicates data type: 0 for picture data
  10. if(event.getIndex() == 0) {
  12. if((event.getData() != null) && ((event.getData().length) > 0)) {
  14. data = event.getData();
  16. bitmap = BitmapFactory.decodeByteArray(data, 0, data.length);
  18. // Process the bitmap...
  20. }
  22. }
  24. }

Still-to-file mode: When the camera mode is set to CAMERA_MODE_STILL_TO_FILE, the captured picture is directly stored to the host-device memory. Before taking a picture using this mode , you must configure a destination file path by calling StartCamera (filePath). When you have triggered the capture operation, your onCameraReceivedFile callback receives path of the captured image:

  1. @Override
  2. public void onCameraReceivedFile(String filePath) { … }

The picture is taken in color, but if you show it on the SmartEyeglass device, it is displayed in monochrome.

Start and stop image streaming

After all setup and configuration is complete, use the following functions to begin and end JPEG streaming programmatically:

  1. mSmartEyeglassControlUtils.startCamera([filePath]);
  2. mSmartEyeglassControlUtils.stopCamera();

If you specify a destination file path as an optional parameter in the start function, captured images are written directly to that file. If you omit the filePath parameter, the captured results are passed to your onCameraReceived callback when you stop the capture operation.

Learn from the code example

For a more complete example of how to use the SmartEyeglass camera, see the SampleCameraExtension sample project in the /samples folder of the SmartEyeglass SDK.