Voice-to-text input

The SmartEyeglass controller has a microphone that you can use to get voice input from the user. To start voice input, users press the Talk button on the controller.

Pressing the Talk button triggers the predefined voice-input screen.

Pressing the Talk button triggers the predefined voice-input screen.

IMPORTANT: While the Talk button is pressed, the system interrupts the screen display to run voice input operation. When this operation is running, the extension has no control and cannot receive user input events from the controller.

The basics

To get started working with voice input, set up a project using the SampleVoiceTextInputExtension; see more about creating SmartEyeglass projects in the How to create your first SmartEyeglass project tutorial.

When designing an application to use the voice text input function, keep in mind that voice input may be hard to interpret correctly when there is ambient noise or wind. We recommend that you suggest an alternative input method to users when outdoors or in public places. For more about UI design, see Design guidelines.

Register for API usage

You will use the voice-text-input API, defined in SmartEyeglassControlUtils. In order to use the methods that invoke speech recognition you must declare this specific permission in your Android manifest. In the AndroidManifest.xml file in your project, make sure you include this element:

<uses-permission android:name="com.sony.smarteyeglass.permission.VOICE_TEXT_INPUT" />

You must also register the Control API version with the HostApp. Your class that extends RegistrationInformation must get the version of this API:

@Override
public int getRequiredControlApiVersion() {
    return 4;
}

The SampleVoiceTextInputExtension project does both of these things. If you don’t use this sample as a base, make sure you do the same.

How voice input works

While the Talk button is pressed, the user’s voice is interpreted continuously. The screen shows the microphone icon, and a text box that displays the text as it is recognized.

Voice input operation active.

Voice input operation active.

The user releases the Talk button to end the voice input operation. This displays a confirmation screen that allows the user to accept, re-input, or edits the resulting text.

Voice input confirmation.

Voice input confirmation.

The voice input functionality is implemented using the Android SpeechRecognizer API. The speech recognition operation streams audio to remote servers over Bluetooth.

  • Speech recognition is resource intensive, and works best with a good internet connection.
  • It is recommended that you do not design for continuous use of this feature, which would consume a significant amount of battery and bandwidth.
  • Voice input streams audio over Bluetooth, so it works only in Normal Power mode. Do not use it in High Performance mode, which switches to a Wireless LAN connection. See the Settings guide for more information on power modes.

Get voice input

The SampleVoiceTextInputExtension demonstrates this basic procedure for using voice text input in your application.

  1. Create a class (such as SampleVoiceTextInputControl) that extends the ControlExtension class and instantiates a member of SmartEyeglassControlUtils.
    class SampleVoiceTextInputControl extends ControlExtension {
        private final SmartEyeglassControlUtils mSmartEyeglassControlUtils;
    
  2. Create a listener for the SmartEyeglass device, and define a callback handler method for the onVoiceTextInput event. The completed voice-input operation sends this event with the text and an error code VOICE_TEXT_INPUT_RESULT_OK (0) on success.
    private final SmartEyeglassEventListener mSmartEyeglassEventListener = new SmartEyeglassEventListener() {
        @Override
        public void onVoiceTextInput(int errorcode, String text) {
            if (errorcode == VOICE_TEXT_INPUT_RESULT_OK0) {
                //Successful. Use acquired text.
            } else {
                //Handle the error.
            }
        }
    };
    
  3. In the constructor method, initialize the SmartEyeglassControlUtils member with your listener, than create the display elements and event handlers.
    SampleVoiceTextInputControl(final String hostAppPackageName, final Context context) {
        super(context, hostAppPackageName);
        this.mSmartEyeglassControlUtils =
    new SmartEyeglassControlUtils(context, hostAppPackageName, mSmartEyeglassEventListener);
        // Create something
    }
    
  4. To show the microphone icon on the SmartEyeglass display, call this method:
    mSmartEyeglassControlUtils.enableVoiceTextInput();
    

    Microphone icon shown on the display.

    Microphone icon shown on the display.

  5. To start the operation, the user must press the Talk button. We recommend that you supply a short instruction on how to use voice input when the user runs your application for the first time.
  6. When you no longer need input, or before closing your application, disable Voice Text Input mode.
    mSmartEyeglassControlUtils.disableVoiceTextInput();
    

    This removes the microphone icon from the display.

Learn from the code example

For a more complete example of how to use voice text input, see the SampleVoiceTextInputExtension sample project in the /samples folder of the SmartEyeglass SDK.

Comments 0

Sort by:

Showing 0 of 0 comments. Show all comments