Spresense Header CodeSpresense Header Code

Spresense - 低功耗下的边缘计算

在干电池供电的Spresense板上体验正真的边缘计算

!
无中文翻译

Spresense SDK

1. Spresense SDK Getting Started Guide

This section describes how to start working with the Spresense SDK.

1.1. Prerequisite

You can develop Spresense SDK on Linux/Windows/macOS. Please setup development environment for your appropriate OS.

Supported Operation System

  • Linux (64bit)

    • Ubuntu 16.04 or later

  • Windows

    • 8.1/10

  • macOS

    • High Sierra (10.13) or later

1.1.1. Setup for Linux

  1. Serial Configuration

    Add user to dialout group

    $ sudo usermod -a -G dialout <user-name>

    Please logout and login after running the above command.

  2. Install development tools

    $ wget https://raw.githubusercontent.com/sonydevworld/spresense/master/install-tools.sh
    $ bash install-tools.sh

    And run this command to activate installed tools.

    $ source ~/spresenseenv/setup

    This command must run in every terminal window. If you want to skip this step, please add this command in your ${HOME}/.bashrc.

  3. Clone the Spresense SDK with submodule

    $ git clone --recursive https://github.com/sonydevworld/spresense.git

1.1.2. Setup for Windows

  1. Install MSYS2

  2. Install development tools

    Run MSYS2 MSYS from start menu, and run following commands.

    MSYS2 MinGW 64-bit and MSYS2 MinGW 32-bit is not supported. Please use MSYS2 MSYS.
    $ curl -L https://raw.githubusercontent.com/sonydevworld/spresense/master/install-tools.sh > install-tools.sh
    $ bash install-tools.sh

    And run this command to activate installed tools.

    $ source ~/spresenseenv/setup

    This command must run in every terminal window. If you want to skip this step, please add this command in your ${HOME}/.bashrc.

  3. Clone the Spresense SDK with submodule

    $ git clone --recursive https://github.com/sonydevworld/spresense.git
  4. Install USB serial driver

    User must install before connecting Spresense board to the PC.

    Please download and install from below.

1.1.3. Setup for macOS

  1. Install development tools provided by Apple

    Open the Teminal and run following command.

    $ xcode-select --install
  2. Install Python3

  3. Install development tools

    $ curl -L https://raw.githubusercontent.com/sonydevworld/spresense/master/install-tools.sh > install-tools.sh
    $ bash install-tools.sh

    And run this command to activate installed tools.

    $ source ~/spresenseenv/setup

    This command must run in every terminal window. If you want to skip this step, please add this command in your ${HOME}/.bash_profile.

  4. Clone the Spresense SDK with submodule

    $ git clone --recursive https://github.com/sonydevworld/spresense.git
  5. Install USB serial driver

    User must install before connecting Spresense board to the PC.

    Please download and install from below.

    On High Sierra (10.13) and later, the installation of the driver may be blocked. Please allow this driver from System Preferences Security & Privacy pane. See Apple Technical Note TN2459 "User-Approved Kernel Extension Loading" for more information.

1.2. USB connection

Connect the Spresense main board to the PC via the USB cable.

spresense musb connect
Figure 1. Main board USB connection

1.2.1. Check the serial port name

SDK tools needs the serial port name for communicate with Spresense board. The serial port name is different on each OS.

  • on Linux

    $ dmesg | grep "cp21.*attached"
    [12220.625979] usb 1-1: cp210x converter now attached to ttyUSB0

    In this case, the port name is /dev/ttyUSB0.

  • on macOS

    Use /dev/tty.SLAB_USBtoUART.

  • on Windows

    Open Settings > Devices, and Spresense board will be shown in Other devices list like Silicon Labs CP210x USB to UART Bridge (COM3). In this case, the port name is COM3. Alternatively, you can check it from Device Manager.

1.3. Setting Up the Spresense SDK

1.3.1. Build instructions

  1. Navigate to the folder where you have stored the cloned Spresense SDK code, and enter the sdk folder:

    $ cd spresense/sdk
  2. Set the initial NuttX kernel configuration:

    $ tools/config.py --kernel release
  3. Set the initial SDK configuration:

    $ tools/config.py examples/hello
  4. Build the example image:

    $ make buildkernel
    $ make

A nuttx.spk file will be created in the sdk folder when this step has successfully finished. This file is the final result and can be flashed onto the Spresense board. See Tutorial to flash SW for details.

1.4. Tutorial to flash SW

1.4.1. Flash the bootloader

The correct bootloader is required for the Spresense board to function.

Bootloader information
  • The bootloader has to be flashed the very first time the board is used.

  • You have to accept the End User License Agreement to be able to download and use the Spresense bootloader binary.

A WARNING message (example shown below) may be shown when using tools/config.py or tools/flash.sh if the bootloader is missing. In this case you have to flash the Spresense bootloader onto the Spresense board.

WARNING: New loader v1.0.003 is required, please download and install.
         Download URL   : https://developer.sony.com/file/download/spresense-binaries-vx.x.x.zip
         Install command:
                          1. Extract loader archive into host PC.
                             ./tools/flash.sh -e <download zip file>
                          2. Flash loader into Board.
                             ./tools/flash.sh -l /home/user/mySpresense/spresense/firmware/spresense -c <port>
  1. Download the EULA binary zip archive.

    Go to the Download URL in WARNING description which will provide a link to the specific version you need, and download it to your PC.

  2. Extract EULA binary zip in your PC by executing:

    $ ./tools/flash.sh -e spresense-binaries-vx.x.x.zip
  3. Flash Spresense bootloader into Spresense board.

    Execute the command that is shown in WARNING description. (In this example <port> is /dev/ttyUSB0):

    $ ./tools/flash.sh -l ../firmware/spresense -c /dev/ttyUSB0
  4. Finish.

    On installation completion, the Spresense board will reboot.

1.4.2. Flash the user nuttx.spk image

Use the tools/flash.sh script to flash nuttx.spk to Spresense board.

Type tools/flash.sh for flashing nuttx.spk into Spresense board.

$ tools/flash.sh -c /dev/ttyUSB0 nuttx.spk
>>> Install files ...
install -b 115200
Install nuttx.spk
|0%-----------------------------50%------------------------------100%|
######################################################################

252480 bytes loaded.
Package validation is OK.
Saving package to "nuttx"
updater# sync
updater# Restarting the board ...
reboot

On installation completion, the Spresense board will reboot.

1.5. Run "Hello, World!"

hello example is a simple program that it shows Hello world!! on the console. So you need serial terminal software. You can use any serial terminal software, screen and minicom for example.

Table 1. Serial Terminal Configuration

Baudrate

115200

Data Bits

8 bit

Parity Bit

none

Stop Bit

1 bit

Flow Control

none

In this chapter, use screen. If you use Linux, install first.

$ sudo apt install screen

Run terminal in 115200 baud

$ screen /dev/ttyUSB0 115200

If nsh> is shown after typing Enter and Esc keys, the Spresense board working properly. nsh> is the command prompt for NuttShell.

NuttShell (NSH)
nsh>

Type hello

NuttShell (NSH)
nsh> hello
Hello, World!!
nsh>
Spresense board will reboot every time you open the serial terminal.

2. Sample Application of GPS(GNSS)

This chapter shows the operation procedure of GPS(GNSS) sample application.

2.1. Build & Flash

  1. Move to the folder where you cloned the Spresense SDK, and enter the sdk folder name:

    $ cd spresense/sdk
  2. Set up the NuttX kernel configuration

    $ tools/config.py --kernel release
  3. Set up the SDK configuration To enable the gnss example application, select examples/gnss.

    $ tools/config.py examples/gnss
    Please refer to How to configure for details on the configuration.
  4. Build the example image:

    $ make buildkernel
    $ make
    Please refer to How to build for details on the build.

    A nuttx.spk file will be created in the sdk folder after make has successfully finished.

  5. Just the same as Hello World example, flash the nuttx.spk to Spresense with tools/flash.sh.

    $ tools/flash.sh -c /dev/ttyUSB0 nuttx.spk
  6. When flashing the board is completed the board is restarted automatically.

2.2. GPS operation confirmation

Loading nuttx.spk to Spresense, you can run the GNSS program.

Open the serial terminal.
$ minicom -D /dev/ttyUSB0 -b 115200 -s

Execute gnss command, the gnss is a built-in application. The following text will be displayed:

tutorial gnss log1
Figure 2. GNSS startup log

If positioning is not available, you see this message:

> No Positioning Data

And the time is displayed that is from 0 o’clock count up when GNSS start.
If the Spresense can receive GPS signals from the satellites (clear view to the sky etc), the time in UTC will be displayed in approximately 1 minute, and the GPS position in approximately 3 minutes.

> Hour:9, minute:13, sec:20, usec:559
> LAT 35.25.6303
> LNG 139.22.1986

Similar text as shown above is displayed, and latitude and longitude can be read.

3. Sample Application of Audio Player

This chapter shows the operation procedure of the sample application of Audio Player.

3.1. Build & Flash

  1. Move to the folder where you cloned the Spresense SDK, and enter the sdk folder name:

    $ cd spresense/sdk
  2. Set up the NuttX kernel configuration

    $ tools/config.py --kernel release
  3. Set up the SDK configuration To enable the audio_player example application, select examples/audio_player .

    $ tools/config.py examples/audio_player
    Please refer to How to configure for details on the configuration.
  4. Build the example image:

    $ make buildkernel
    $ make
    Please refer to How to build for details on the build.

    A nuttx.spk file will be created in the sdk folder after make has successfully finished.

  5. Just the same as Hello World example, flash the nuttx.spk to Spresense with tools/flash.sh .

    $ tools/flash.sh -c /dev/ttyUSB0 nuttx.spk
  6. When flashing the board is completed the board is restarted automatically.

  7. For Audio Player, it is necessary to load the DSP binary for decode. You can choose to place the DSP binary on either a SD card or SPI-Flash. Here is how to load from the SD card.

    Specify the path of DSP binary in the application code ( audio_player_main.cxx ). In audio_player_main.cxx , it is specified by DSPBIN_PATH .

    #define DSPBIN_PATH "/mnt/sd0/BIN"

    This code select SD card.

    If you want to use SPI-flash, please specify /mnt/spif/BIN . Also, if you specify null, it will be the default path set in Config.

    This /mnt/sd0/BIN becomes BIN/ under the root directory when it is read by the PC.
    Create this directory and place the DSP for the required codec here.

    When you want decode MP3 files,
    select MP3DEC under spresense/sdk/modules/audio/dsp/

    Write the music file you want to play simultaneously to the SD card. audio_player_main.cxx is specified in AUDIOFILE_ROOTPATH .

    #define AUDIOFILE_ROOTPATH "/mnt/sd0/AUDIO"

    Therefore, when read on the PC, under AUDIO/ under the root directory, please place the audio files you want to play. It can also be placed in subdirectories.

  8. About this sample

    The current Audio Player sample is playing a simple PlayList. So, specify the location and file name of the playlist file and play music files. In audio_player_main.cxx , specify the path with PLAYLISTFILE_PATH and the file name is specified in PLAY_LIST_NAME .

    #define PLAYLISTFILE_PATH "/mnt/sd0/PLAYLIST"
    #define PLAY_LIST_NAME "TRACK_DB.CSV"

    Therefor, when read on the PC, under PLAYLIST/ under the root directory, Create a file called TRACK_DB.CSV .

    For the contents of TRACK_DB.CSV , see README.txt under spresense/sdk/modules/audio/playlist/ .

You can play music by preparing all of them.

3.2. Operation check of Audio Player

When this nuttx.spk is loaded to the actual machine, the Audio Player program is executed.

Open the serial terminal as you did in the Hello sample.

$ minicom -D /dev/ttyUSB0 -b 115200 -s

When you run the player app that was builtin,

tutorial player log
Figure 3. The log of music playback.

The log is displayed and the audio is played back.

If an error occurs, refer to Error Information of Audio SubSystem.

4. Sample application of Audio Recorder

This chapter shows the operation procedure of the sample application of Audio Recorder.

See Audio Recorder Functions for code structure, data flow, API reference etc.

4.1. Build & Flash

  1. Move to the folder where you cloned the Spresense SDK, and enter the sdk folder name:

    $ cd spresense/sdk
  2. Set up the NuttX kernel configuration

    $ tools/config.py --kernel release
  3. Set up the SDK configuration

    1. To enable the audio_recorder example application, select examples/audio_recorder .

      $ tools/config.py examples/audio_recorder

      By doing this configuration, settings for audio_recorder will be made.
      Among them, the following items can be changed as needed. Please do not change the other configuration.

      [SDK audio]                    <= Y
        [Audio Recorder]             <= Y
          [DSP imange mount path]    <= /mnt/sd0/BIN  (1)
      1 This is the default location for DSP binary in SD card. If /mnd/spif/BIN is set, the default path is SPI-flash.
      Please refer to How to configure for details on the configuration.
    2. If you want to use your own signal processing for recording audio, set counfiguration of Preprocess as follow.

      In AudioRecorder sample, you can perform your own signal processing on the recorded audio.
      If you want to do this you need to enable [Use preprocess] in the config menu.

      Open the config menu.

      $ tools/cofig.py -m

      Check [Use preprocess] .

      [Examples]
        [Audio recorder example]
          [Use preprocess]        <= Y  (1)
      1 Enable Preprocess
      For more information on Preprocess , please refer to SDK Developer Guide Set preprocess.
  4. Do builds

    $ make buildkernel
    $ make
    Please refer to How to build for details on the build.

    A nuttx.spk file will be created in the sdk folder after make has successfully finished.

  5. Just the same as Hello World example, flash the nuttx.spk to Spresense with tools/flash.sh .

    $ tools/flash.sh -c /dev/ttyUSB0 nuttx.spk
  6. When flashing the board is completed the board is restarted automatically.

  7. For Audio Recorder, it is necessary to load the DSP binary for encoding. You can choose to place the DSP binary on either an SD card or SPI-Flash. Here is how to load from the SD card.

    Specify the path of DSP binary in the application code. In audio_recorder_main.cxx , it is specified by DSPBIN_PATH .

    #define DSPBIN_PATH "/mnt/sd0/BIN"

    This code selects the SD card.

    If you want to use SPI-flash, please specify /mnt/spif/BIN . Also, if you specify null, it will be the default path set in Config.

    This /mnt/sd0/BIN becomes BIN/ under the root directory when it is read by the PC.
    Create this directory and place the DSP for the required codec here.

    When you want to encode MP3 files,
    select MP3ENC under spresense/sdk/modules/audio/dsp/
    The combinations of Codec type and DSP binary for other encoding are shown in the table below.

    Codec DSP Binary

    MP3

    MP3ENC

    LPCM

    SRC

    Also, if Preprocess is used, place the Preprocess binaries on the SD card too.
    The binary is PREPROC under spresense/examples/audio_recorder/worker/src .

    In this sample application, PREPROC includes a simple RCfilter by default.
    If you want to customize your own signal processing etc., please refer to here.
  8. The recorded audio is put on the SD card. The recording path is also set by the macro value in the application code (audio_recorder_main.cxx).
    The macro on audio_recorder_main.cxx is RECFILE_ROOTPATH , please change the application code as needed.

    #define RECFILE_ROOTPATH "/mnt/sd0/REC"

    If it is this, it will be recorded on the SD card.

4.2. Operation check of Audio Recorder

When this nuttx.spk is loaded to the actual machine, the Audio Player program is executed.

Open the serial terminal as you did in the Hello sample.

$ minicom -D /dev/ttyUSB0 -b 115200 -s

When you run the recorder app that was builtin,

tutorial recorder log
Figure 4. The log of sound recording.

The log is displayed and the audio is recording.

Recorded audio can be played back on a PC. At that time, there is an audio file in REC/ under the SD card root directory.

If an error occurs, refer to Error Information of Audio SubSystem.

4.3. About customizing of DSP binary (PREPROC)

this chapter shows how to customize the DSP binary (PREPROC).

4.3.1. Step 1. Edit the code of PREPROC

Describes the code structure of PREPROC and the editing location.
The code is divided into two parts: the part to edit the user and the part provided as a framework.

user-edited code

It is a code that users should edit mainly.
It can make unique signal processing by editing these codes.+

The DSP code is in the worker directory, which has the userproc directory.
The user writes signal processing only in the userproc directory, and other things basically do not need to be changed.

Since main.cpp provides startup processing and data communication control with Main CPU, do not change it.

diag c218e3632dfa916b571321d164d9dfb5
Figure 5. The structure of source code
main.cpp

Startup processing and DSP communication processing are written. There is no need to edit.

userproc_command.h

It is a header file that defines the communication command with DSP.
Describe your necessary parameters in this file.

userproc.h

The header file of user code.

userproc.cpp

The sourace file of user code.
Write or call signal processing to this file.

APIs are provided for user code

userproc.cpp provides a framework for Init , Exec , Flush , Set commands.
The user code can support the processing in DSP by writing the unique contents.

Describes the process that the user should write.
(* By default, an RC filter is included as a sample.)

This framework assumed that the state transition inside DSP like in the figure below.

diag 47c757548935d399d55d4b644b03eeec
Figure 6. the state transition in DSP

Program the process by each command as following flow.

  1. DSP starts when AUDCMD_SETRECORDERSTATUS is called.

  2. Set necessary parameters (number of channels, bit length, etc.) with the Init command.

  3. When recording starts, the captured audio data is periodically sent to the DSP with the Exec command, so it can do a unique filter processing.

  4. If you want to change DSP internal parameters at any time, you can use Set command .The execution timing of this command is in the order of command reception including Exec .

  5. When recording stop, the Flush command is sent after the last audio data on Exec , so if termination processing is necessary, the processing is performed here.

Definition of command

The data types used by each function are described in userproc_command.h , and the contents can be freely written.

The format of each command is as shown below.
The minimum required parameters are placed in the top white area. Please do not change these.

The part of User param (purple part)in the figure below in userproc_command.h , you should define your parameters.

diag 3d57cbbb80b2fb16206e83f6942e25e8
Figure 7. the format of command

Each command is discribed in the following.

struct InitParam : public CustomprocCommand::CmdBase
  • Parameter for Init processing.
    All parameter defined by reserved , so you will change them to necessary parameters such as the number of channels and bit length.

struct ExecParam : public CustomprocCommand::CmdBase
  • Parameter for Exec processing.
    The address and size of audio data is defined in CustomprocCommand::ExecParamBase from which it is inherited as in ExecParam in the figure above.
    For details, see sdk/modules/include/audio/dsp_framework/customproc_command_base.h .

struct FlushParam : public CustomprocCommand::CmdBase
  • Parameter for Flush processing.
    The address and size of audio data is defined in CustomprocCommand::FlushParamBase from which it is inherited as in ExecParam in the figure above.
    For details, see sdk/modules/include/audio/dsp_framework/customproc_command_base.h .

struct SetParam : public CustomprocCommand::CmdBase
  • Parameter for Set processing.
    Define various dynamically changed parameters. By default, RC filter On/Off and coefficients are defined as sample.

Each functions

The following functions are written in userproc.cpp . The contents can be written freely.
Processing is performed according to each command definition.

void UserProc::init(InitParam *)
  • Write your initialize processing according to InitParam.
    It is executed by AUDCMD_INITMFE command from application code.
    (Nothing is done by default)

void UserProc::exec(ExecParam *)
  • Write your signal processing according to ExecParam. When you start recording, it will be called periodically from the SDK.
    1 frame is 768 samples when recording setup is LPCM, 1152 for MP3 (but 1728 at 16 kHz) samples.
    Get data from the input data address, do your signal processing, and write to the output data address. (RC filtering is written by default)

void UserProc::flush(FlushParam *)
  • Write the flush (termination) process according to FlushParam.
    when recording stops, it will be called only once from the SDK.
    For example, If it is delay filter like IIR or FIR filter, flush may be needed as filer clear.
    If there is data to be output, write to the output data address.
    (Nothing is done by default)

void UserProc::set(SetParam *)
  • Write the set (change parameter) process according to SetParam.
    It is executed by AUDCMD_SETMFE command from application code.
    (By default, the RC filter coefficient is set.)

4.3.2. Step 2. Build PREPROC binary

If you enable Preprocess in configuration, PREPROC binary will be created automatically when this application is built.
The path created is PREPROC under spresense/examples/audio_recorder/worker/src .
Put this in the /mnt/sd0/BIN ( \BIN viewed from the PC) folder on the SD card.