Developer World Spresense
日本語 中文
Table of Contents

1. Spresense Arduino Sensing Library

1.1. Let’s try Step Counter on Spresense

1.1.1. An example sketch of Step Counter

The following are the requirements to run the Step Counter sketch.

In this sketch, a physical sensor client using the BMI160 library works with a logical sensor client using the AESM. The acceleration data obtained from BMI160 is input, the activity recognition is performed by using machine learning (AI), and the recognition result is output.

AESM is installed by Tools > Burn Bootloader on Arduino IDE menu. Please refer to Install Bootloader for details.

On the Arduino IDE, select File→ Examples → Sensing under "Examples for Spresense" → application → step_counter and open the example sketch.

arduino stepcounter menu1
arduino stepcounter menu2

Compile this sketch and upload it to the Spresense board.

When you open serial monitor, the pedometer and other information will be displayed in real time.

arduino stepcounter result
tempo

Number of steps per second [Hz]

stride

Stride [cm]
This is fixed value.

speed

Moving speed [m/s]

distance

Moving distance [m]
The option of calculating the distance by fusing GPS positioning data will be supported in the future.
The current released software does not use GPS data.

step

Number of steps

move-type

Activity recognition result (stopping, walking and running)

1.1.2. Try to change the physical sensor of Step Counter

Here, let’s change the physical sensor to another sensor.

  • Sensor board to ROHM’s SPRESENSE-SENSOR-EVK-701.

  • KX122 Arduino Library

    • How to install KX122 Arduino library

      1. Select Clone or download → Download ZIP on GitHub site and download a ZIP file.

      2. Extract the downloaded ZIP file into any folder.

      3. On Arduino IDE menu, open Sketch → Include Library → Add .ZIP Library…​ and select the KX122 folder from the extracted folder.

      4. If Library added to your libraries. is shown on Arduino IDE, the installation is successful.

How to change step_counter.ino is below.

  • Include library

    Change to inclue <KX122.h>.

    #include <BMI160Gen.h>

    to

    #include <Wire.h>
    #include <KX122.h>
    KX122 kx122(KX122_DEVICE_ADDRESS_1F);
  • setup() function

    Change to initialize KX122. The accel_rate is set to 50 Hz by default of KX122 library.

      /* Initialize device. */
    
      BMI160.begin(BMI160GenClass::I2C_MODE, i2c_addr);
    
      /* Set device setting */
    
      BMI160.setAccelerometerRange(accel_range);
      BMI160.setAccelerometerRate(accel_rate);

    to

      Wire.begin();
      kx122.init();
  • loop() function

    Change to get the acceleration data from KX122.

      float x;
      float y;
      float z;
    
      /* Read raw accelerometer measurements from BMI160 */
    
      BMI160.readAccelerometerScaled(x, y, z);
    
      AccelSensor.write_data(x, y, z);

    to

      float acc[3];
    
      /* Read raw accelerometer measurements from KX122 */
    
      kx122.get_val(acc);
    
      AccelSensor.write_data(acc[0], acc[1], acc[2]);

After applying the above modifications, you should be able to work the Step Counter with KX122.

1.1.3. Let’s try to create the unique logical sensor of Step Counter

If you want to implement your own Step Counter algorithm instead of Sony’s provided Step Counter algorithm, create a logic sensor client as follow.

Create a logical sensor class that inherits SensorClient .

For example, implement MyCounterClass in MyCounter.h. Include the header file as needed. For example, SensorClient.h is required, but sensing/logical_sensor/step_counter.h is necessary only if you want to use the StepCounter definition.

Also, as an example, the figure below shows the storage location of generally used code.

diag 99c0d03fa17e569de5628b57e9395653
Figure 1. Example of MyCounter code storage location

Above diagram is an example of Windows environment. The Arduino directory is ~/Arduino in Ubuntu environment and ~/Documents/Arduino in macOS environment.

class MyCounterClass : public SensorClient

Override the following methods of MyCounterClass on MyCounter.cpp .

bool begin(int id,
           uint32_t subscriptions,
           int      rate,
           int      sample_watermark_num,
           int      size_per_sample));

int publish(FAR void* data,
            uint32_t  size_per_sample,
            uint32_t  freq,
            uint32_t  sample_watermark_num,
            uint32_t  timestamp);

int subscribe(sensor_command_data_mh_t& data);
  • Implementation of begin() :

Set initial settings for MyCounter. It is necessary to register a callback function to receive published accelerometer data. The callback function calls MyCounter’s subscribe() and get the data.

Description example
unsigned char mycounter_cb(sensor_command_data_mh_t &data)
{
  return MyCounter.subscribe(data);
}

bool MyCounterClass::begin(int      id,
                           uint32_t subscriptions,
                           int      rate,
                           int      sample_watermark_num,
                           int      size_per_sample)
{
  return SensorClient::begin(id,
                             subscriptions,
                             rate,
                             sample_watermark_num,
                             size_per_sample,
                             mycounter_cb);
}
  • Implementation of subscribe() :

You can create your own Step Counter by subscribing to the accelerometer data and processing it with your own algorithm.

If you use AccelSensorClass provided by the library here, the data to be sent is 50 Hz, 50 samples, and it is the data of x, y, z axis aligned like st_accel_axis in the following. Call Subscribe() in the base class ( SensorClient ) to get this data.

Description example
int MyCounterClass::subscribe(sensor_command_data_mh_t& data)
{
  struct st_accel_axis
  {
    float x;
    float y;
    float z;
  } *accel_axis;

  accel_axis = static_cast<accel_axis*>(SensorClient::subscribe(data));
  uint32_t  timestamp = data.time;

  SensorResultStepCounter output;

  /*
    Implement your own algorithm and put a value in output.
   */

  publish(&output,
          sizeof(SensorResultStepCounter),
          1, /* 1Hz */
          1, /* 1sample */
          timestamp);
}
The subscribed data is sensor_command_data_mh_t . Please refer to here for this data structure.
  • Implementation of publish() :

publish() send the data calculated by MyCounter.
The StepCounterReader is provided as a sample of the reading application client, and if you use it as it is, the data structure to publish is

SensorResultStepCounter

As mentioned above, when passed in the form of SensorResultStepCounter , publish() of the base class ( SensorClient ) can be used as it is.

1.1.4. Let’s try to change the data according to your logic sensor

Of course, you may want to change the resulting sensor data to match your own logic sensor.

In that case, use your defined data type (e.g. MyStepCounterResult) in publish() of MyCounter class.

int MyCounterClass::subscribe(sensor_command_data_mh_t& data)
{
  ..(Omission)..

  MyStepCounterResult output;

  publish(&output,
          sizeof(MyStepCounterResult),
          1, /* 1Hz */
          1, /* 1sample */
          timestamp);
}

Please implement as follows in Application ( StepCountReaderClass ).

subscribe()
{
  MyStepCounterResult* steps = reinterpret_cast<MyStepCounterResult*>(ApplicationSensorClass.subscribe(data));
}

1.2. Let’s try to create new sensor client

1.2.1. Let’s try to create new physical sensor client

[T.B.D]

2. Spresense Arduino multi-core environment

This section describes the multi-core programming development environment using Arduino IDE.

In Arduino multi-core environments, please use the MultiCore MP library. The application CPU has total of 6 cores, consisting of 1 MainCore and 5 SubCore. In the normal single core application, the user programs only use MainCore. In the multi-core application environment described here, you can program all cores including SubCore.

MainCore

MainCore is always started at power on.

SubCore(s)

SubCore(s) are launched from MainCore. There are total of 5 SubCores.

In this tutorial, we show two examples with the MultiCore MP library.

Multicore Boot Example

Boot 4 SubCores and perform LED blinking using multicore.
By running this sample, you can understand the procedure to boot SubCore.

Multicore MessageHello Example

Communicate messages between MainCore and SubCore.
By running this sample, you can understand how to communicate between MainCore and SubCore.

2.1. Multicore Boot example

The main steps are:

  1. Open a sketch using Arduino IDE

  2. Select core

  3. Compile & upload

Repeat above for the number of cores you want to program.

Please follow the procedure as below.

  • Launch Arduino IDE from your desktop
    (If there is no icon on your desktop, please launch from the start menu etc.)

    arduino multicore desktop
    In multi-core programming, please note when you launch multiple Arduino IDE windows.
    If you open a new window with File→ New from the Arduino IDE menu, if you switch the menu of Core selection in one window, the setting is reflected to all the windows.
    As a workaround, when you open multiple Arduino IDE windows for every Core, please launch Arduino IDE from the icon on your desktop. Then, menu settings such as Core selection can be set independently for each window.
  • MainCore programming

    • Open a sketch from Arduino IDE menu File → Examples → MultiCore MP → Boot → Main.

      arduino multicore boot0
      arduino multicore boot1 main
    • Explanation for Main.ino sketch

      MP.begin(subid) boot 4 SubCores.

      arduino multicore boot4 main
    • Select MainCore from Arduino IDE menu Tools → Core → MainCore.

      arduino multicore boot2 main

      You can see that MainCore is selected in the status bar.

      arduino multicore boot3 main
      In Arduino IDE 1.8.9, there is an issue, the status bar is not displayed correctly after switching menu.
    • Press the Upload button to do Compile & Upload.

      arduino multicore boot5 main
      You cannot upload multiple core programs from multiple windows at the same time.
      Upload each core one by one so that the upload operation is not executed simultaneously.
    • Size of used memory is displayed on the log of compilation result.

      MainCore always uses 768 KByte of memory.

      arduino multicore boot6 main

      If Upload is successful, MainCore programming is completed.

      • As a test, try to open the serial monitor without SubCore program.
        Since there is no SubCore, you can see that MP.begin(1~4) returns error.

        arduino multicore boot7 serial err

        If the serial monitor is open, uploading from other windows may fail.
        Please close the serial monitor after this trial.

Then we will program the SubCore.

  • SubCore programming

    • Open a sketch from Arduino IDE menu File → Examples → MultiCore MP → Boot → Sub1.

      arduino multicore boot0
      arduino multicore boot1 sub1
    • Explanation for Sub1.ino sketch

      In setup(), by calling MP.begin(), notify MainCore that startup is complete.
      In loop(), output log by MPLog() and blink a LED.

      arduino multicore boot4 sub1
    • Select SubCore 1 from Arduino IDE menu Tools → Core → SubCore 1.

      arduino multicore boot2 sub1

      You can see that SubCore 1 is selected in the status bar.

      arduino multicore boot3 sub1
      In Arduino IDE 1.8.9, there is an issue, the status bar is not displayed correctly after switching menu.
    • Press the Upload button to do Compile & Upload.

      arduino multicore boot5 sub1
      You cannot upload multiple core programs from multiple windows at the same time.
      Upload each core one by one so that the upload operation is not executed simultaneously.
    • Size of used memory is displayed on the log of compilation result.

      You can see that this SubCore sample sketch uses 128 KBytes of memory.
      This size increases or decreases depending on the user program.

      arduino multicore boot6 sub1

      If Upload is successful, SubCore 1 programming is completed.

  • Follow the same procedure for SubCore 2, 3 and 4 and execute Compile & Upload.

    • Open Sub2 sketch, select SubCore 2, and Compile & Upload.

    • Open Sub3 sketch, select SubCore 3, and Compile & Upload.

    • Open Sub4 sketch, select SubCore 4, and Compile & Upload.

  • Operation check

    • Blink 4 LEDs on the Spresense main board.

    • Open the serial monitor, and you can see the log from each core.

      arduino multicore boot7 serial
  • Summary

    • We have learned how to boot SubCore using a simple example sketch.
      Add a call to MP.begin() when you port an existing sketch to multi-core environment.

    • The basic usage, such as Arduino IDE’s Compile & Upload method, is the same as usual in multi-core environments. The main difference is that a menu of core selections has been added in Tools→ Core.

2.2. Multicore MessageHello example

The main steps are the same as Multicore Boot example.
In this example, MainCore and SubCore(s) use a common sketch.

  1. Open a sketch using Arduino IDE (only once)

  2. Select core

  3. Compile & upload

Repeat above for the number of cores you want to program. This example runs all SubCores.

Please follow the procedure as below.

  • Launch Arduino IDE from your desktop
    (If there is no icon on your desktop, please launch from the start menu etc.)

    arduino multicore desktop
    In multi-core programming, please note when you launch multiple Arduino IDE windows.
    If you open a new window with File→ New from the Arduino IDE menu, if you switch the menu of Core selection in one window, the setting is reflected to all the windows.
    As a workaround, when you open multiple Arduino IDE windows for every Core, you always launch Arduino IDE from the icon on your desktop. By doing so, menu settings such as Core selection can be set independently for each window.
  • MainCore programming

    • Open a sketch from Arduino IDE menu File → Examples → MultiCore MP → Message → MessageHello.

      arduino multicore boot0
      arduino multicore hello1
    • Explanation for MessageHello.ino sketch

      We use #ifdef SUBCORE ~ #else ~ #endif to switch between MainCore and SubCore implementations in this sketch.

      • MainCore implementation

        In setup(), MP.begin(subid) boot 5 SubCores
        By calling MP.RecvTimeout(MP_RECV_POLLING), set polling in receiver mode. In this mode, when MP.Recv() is called, Recv() do polling with non-blocking.
        In loop(), MP.Recv() receives the packet address from each SubCore, and outputs the message in the packet.

        arduino multicore hello4 main
      • SubCore implementation

        In setup(), by calling MP.begin(), notify MainCore that startup is complete.
        In loop(), put "Hello" message into packet, and send the packet address by MP.Send().

        arduino multicore hello4 sub
    • Select MainCore from Arduino IDE menu Tools → Core → MainCore.

      arduino multicore hello2 main

      You can see that MainCore is selected in the status bar.

      arduino multicore boot3 main
      In Arduino IDE 1.8.9, there is an issue, the status bar is not displayed correctly after switching menu.
    • Press the Upload button to do Compile & Upload.

      arduino multicore hello5 main
      You cannot upload multiple core programs from multiple windows at the same time.
      Upload each core one by one so that the upload operation is not executed simultaneously.
    • Size of used memory is displayed on the log of compilation result.

      MainCore always uses 768 KByte of memory.

      arduino multicore boot6 main

      If Upload is successful, MainCore programming is completed.

Then we will program the SubCore.

  • SubCore programming

    • Select SubCore 1 from Arduino IDE menu Tools → Core → SubCore 1.

      arduino multicore hello2 sub1

      You can see that SubCore 1 is selected in the status bar.

      arduino multicore boot3 sub1
      In Arduino IDE 1.8.9, there is an issue, the status bar is not displayed correctly after switching menu.
    • Press the Upload button to do Compile & Upload.

      arduino multicore hello5 main
      You cannot upload multiple core programs from multiple windows at the same time.
      Upload each core one by one so that the upload operation is not executed simultaneously.
    • Size of used memory is displayed on the log of compilation result.

      You can see that this SubCore sample sketch uses 128 KBytes of memory.
      This size increases or decreases depending on the user program.

      arduino multicore boot6 sub1

      If Upload is successful, SubCore 1 programming is completed.

  • Follow the same procedure for SubCore 2, 3, 4 and 5 and execute Compile & Upload.

    • Select SubCore 2, and Compile & Upload.

    • Select SubCore 3, and Compile & Upload.

    • Select SubCore 4, and Compile & Upload.

  • Operation check

    • Open the serial monitor.

    • MainCore receives the message sent from each SubCore and outputs them to the serial monitor.

      arduino multicore hello7 serial
  • Summary

    • We have learned how to communicate between MainCore and SubCore.

    • It can communicate by sending the address of message because of the shared memory.

    • You can use a common sketch for multi-core programming.

2.3. Notices

  • See developer guide for the details of MultiCore MP library.

  • If you want to remove SubCore binaries on the Spresense board, please do Install Bootloader. When installing the loader, remove all SubCore binaries and then start installing the loader.

    arduino multicore remove subcore

    Note that SubCore does not run unless you call MP.begin() from MainCore. Therefore, there is no problem even if the old SubCore remains on your Spresense board.

  • If you open multiple Arduino IDE windows for every Core, please launch Arduino IDE from the icon on your desktop. Then, menu settings such as Core selection can be set independently for each window.

  • Use Arduino IDE to open multiple windows and select core is complicated, but you can use command line tools such as arduino-builder and arduino-cli. Also, you can upload the multiple binaries for all cores at once by using flash_writer tool. How to use the command line tools will be updated in this manual later.