Using Spresense with an array of multiple microphones
The test unit prototype comprises the Spresense board fitted with an 8-mic array, WiFi and an inertial/magnetic sensor (IMU). The mic array perceives all sound signals around the wearer and feeds these streams to the Spresense unit. The WiFi unit allows developers to read out real-time data and adjust settings and the IMU sensor helps position and reorient the unit in the spatial domain. All sensor data is collected, filtered, and extracted by the Spresense unit.
This test unit is used to explore new computing models to accurately and efficiently calculate where different sources are located in the 3D space. The next step is to segment these sources and only let the user hear what's relevant (e.g. a conversation across the table). The microphones are sampled at 8000 Hz. Samples are processed in batches of 16000 with an overlap of 6000 samples. The IMU sensor is sampled at 100 Hz. The software components used for this solution are open source.
Testing the new algorithms against real sound sources and environments
To test the computing models, the research team considered three scenarios:
- Scenario 1: Simulation: Stationary mic frame listening to two sound sources at different angles (a man and a woman)
- Scenario 2: Experiment: Stationary mic frame listening to two sound sources at different angles (a man and a woman)
- Scenario 3: Moving mic frame with one man talking
Six different methods of computing beamforming were evaluated on these scenarios. Below is one of the result outputs where the two gray lines at the side indicate the true angle of the two sound sources and the colorful pattern shows the calculation over time.