Raspberry Pi Pico Real-time Sound spectrum

In this tutorial, you will create a real-time audio spectrogram visualizer using a Raspberry Pi development board with an external digital microphone and a TFT LCD display. With it, you can visualize the real-time sounds of your surroundings! The following is a preview of the tool when it is running.

The Adafruit PDM MENS expansion board is used here ( instructions ). We will also use some common digital signal processing (DSP) techniques to process digital audio data directly on the Raspberry Pi Pico instead of just transmitting the digital audio data to the PC via a USB cable. By using DSP technology, the converted audio signal is displayed in real time on the Adafruit 2.0 320×240 color IPS TFT display with a Micro SD card expansion board.

What is an Audio Spectrogram?

The audio spectrogram is used to visualize a real-time audio signal represented by amplitude into a frequency format of the real-time audio signal.

In the figure below, the original audio signal is shown on the left, and the spectrum of the audio signal is shown on the right.

Through the color intensity in the spectrogram, you can see that there is a direct correlation between the amplitude of the audio signal and its frequency.

For a more detailed overview of DSP techniques used to create audio spectrograms, check out the “Fixed-point DSP for Data Scientists” deep dive guide on Towards Data Science .

Spectrograms are also used in audio systems based on machine learning (ML) to convert an audio signal into a spectrogram so that a 2D “image representation” can be defined in the audio signal using computer vision techniques. Real-life examples of this include audio or speech recognition , keyword spotting.

Schematic diagram

Code

The code file can be downloaded here:
https://make.quwj.com/project/471

Production method

To create a spectrogram and display it in real time on the LCD screen, follow the steps below.

1. Collect N audio samples using a digital microphone.
2. Apply a Hanning window to the collected audio samples.
3. Run a Fast Fourier Transform (RFFT) using the input from the previous step.
4. Calculate the magnitude of the RFFT.
5. Map each RFFT magnitude to a color value to display on the LCD display.
6. Display a new line on the LCD.
7. Scroll to a new line and repeat.

If we choose an RFFT size of 256, we will have 128 available amplitude outputs to display on the screen, and since this is less than the 240 pixels per row of the display, we can display each row twice to maximize the visual visibility of the screen.

To achieve a faster visual response, we can collect 64 new audio samples from the microphone at a time (instead of waiting for those 256 new audio samples) and combine them with the previous 192 (256-64) most recent samples in each loop. At a sampling rate of 16 kHz, we will have 64/16000 seconds to perform all the calculations and update the display. This means that each iteration only takes 4 milliseconds.

We will use the microphone-library-for-pico library for Pico to capture data from a digital microphone. The CMSIS-DSP library
for Arm will be used to process the audio data in real time.

CMSIS-DSP is optimized for Arm Cortex-M processors , including the Arm Cortex-M0+ based on the Raspberry Pi Pico’s RP2040 microcontroller (MCU). The Pico’s ST7789 library will be used to drive the outputs of the ST7789 TFT display.

Hardware Setup

Solder male headers to your Raspberry Pico board, Adafruit PDM MEMS Microphone Shield, and 2″ 320×240 Color IPS TFT Display with micro SD Card Shield so they can be plugged into a breadboard. For more information on soldering GPIO pins to a Raspberry Pico.

Once both are soldered, place them on a breadboard and set up the wiring as shown below.

Tabular diagram of wiring setup

+---------+-------------------+     +---------+-------------------+
| PDM Mic | Raspberry Pi Pico |     | ST7789  | Raspberry Pi Pico |
|---------+-------------------|     |---------+-------------------|
|    3V   |        3V3        |     |   VIN   |        3V3        |
|---------+-------------------|     |---------+-------------------|
|    GND  |        GND        |     |   GND   |        GND        |
|---------+-------------------|     |---------+-------------------|
|    SEL  |        GND        |     |   SCK   |      GPIO18       |
|---------+-------------------|     |---------+-------------------|
|    DAT  |       GPIO2       |     |   MOSI  |      GPIO19       |
|---------+-------------------|     |---------+-------------------|
|    CLK  |       GPIO3       |     |   CS    |      GPIO17       |
+---------+-------------------+     |---------+-------------------|
                                    |   RST   |      GPIO21       |
                                    |---------+-------------------|
                                    |   D/C   |      GPIO20       |
                                    +---------+-------------------+

When you are finished, your breadboard should look like this.

Setting up Pico SDK development environment

First you need to set up your computer with the Pico SDK for Raspberry Pi and the required toolchain. Reference:
https://datasheets.raspberrypi.org/pico/getting-started-with-pico.pdf

Section 2.1 of this guide applies to all operating systems. Below are the operating sections specific to each operating system.

• Linux: Section 2.2

• macOS: 9.1 section

• Windows: Chapter 9.2

Obtaining and compiling the Pico-audio-spectrogram application

Make sure the PICO SDK environment variables are set.

export PICO_SDK_PATH=/path/to/pico-sdk

Clone the git repository and change directory in a terminal window

cd ~/ 
git clone --recurse-submodules https://github.com/ArmDeveloperEcosystem/audio-spectrogram-example-for-pico.git
cd audio-spectrogram-example-for-pico

Create a build directory and change directory to:

mkdir build
cd build

Run camake and make and compile:

cmake .. -DPICO_BOARD=pico
make

Press and hold the BOOTSEL button on the motherboard while plugging the motherboard into your computer using the USB cable.

Copy the audio_spectrogram.uf2 file to the installed Raspberry Pi Pico bootable ROM disk:

cp -a audio_spectrogram.uf2 /Volumes/RPI-RP2/.

Test it out

You can now try making some sounds, say a few different words, and see what shows up on the spectrogram.

Here is how the word “yes” will appear on the display.

Likewise, this is how the word “no” will appear on the display.

Examples of various sounds in the “ESC-50: Environmental Sound Classification Dataset ” are shown in the figure below:

Summarize

This guide shows how to make a real-time audio spectrogram visualizer using a Raspberry Pi Pico board with an external digital microphone and a TFT LCD. Initially, the project uses the Pico- specific microphone library to allow the microphone to capture 64 audio samples at a time, and then uses the Arm CMSIS-DSP library to convert the audio samples into a spectrogram.

Finally, Pico’s ST7789 library is used to display the image on the TFT LCD display one line at a time.

You can download the code of this project in the project’s file library:
https://make.quwj.com/project/471