Table of Contents

Untouchable LIMiTS

Introduction

Looper Instrument & Metronome inside Theremin Synthesizer is a musical instrument that combines the functionality of a synthesizer, an optical-sensor-based theremin, and a loop station, all processed on an ESP32-S3 microcontroller.

General description

The project is structured around the ESP32-S3-WROOM-1 (N16R8) microcontroller, which runs the audio synthesis on one processor core and the user interface/metronome on the other core, ensuring zero-latency performance.

System Architecture (Main Blocks):

Hardware Design

Bill of Materials (BoM):

Block Schema

Electronic schema

Pictures

Pinout & Hardware Mapping

To ensure stability and prevent conflicts with the ESP32-S3's internal memory and boot processes, several pins were explicitly avoided:

Pin Constant / Name GPIO Pin Component / Module Notes & Justification
PIN_I2C_SDA 15 I2C Bus Data (OLED + VL53L1X) SAFE — Not used by Octal PSRAM. Shares the bus for display and sensor.
PIN_I2C_SCL 16 I2C Bus Clock (OLED + VL53L1X) SAFE — Not used by Octal PSRAM.
PIN_ENC_A 17 EC11 Rotary Encoder Standard digital input.
PIN_ENC_B 18 EC11 Rotary Encoder Standard digital input.
PIN_ENC_PUSH 21 EC11 Rotary Encoder Push button for menu selection.
PIN_ENC_KEY 47 EC11 Rotary Encoder Additional encoder key/switch input.
KEY_PINS 1, 2, 4, 5, 6, 7, 8, 9, 10, 11 Touch Keyboard (10 Keys) Uses native S3 touch-capable GPIOs. Consciously avoids strapping pins.
PIN_BPM_POT 12 Analog Potentiometer (BPM) ADC channel — safe and stable for adc_oneshot_read().
PIN_VOL_POT 13 Analog Potentiometer (Volume) ADC channel — safe and stable for adc_oneshot_read().
PIN_MASTER_VOL 14 Analog Potentiometer (Master) ADC channel — safe and stable for adc_oneshot_read().
PIN_AUDIO_PDM 40 Audio Output (PDM via I2S) SAFE — Placed outside of the reserved PSRAM range.
PIN_BUZZER 41 Metronome Buzzer Independent passive buzzer driven via LEDC PWM.
PIN_CH2_PLAY 42 Looper Control (CH2 Play) Re-mapped from 48 (which was colliding with the onboard RGB LED). Safe MTMS pin after GPIO reset.
PIN_CH2_REC 38 Looper Control (CH2 Record) Tactile button (active-LOW with internal pull-up).
PIN_CH1_PLAY 20 Looper Control (CH1 Play) Tactile button (active-LOW with internal pull-up).
PIN_CH1_REC 19 Looper Control (CH1 Record) Tactile button (active-LOW with internal pull-up).

Software Design

The firmware is written in ESP-IDF (v6.x) using the C programming language. The architecture heavily utilizes FreeRTOS to divide the immense computational load of real-time audio synthesis from user interaction and sensor polling.

Firmware Overview:

  • Development Environment: ESP-IDF v6.x with the Xtensa toolchain, using CMake.
  • Third-Party Libraries: Primarily relies on native ESP-IDF drivers (I2C, LEDC, ADC, Touch Sensor, I2S).
  • Key Algorithms & Implementations:
    1. Dual-Channel Looper Engine: Replaced basic overdubbing with a highly advanced dual-channel state machine. Records timestamped NOTE_ON, NOTE_OFF, and VOL_CHANGE events relative to the metronome.
    2. Dynamic BPM Scaling: Looper playspeed is relative to the metronome's real-time BPM. The user can speed up or slow down a recorded loop on the fly without shifting its pitch.
    3. 15-Voice Polyphonic Synthesizer: Calculates sine, square, triangle, and sawtooth waveforms on the fly. Generates 16-bit PCM audio that is pushed to the I2S peripheral in PDM TX mode.
    4. Thread-Safe I2C: A custom shared_i2c.c wrapper uses FreeRTOS mutexes to allow both the OLED display (UI Task) and the VL53L1X sensor (Theremin Task) to safely communicate on the exact same physical I2C pins without corrupting data.

Firmware Architecture

The project utilizes the ESP32-S3's dual-core processor to guarantee that audio never stutters:

Hardware & Interrupt Safety

Because audio instruments require instantaneous feedback, all buttons and keys use hardware interrupts.

To prevent Kernel Panics during heavy system load, the Interrupt Service Routines (ISRs) never use standard logging (ESP_LOGI) or blocking functions. They only update minimal state variables and trigger events instantly using esp_rom_printf for thread-safe debug logging.

Notable Algorithms

Dynamic BPM Playhead Scaling (looper.c): 

uint32_t cur_bpm = metronome_get_bpm();
int64_t scaled_delta = (delta_us * cur_bpm) / s_channels[ch].recording_bpm;
s_channels[ch].playhead_us += scaled_delta;

Instead of recording raw audio (which takes massive memory and locks the tempo), the Looper records “events”. During playback, the time delta is multiplied by the ratio of the current BPM to the recorded BPM. This allows time-stretching loops in real-time.

Hardware Noise Gating (metronome.c): 

/* Hardware deadzones to guarantee true 0% and 100% and prevent ADC boundary jitter */
if (s_master_vol <= 4) s_master_vol = 0;
if (s_current_vol <= 4) s_current_vol = 0;

Analog potentiometers inherently experience electrical noise. When turned almost all the way down, “ADC boundary jitter” would cause the metronome to randomly fluctuate between 0% and 5% volume, resulting in phantom/random beeps. This simple hardware deadzone explicitly suppresses the noise floor.

Collision-Free Polyphony (synth.c / looper.c): To ensure the live instruments and the two looper channels never steal each other's audio channels, the synthesizer allocates a massive pool of 15 voices.

This guarantees robust sound even during complex, multi-layered jamming.

Results Obtained

The final result is a fully functional, standalone digital instrument that requires no PC to operate. The hardware RC filter successfully converts high-frequency PDM digital signals into a smooth analog waveform that drives standard headphones. The capacitive keyboard and optical Theremin successfully mimic traditional musical input, and the Dual-Channel Looper operates flawlessly in sync with the hardware metronome.

Validation

Validation was performed iteratively:

Conclusions

The project successfully proves that advanced DSP (Digital Signal Processing) and multi-layered audio sequencing can be achieved entirely on a low-cost microcontroller. The most difficult challenge was managing the ESP32's memory constraints and routing hardware interrupts without crashing the RTOS scheduler.

If I were to build a v2.0, I would add an external I2S DAC (like a PCM5102a) for true high-fidelity 24-bit audio output, and I would design a custom PCB to eliminate the inherent electrical noise caused by breadboard jumper wires.

Download

The complete project source code is available on GitLab.

Journal

Bibliography / Resources