See it run
Five operators
Every operator takes raw audio. The four analyzers have a Mode parameter that switches between real-time and batch analysis, and each runs its own FFT internally in both modes.
- OUTPUTEssentia SpectrumFFT magnitude + phase as a static sample buffer (fftSize/2+1 bins), for visualization, GLSL, resynthesis, or custom processing. The analyzers do not use it.
- ANALYSISEssentia SpectralMFCC, centroid, flux, rolloff, contrast, HFC, complexity, mel bands, PCA
- ANALYSISEssentia TonalPitch (YinFFT), HPCP chroma, key/scale, dissonance, inharmonicity
- ANALYSISEssentia RhythmOnset detection, BPM estimation, beat phase/confidence, kick/perc/hat drum triggers (optional)
- METERINGEssentia LoudnessEBU R128 loudness, RMS energy, zero-crossing rate
Full parameters, defaults and every output channel for each operator: the reference.
Real-time or a whole file
Wire an Audio CHOP straight into any analyzer. Incoming audio accumulates in an internal ring buffer; each cook the operator windows and FFTs the latest full analysis window and outputs 1 sample per channel at TD's cook rate. Each operator uses the window its algorithm wants (Tonal resolves semitones, Rhythm keeps transients sharp) with no shared FFT node to configure.
Set Mode to Batch and the operator processes an entire audio buffer (e.g. from an Audio File In CHOP) on a background thread. Output contains one sample per analysis frame across the full file. Auto Compute is on by default, so the analysis re-runs whenever the input audio changes; turn it off and press Compute to run on demand. Auto Compute follows the audio only: change an analysis parameter and the cached result stays on display, with a warning that it is stale until you press Compute. Flipping Mode never requires rewiring: both modes read the same audio input.
Requirements
| Windows | x64. The installer puts the plugins in your user folder, no admin rights. |
| macOS | Apple Silicon and Intel (universal build). |
| TouchDesigner | Loads the five operators as custom CHOPs from its Plugins folder. |
| Input | Mono: the operators analyze one channel. Average stereo to mono first (see Mono by design). |
Get EssentiaTD
- WINWindows installerEssentiaTD-Setup.exe · x64 · per-user, no admin rights
- MACmacOS installerEssentiaTD.pkg · Apple Silicon and Intel
- ZIPZips and older versionsEvery release on GitHub
- SRCSource codeC++, AGPL-3.0, build it yourself
Free and open source under AGPL-3.0. The download links always point to the latest release.
Using EssentiaTD.
Installing, how the operators connect, FFT settings, drum triggers, real-time timing and the details that change your numbers.
Installing
Close TouchDesigner, run the installer for your platform, then start TouchDesigner again. The operators appear in the OP Create Dialog under CHOP, named Essentia Spectrum, Essentia Spectral and so on.
| Installs to | |
|---|---|
| Windows | Documents\Derivative\Plugins\Essentia for the current user. The installer checks that TouchDesigner is closed. |
| macOS | ~/Library/Application Support/Derivative/TouchDesigner099/Plugins/Essentia. Installed from the pkg, the plugins load without per-bundle Gatekeeper prompts. |
Prefer copying files yourself? Download the zip for your platform from the releases page and copy the five plugins (.dll on Windows, .plugin on macOS) into the same folder. TouchDesigner scans subfolders. Unsigned plugins from a zip ask for permission once each on first load.
Building from source: see Build from source in the repository.
Signal flow
One topology for both modes: audio wires straight into every operator, and each runs its own FFT internally with the window its algorithm wants. There is no shared FFT node and no ordering constraint between analyzers.
A shared spectrum forces one FFT size onto operators with conflicting needs: Tonal needs 4096 to resolve semitones, but feeding Rhythm a 4096 spectrum instead of 1024 cut onset-detection F1 from 0.47 to 0.13 on a measured 263-onset reference (long windows blunt the transients onset detection keys on). v2.0 removes the conflict: each operator gets the window its algorithm wants, on the same audio wire. The converted Rhythm at window 1024 reproduces the v1.x implementation's onset F1 exactly.
A Spectrum CHOP wired into Spectral, Tonal, or Rhythm raises an error in both modes ("As of v2.0 this operator analyzes raw audio and computes its own FFT. Connect the audio directly"). Delete the Spectrum node from the analysis chain and wire the audio straight in. If you tuned an upstream Spectrum's FFT Size for Tonal, set Tonal's own FFT Size instead (or leave it on auto). Essentia Spectrum remains available as a raw magnitude + phase output.
FFT settings live on each operator
Each analyzer's Analysis page carries its FFT parameters. FFT Size and Window Type apply in both modes; Hop Size and Zero Padding are batch-only (realtime analyzes the latest window once per cook, and zero padding narrows bin spacing without adding true resolution).
- Spectral defaults to 2048.
- Tonal defaults to auto: the smallest FFT whose bin spacing separates adjacent semitones at the incoming sample rate (4096 at 44.1/48 kHz, 8192 at 88.2/96 kHz). Picking a smaller size explicitly raises a warning in both modes.
- Rhythm batch uses FFT 2048 / hop 512; realtime has its own Window Size menu (512-4096, default 1024, see below).
Tonal and FFT resolution
Key and HPCP need bin spacing of at most 15.56 Hz (a semitone at middle C), which is why Tonal's auto default resolves to 4096 or 8192 depending on sample rate. Two things are worth knowing about the too-coarse warning:
It means "may be unreliable", not "is wrong". Measured across three tracks, the smallest adequate FFT size was content-dependent: 1024, 2048 and 4096 respectively. 4096 is simply the smallest size at which all three were correct.
Do not judge tonal output by how confident it looks. The wrong answer is not noisy or obviously broken; it presents as a stable, settled key. On one test track a 1024-point analysis produced the wrong key at strength 0.608 while the correct answer at 4096 reported only 0.531. key_strength cannot be used to detect this failure.
Full method and measurements: docs/tonal-fft-resolution.md.
Drum triggers
The Rhythm CHOP's Drums page adds six channels after the rhythm channels when Enable Drums is on, in both modes:
| channel | band | what it is |
|---|---|---|
low, mid, high |
30-150 Hz, 200-2500 Hz, 10-20 kHz | band energy, 0 to 1 relative to its own recent peak |
kick, perc, hat |
the same three bands | 1 for one cook when a hit is detected, otherwise 0 |
Names follow the palette's audioAnalysis; its snare is perc here, because the mid band fires on snares, claps, rims and toms alike. Drum Window (1024 / 2048, default 2048) is the detector's own FFT, independent of Window Size and FFT Size. Kick / Perc / Hat Sensitivity (0-1, default 0.5): higher fires more often; at 0 only a hit louder than anything recent fires.
Each band fires on a jump in its own log-compressed spectral flux above a fraction of that band's running peak (perc also above twice its mean over the last 0.1 s), with a 50 ms refractory and a level gate around -80 dBFS so near-silence never triggers. It does not know what a drum sounds like: on a full mix a bass line can fire kick and a guitar or vocal can fire perc.
Measured on the held-out half of a 171-track drum corpus (F-measure at +/-50 ms), Realtime at 60 fps with the defaults: kick 0.835, perc 0.632 (scored on snares; 0.443 on full band mixes), hat 0.813, median latency 26.7 / 23.1 / 19.3 ms. Batch (Hop Size 512): 0.882 / 0.645 / 0.825. Batch drums step at 512 samples or finer whatever the Hop Size, and land close to each hit's true time. The batch onset channel does not: its frame f covers the audio from f x Hop Size onward, so it fires about FFT Size / 2 early (measured 29 ms early at the defaults, against -3 ms for kick). Keep that offset in mind when combining onset with the drum triggers in Batch. Drums on costs about 0.05 ms per cook. The six rhythm channels are bit-identical with drums on or off.
If the operator misses cooks (Mode on Batch, nothing pulling it, a rewired input), the drum window refills instead of joining old audio to new, and the Info CHOP channel drum_resyncs counts those restarts.
Analysis window vs cook rate
Realtime analysis reads the latest window of audio once per cook. Two consequences worth knowing:
If the per-cook audio chunk is larger than the analysis window, the excess is never analyzed. At 30 fps and 48 kHz each cook delivers 1600 samples; a 1024 window skips 576 of them (36% of the audio), and onset detection degrades sharply: measured onset F1 collapsed from 0.461 to below 0.16 across all windows at 30 fps. The operator warns whenever window < sampleRate / fps and tells you which way to fix it (raise the window, or raise TD's fps). Note this also fires for a 512 window at 60 fps (800-sample cooks): 512 is only fully sound above ~94 fps.
Longer windows are not safer. Onset timing smears with window length; at 4096 the onsets that fire are mostly mistimed beyond a 50 ms tolerance (precision 0.056). Keep the default 1024 unless you have a specific reason.
Batch BPM and sample rate
Batch BPM uses Essentia's RhythmExtractor2013, which requires 44.1 kHz input. Non-44.1 kHz audio is resampled internally (libsamplerate) before BPM extraction; onset detection is frame-based and rate-aware, so it never needed resampling. Verified: the same 48 kHz file analyzed natively at 44.1 kHz and through the resampler agree to two decimals.
If you have batch BPM numbers from v1.1.8: that release shipped without the internal resampler, so batch BPM on non-44.1 kHz files was computed at the native rate and came out scaled by roughly
rate/44100(about 9% high on 48 kHz files), with an easy-to-miss "Resample to 44100 Hz failed" warning. Expect v2.0 to report different (correct) values on the same files.
Raw data, not a display
Essentia Spectrum outputs a linear-bin FFT magnitude spectrum plus phase. Linear bins are what analysis and DSP code expect, but they look bottom-heavy when plotted directly; most musical detail is crammed into the lower bins. Use it for GLSL, resynthesis, or custom feature work. For a perceptually scaled on-screen spectrum, TouchDesigner's built-in Audio Spectrum CHOP is the better display choice.
Mono by Design
The suite processes a single audio channel. This is intentional: stereo analysis would double every output channel (e.g., mfcc0_L, mfcc0_R, spectral_centroid_L, spectral_centroid_R), making the output unwieldy and harder to map in a visual context.
If you need stereo-aware analysis, select each channel independently using a Select CHOP and run two separate analysis chains. This keeps the output organized and lets you choose which features to extract per channel.
Recommended approach for stereo sources. In most audio-reactive scenarios, collapsing to mono before analysis preserves all relevant information. Sum left and right with a Math CHOP (Combine Channels = Average) before feeding the analyzers. This captures the full frequency content of both channels without phase cancellation artifacts that a simple channel pick might miss.
Channel choice measurably shifts results. Batch mode analyzes channel 0 (left) only when fed a stereo CHOP. It warns, but the numbers still differ from a mono average. Measured on the same file with identical settings, Tonal's batch key_strength reads 0.825 from the L+R average and 0.882 from the left channel alone (same detected key). When comparing analysis values across sessions or against published numbers, always state how the audio was collapsed to mono.
Read the warnings
The operators report problems on the node instead of silently producing wrong numbers: an FFT too coarse for key detection, an analysis window shorter than the audio delivered per cook, a multichannel input (only channel 0 is analyzed), or batch results that are stale after a parameter change. In Realtime, if Spectral or Tonal rejects a parameter value for one feature (for example more mel bands than the FFT has bins), only that feature switches off and the warning carries Essentia's own reason; the operator errors only when no enabled feature can run.