EssentiaTD

Reference

Every parameter, every channel.

All five operators: what each one computes, its parameters and defaults, and what every output channel measures and is good for.

EssentiaTD v2.1.0Powered by Essentia · Darien Brito
Operator

Essentia Spectrum

OUTPUT

Computes the FFT from raw audio and outputs the raw result: spectrum (magnitude) and phase channels of fftSize/2+1 bins as a static sample buffer. For visualization, GLSL, resynthesis, or custom feature work. The analyzers run their own FFT and do not read from it.

  • InputAudio CHOP (mono, first channel used)
  • Output2 channels (spectrum, phase) × (fftSize + zeroPad)/2 + 1 samples: 513 at the default FFT 1024. A static indexed buffer (sample = frequency bin), not time-domain. Bins are spaced sampleRate / (fftSize + zeroPad) apart.
  • ProcessingLatest fftSize samples each cook → Windowing (Hann / Hamming / Triangular / Blackman-Harris) → Zero Padding (optional) → FFT → CartesianToPolar (magnitude + phase)

Parameters

ParameterTypeDefaultOptions / RangeDescription
FFT SizeMenu1024512 / 1024 / 2048 / 4096 / 8192 / 16384Window size for FFT computation
Hop SizeInt51264 – 16384No effect on the output: Spectrum analyzes the latest window once per cook. Reported on the info channels only
Window TypeMenuBlackman-Harris 62Hann / Hamming / Triangular / Blackman-Harris 62/70/74/92Window function applied before FFT
Zero PaddingMenuNoneNone / Half FFT / Full FFTAppends zeros before the FFT: more, finer-spaced bins (769 or 1025 at FFT 1024), interpolated rather than truly sharper
FFT Size & Quality Tradeoff

Larger FFT sizes improve frequency resolution (more bins, better at distinguishing close pitches) at the cost of time resolution (each frame covers more audio, smearing transients). This is exactly why each analyzer runs its own FFT: Tonal defaults to a large window (auto: 4096+), Rhythm to a short one (1024). For Spectrum itself the default 1024 is a good general-purpose balance; go bigger only when you need finer bins.

Window Type

Each window trades main-lobe width for side-lobe suppression. The default Blackman-Harris 62 follows Essentia's recommended analysis configuration; Hann is the lighter classic alternative. The higher Blackman-Harris variants offer progressively stronger side-lobe suppression (70/74/92 dB) at the cost of wider main lobes.

Zero Padding

Appends zeros to the windowed frame before FFT, which interpolates spectral bins without changing frequency resolution. This improves the accuracy of peak-based descriptors (centroid, rolloff, pitch) and produces smoother spectrum plots. “Half FFT” adds fftSize/2 zeros; “Full FFT” doubles the frame.

Output channels

ChannelRangeWhat it does & how to use it
spectrum0+ (fftSize/2+1 samples)Raw FFT magnitude bins. Visualize as a bar graph, 3D terrain, or heatmap. Use as input for custom analysis. Bin 0 is DC and the last bin is Nyquist (sampleRate / 2); lower bins are bass, upper bins treble. Slice a range of samples with a Trim CHOP to isolate frequency bands.
phase−π to π (radians)Phase of each bin. Pairs with spectrum for resynthesis, phase-vocoder style effects, or phase-aware custom features. Rarely useful to visualize directly.
Operator

Essentia Spectral

ANALYSIS

Computes spectral shape descriptors and mel-frequency band energies, with optional PCA reduction of the feature set. Supports Realtime (per-frame, own FFT) and Batch (full-file offline analysis). Each feature group can be independently toggled.

  • InputAudio / File In CHOP (raw audio, both modes)
  • OutputRealtime: 1 sample per channel at frame rate | Batch: N samples (one per analysis frame). Channels: mfcc0–mfcc12, spectral_centroid, spectral_flux, spectral_rolloff, spectral_contrast0–spectral_contrast5, hfc, spectral_complexity, mel bands (mel0_0_129 style, band index then its low and high edge in Hz, with Mel Freq Names on; plain mel0–melN with it off), pc0–pcN (if PCA enabled), pc_var0–pc_varN (if PCA Variance Channels enabled). Only enabled features produce channels.
  • FeaturesMFCC (timbral fingerprint, 13 coefficients) ·Centroid (brightness) ·Flux (rate of spectral change) ·Rolloff (85% energy frequency) ·Contrast (peak-valley difference in 6 bands) ·HFC (high-frequency content) ·Complexity (number of spectral peaks) ·Mel Bands (perceptual frequency band energies)

Parameters

ParameterTypeDefaultDescription
ModeMenuRealtimeRealtime (per-frame from raw audio, own FFT) / Batch (full-file offline)
FFT SizeMenu2048Analysis window, 512 to 16384. Applies in both modes (page Analysis)
Hop Size (Batch)Int1024Samples between analysis frames. Batch only (Realtime analyzes the latest window once per cook)
Zero Padding (Batch)MenuNoneNone / Half FFT / Full FFT. Batch only
Window TypeMenuBlackman-Harris 62Window function. Applies in both modes
Compute (Batch)Pulse–Run the batch analysis now. Only visible in Batch mode
Auto Compute (Batch)ToggleOnRe-run when the input audio changes (parameter changes only raise a stale warning). Only visible in Batch mode
Enable MFCCToggleOnEnable/disable MFCC output channels
MFCC CountInt13Number of MFCC coefficients (1–20)
MFCC Low FreqFloat0 HzLower frequency bound for MFCC mel filters
MFCC High FreqFloat11000 HzUpper frequency bound for MFCC mel filters
Enable CentroidToggleOnEnable spectral centroid
Enable FluxToggleOffEnable spectral flux
Flux Half RectifyToggleOffOnly count energy increases (onset emphasis)
Flux NormMenuL2L1 or L2 norm for difference computation
Enable RolloffToggleOffEnable spectral rolloff
Rolloff CutoffFloat0.85Energy fraction threshold (0.5 = median, 0.85 = standard, 0.95 = brightness)
Enable ContrastToggleOffEnable spectral contrast
Contrast BandsMenu6Number of octave sub-bands (4 / 6 / 8)
Enable HFCToggleOnEnable high-frequency content
HFC TypeMenuMasriMasri / Jensen / Brossier: different HFC formulations
Enable ComplexityToggleOnEnable spectral complexity
Complexity ThresholdFloat0.005Minimum peak magnitude to count (0–0.1)
Enable Mel BandsToggleOnEnable mel band output channels
Mel Bands CountMenu4024 / 40 / 60 / 80 / 128
Mel Low FreqFloat0 HzLower frequency bound for mel filters
Mel High FreqFloat22050 HzUpper frequency bound for mel filters
Mel Freq NamesToggleOnInclude frequency ranges in channel names
Log Mel (dB Scale)ToggleOffConvert mel band output to dB scale
PCA PAGE
Enable PCAToggleOffEnable PCA dimensionality reduction: projects all enabled spectral features into a compact set of principal components
PCA ComponentsInt3Number of principal components to output (2–16). Clamped to the number of enabled spectral channels
PCA Window Size (RT)Menu512Circular buffer size for covariance estimation (128 / 256 / 512 / 1024 / 2048 / 4096). Larger = more stable, smaller = more responsive
PCA Update Rate (RT)Int1How often to recompute eigenvectors per second (1–60 Hz). Lower values reduce CPU load
PCA Variance ChannelsToggleOffAppend pc_var0–pc_varN channels showing the variance ratio explained by each component
PCA: Dimensionality Reduction

PCA reduces many correlated spectral features (MFCCs, mel bands, centroid, etc.) into a few decorrelated dimensions that capture the most variance. With 3 components you get a compact "spectral signature" suitable for driving 3D position, color (RGB), or clustering. Realtime mode uses a sliding window with throttled updates for stability; Batch mode computes PCA over the entire file. Sign-flip correction ensures temporal coherence between recomputations.

MFCC Frequency Bounds

The default 0–11000 Hz covers the full speech/music range. For voice-only analysis, narrow to 80–3400 Hz to exclude sub-bass and high-frequency noise. For full-band analysis, set High Freq to the Nyquist (sampleRate/2).

HFC Type

Masri weights by energy×frequency (default), Jensen by amplitude×frequency² (stronger high-frequency emphasis), Brossier by amplitude×frequency (linear). Jensen and Brossier respond more aggressively to transients in the upper spectrum.

Output channels

ChannelRangeWhat it does & how to use it
mfcc0–mfcc12~−50 to 50Timbral fingerprint. MFCCs capture the "texture" of a sound independent of pitch. Use them to cluster similar sounds, drive visual style transitions based on tonal character, or distinguish instruments. Feed into a Math CHOP for normalization, then map to shader uniforms.
spectral_centroid0 – sr/2 HzBrightness. The "center of mass" of the spectrum. High values mean bright, shimmery sounds; low values mean warm, bassy tones. Map to color temperature (orange ↔ blue), particle speed, or lighting intensity.
spectral_flux0+Rate of spectral change. Spikes when the timbre shifts suddenly: a new instrument enters, a filter sweeps, or a transition occurs. Use to trigger visual scene changes, glitch effects, or novelty highlights.
spectral_rolloff0 – sr/2 HzEnergy distribution edge. The frequency below which 85% of spectral energy lies. Distinguishes bright mixes from dark ones. Control cutoff-style visual filters, blur intensity, or fog density.
spectral_contrast0–5unboundedPer-band tonal vs. noisy content. Six sub-bands measuring peak-to-valley ratio. Drive multi-layer visual intensities: assign each band to a separate geometry layer or glow ring for a multi-band reactive sculpture.
hfc0+High-frequency content. Sensitive to cymbals, hi-hats, and sibilance. Use for sparkle/shimmer particle effects, percussive high-end triggers, or treble-reactive lighting.
spectral_complexity0+Number of spectral peaks. Simple tones (sine, flute) score low; complex sounds (orchestra, noise) score high. Drive visual density: particle count, fractal detail, or geometric subdivision level.
mel0–melN0+Perceptual frequency band energies. Mel bands approximate human hearing: evenly spaced in perceived pitch, not linear frequency. Ideal for multi-band audio visualizers, frequency-mapped color gradients, per-band particle emitters, or as input features for ML classifiers.
pc0–pcNunboundedPrincipal components (PCA). Reduces all enabled spectral features into a compact set of decorrelated dimensions ordered by variance. pc0 captures the dominant pattern of variation across your features. Map to position, color, or shape parameters for a low-dimensional "summary" of the audio's spectral character. Requires Enable PCA = On.
pc_var0–pc_varN0 – 1Variance ratios. How much of the total feature variance each principal component explains. Use to auto-weight visual mappings: components with higher variance carry more information. Requires Enable PCA and PCA Variance Channels = On.
Operator

Essentia Tonal

ANALYSIS

Detects pitch, harmonic content, musical key, dissonance, and inharmonicity. Supports Realtime (per-frame with EMA smoothing) and Batch (full-file with global or windowed key detection). FFT Size defaults to auto: semitone-safe at any sample rate.

  • InputAudio / File In CHOP (raw audio, both modes)
  • OutputRealtime: 1 sample per channel | Batch: N samples per channel. Channels: pitch, pitch_confidence, note (with Enable Pitch Note), chroma note_a–note_gs, key, major_minor, key_strength, dissonance, inharmonicity. With Musical Labels off the names become pitch_note, hpcp0–hpcpN and key_scale. Only enabled features produce channels.
  • FeaturesPitch (YinFFT algorithm, Hz + confidence) ·HPCP (harmonic pitch class profile / chroma, 12/24/36 bins) ·Key (key + major/minor + strength) ·Dissonance (sensory roughness) ·Inharmonicity (deviation from harmonic series)

Parameters

ParameterTypeDefaultDescription
ModeMenuRealtimeRealtime (per-frame from raw audio, own FFT) / Batch (full-file offline)
FFT SizeMenuautoBoth modes. Auto picks the smallest semitone-safe window for the incoming sample rate (4096 at 44.1/48 kHz, 8192 at 88.2/96 kHz); explicit smaller sizes warn
Hop Size (Batch)Int2048Samples between analysis frames. Batch only
Window TypeMenuBlackman-Harris 62Window function. Applies in both modes
Zero Padding (Batch)MenuNoneNone / Half FFT / Full FFT. Batch only
Compute (Batch)Pulse–Trigger batch analysis
Auto Compute (Batch)ToggleOnRe-run when the input audio changes (parameter changes only raise a stale warning)
Pitch AlgorithmMenuYinFFTYinFFT
HPCP SizeMenu1212 (default, 1 per semitone) / 24 / 36 bins
Enable PitchToggleOnEnable pitch detection
Pitch Min FreqFloat20 HzMinimum detectable frequency (constrain to instrument range)
Pitch Max FreqFloat22050 HzMaximum detectable frequency
Pitch ToleranceFloat1.0Peak detection strictness (lower = fewer octave errors, more unvoiced frames)
Enable HPCPToggleOnEnable chroma output
HPCP HarmonicsInt0Harmonic contributions to the chroma channels (0–12; 0 = fundamental only, 3–5 for harmonic instruments)
Reference FreqFloat440 HzTuning reference, 415–466 Hz (415 = Baroque, 432 = alternative, 440 = standard)
HPCP Non-LinearToggleOffApply peak-sharpening post-processing
HPCP NormalizedMenuUnit MaxUnit Max / Unit Sum / None
Enable KeyToggleOnEnable key detection
Key Frames (RT)Int8HPCP frames to average for key detection (1–300). Realtime only
Key ProfileMenuBgateBgate (default) / Temperley / Krumhansl / EDMA / Diatonic / Gomez
Peak ThresholdFloat0.00001Minimum spectral peak magnitude. Filters noise-floor peaks from HPCP/Key/Dissonance/Inharmonicity
Peak Max FreqFloat3500 HzUpper frequency limit for spectral peak detection; tonal content lives below 3500 Hz
Enable DissonanceToggleOnEnable dissonance output
Enable InharmonicityToggleOnEnable inharmonicity output
Musical LabelsToggleOnUse note names (A through G#) instead of indices for HPCP channels
Enable Pitch NoteToggleOffOutput pitch-to-note-class channel
Smoothing (RT)Float0.5EMA smoothing coefficient (0 = none, 0.99 = maximum). Realtime only; does not affect key output
Key Mode (Batch)MenuGlobalGlobal (single key for whole file) / Windowed (per-frame key). Batch only
Key Window Size (Batch)Int8HPCP frames for windowed key detection (1–300). Batch only, visible when Key Mode = Windowed
Recommended Settings

The defaults follow Essentia’s MusicExtractor recommendations. Here are per-use-case tweaks:

FFT Size is a correctness setting for key, not a latency knob. The spectrum has to separate adjacent semitones, so the requirement scales with sample rate: 4096 at 44.1/48 kHz, 8192 at 96 kHz. Auto picks the right one. Below it the key goes wrong with a confident-looking strength (measured on one track at otherwise identical settings: 4096 correct, 2048 a fifth away, 1024 elsewhere again, in both Realtime and Batch). The operator warns when you cross that line, so if you see “Spectrum too coarse for key/HPCP”, raise FFT Size rather than trusting the reading.

Key Detection (best accuracy): the defaults are already the accurate setting. The key stage runs its own chroma pinned to Essentia’s KeyExtractor configuration, so HPCP Size, HPCP Harmonics and HPCP Non-Linear do not affect the key output. Worth tuning: Key Profile (Bgate by default, matching Essentia), Key Frames = 30–60 in Realtime so the window spans whole chords rather than a fraction of one. For batch, use Global key mode unless tracking modulations.

Pitch Tracking: Pitch Algorithm = YinFFT (default). Constrain frequency range to your source: voice 80–800 Hz, guitar 80–1200 Hz, bass 30–300 Hz. Smoothing = 0.3 for responsiveness.

Chord / Harmony Analysis: HPCP Size = 36, HPCP Harmonics = 8, HPCP Non-Linear = On, Key Profile = EDMA (electronic/dance) or Temperley (pop/rock).

Live Visuals (fast RT): HPCP Size = 12 (default), Musical Labels = On (default), Smoothing = 0.5, Key Frames = 8 (default). Enable Pitch Note for a 0–11 note class output.

Key Profile

Different profiles are tuned for different genres. Bgate (default) is Essentia’s own default and the profile its KeyExtractor uses. Temperley is a well-tested general-purpose alternative. Krumhansl is the classical music theory standard. EDMA is designed for electronic/dance music. Diatonic is the simplest model. Gomez is optimized for guitar-heavy material.

Pitch Frequency Range

Constraining to instrument-appropriate bands eliminates octave errors. Common ranges: guitar 80–1200 Hz, voice 80–800 Hz, bass 30–300 Hz.

HPCP Harmonics

When set to 0 (default), only the fundamental contributes to chroma. Setting to 3–5 makes HPCP more robust for harmonic instruments (piano, guitar, voice) where overtones reinforce the pitch class. This affects the chroma output channels (note_* / hpcpN) only. Key detection is unaffected: it runs its own chroma fixed at 4 harmonics, matching Essentia’s KeyExtractor.

Peak Threshold & Max Freq

The default threshold (0.00001) and max frequency (3500 Hz) match Essentia’s MusicExtractor. These filter noise-floor peaks and limit analysis to the tonal range where pitch classes are meaningful. Increase threshold for noisier signals; raise max frequency above 3500 only for very high-pitched instruments.

Output channels

ChannelRangeWhat it does & how to use it
pitch0+ HzFundamental frequency. Detected via YinFFT. Map to note names for generative music notation, control animation speed or vertical position with pitch, or track vocals for lip-sync effects.
pitch_confidence0 – 1Pitch reliability. Low during noise or silence, high for clear tonal content. Use as a gate: only apply pitch-driven effects when confidence exceeds a threshold. Crossfade between pitched and unpitched visual modes.
note0 – 11, or −1Pitch as a note class. Needs Enable Pitch Note. The detected pitch rounded to the nearest semitone: 0 = C, 1 = C#, ... 9 = A, ... 11 = B, relative to Reference Freq. Reads −1 when no pitch is detected. Named pitch_note with Musical Labels off. Drive note-indexed palettes, sample triggers, or per-note geometry.
note_a–note_gs0 – 1Chroma / pitch class energy. 12 bins (one per semitone, A through G#) with Musical Labels on (default). Bin 0 = A (reference frequency). Visualize as a harmony wheel, map each note to a unique color, detect chord changes for scene transitions, or build a real-time piano-roll display. HPCP Size 24 or 36 adds sub-semitone bins (note_a, note_a+, ... or note_a, note_a~, note_a+, ...). With Musical Labels off the channels are hpcp0–hpcpN.
key0 – 11Musical key. Encoded integer (0=C, 1=C#, ... 11=B). Assign color palettes or scene themes per key. Drive generative art parameters that shift as the music modulates to a new key.
major_minor0 or 1Major or minor. 0 = major, 1 = minor. Use for mood: major keys → warm, bright, expansive palettes; minor keys → cool, dark, constrained palettes. Switch between two visual presets.
key_strength0 – 1Key confidence. Higher when the harmonic content strongly matches a key template. Blend strength into color saturation or confidence-gated transitions.
dissonance0 – 1Sensory roughness. Measures perceptual dissonance between spectral peaks. Map to visual chaos: glitch intensity, distortion amount, turbulence in fluid simulations, or camera shake.
inharmonicity0 – 1Deviation from harmonic series. Low for flutes and voices, high for bells and percussion. Distinguish melodic from percussive sources, or drive material textures (metallic vs. organic).
Operator

Essentia Rhythm

ANALYSIS

Detects onsets and estimates tempo, with optional kick / perc / hat drum triggers. Realtime analyzes its own short window (default 1024) with TempoTapDegara BPM and Onsets-style adaptive thresholding. Batch uses RhythmExtractor2013 for full-file BPM and beat tracking.

  • InputAudio / File In CHOP (raw audio, both modes)
  • OutputRealtime: 1 sample per channel | Batch: N samples per channel. 6 channels: onset (0/1 trigger), onset_strength, bpm, beat (0/1 trigger), beat_phase (0–1 sawtooth), beat_confidence. With Enable Drums on, 6 more: low, mid, high (band levels 0–1) and kick, perc, hat (0/1 triggers)
  • ProcessingRT: OnsetDetection/SuperFlux → Onsets-style adaptive threshold → onset trigger. ODF history → TempoTapDegara → BPM + tick-anchored beat phase. Batch: RhythmExtractor2013 → BPM/beats + per-frame onset detection via Onsets algorithm.

Parameters

ParameterTypeDefaultDescription
ModeMenuRealtimeRealtime (per-frame from raw audio, own FFT) / Batch (full-file offline)
Window Size (RT)Menu1024Realtime analysis window (512 / 1024 / 2048 / 4096). 1024 is the measured best default; the operator warns if the window is shorter than the per-cook audio chunk
Window TypeMenuBlackman-Harris 62Window function. Applies in both modes
FFT Size (Batch)Menu2048Window size for the batch FFT (512 to 16384). Batch only; Realtime uses Window Size
Hop Size (Batch)Int512Samples between analysis frames (~11.6 ms at 44100 Hz). Batch only
Zero Padding (Batch)MenuNoneNone / Half FFT / Full FFT. Batch only
Compute (Batch)Pulse–Trigger batch analysis
Auto Compute (Batch)ToggleOnRe-run when the input audio changes (parameter changes only raise a stale warning)
Onset MethodMenuComplexHFC / Complex / Flux / Mel Flux / RMS / SuperFlux
Onset SensitivityFloat0.50.0 (rare triggers) – 1.0 (frequent). In batch mode, maps to Onsets algorithm alpha and silence threshold
BPM Min / MaxInt60 / 180BPM search range (internally clamped to [40,180] / [60,250] for Essentia algorithms)
Rhythm Method (Batch)MenuDegaraDegara / Multi-Feature: RhythmExtractor2013 algorithm variant, Batch only
DRUMS PAGE
Enable DrumsToggleOffAdds low mid high kick perc hat after the rhythm channels, both modes. The rhythm channels are bit-identical either way
Drum WindowMenu20481024 / 2048: the drum detector's own FFT, independent of Window Size and FFT Size. 2048 catches kicks better, 1024 reacts up to 10 ms sooner
Kick / Perc / Hat SensitivityFloat0.50 – 1 per class. Higher fires more often; at 0 only a hit louder than anything recent fires
Drum Triggers: What to Expect

Each band fires on a jump in its own spectral flux relative to that band's recent hits (perc also has to clear twice its average over the last 0.1 s), with a 50 ms refractory and a level gate so near-silence never fires. It does not know what a drum sounds like: on a full mix a bass line can fire kick and a guitar or vocal can fire perc.

Measured on the held-out half of a 171-track drum corpus (F-measure at ±50 ms), Realtime at 60 fps with the defaults: kick 0.835, perc 0.632 (0.443 on full band mixes), hat 0.813; median latency 27 / 23 / 19 ms. Batch at Hop Size 512: 0.882 / 0.645 / 0.825, and within 0.002 of that at Hop Size 1024 or 2048. Cost: about 0.05 ms per cook.

It adapts to the loudest recent hits: after switching to a much quieter source give it up to half a minute, or toggle Enable Drums. In Batch, drum triggers land near each hit's true time while onset fires about FFT Size / 2 early (29 ms at the defaults), so allow for that offset when combining them. The Info CHOP channel drum_resyncs counts restarts after cooks the operator missed (Mode on Batch, nothing pulling it); it stays 0 in normal playback.

Onset Method

Complex (default) uses both magnitude and phase for the most accurate general-purpose onset detection. HFC emphasizes high-frequency transients, good for percussive material. Flux measures overall spectral change. Mel Flux applies mel-weighted spectral difference, more robust for harmonic/melodic content. RMS uses simple energy change: fast and reliable for broadband signals. SuperFlux uses TriangularBands + SuperFluxNovelty for music with soft/gradual onsets.

Beat Detection

Realtime: BPM estimation uses Essentia’s TempoTapDegara algorithm, run periodically (~1.5 s) on accumulated onset detection history. BPM is derived from median tick intervals; beat phase is anchored to tick positions for audio-synchronized animation. The tempo clock runs on elapsed audio rather than on cook count, so BPM stays correct when TouchDesigner drops frames. Once the onset history is fully silent (a paused transport, digital silence), bpm holds its last value and beat_confidence drops to 0 instead of inventing a tempo. Batch: RhythmExtractor2013, which requires 44.1 kHz input; non-44.1 kHz audio is resampled internally (see the sample-rate callout below).

Best Quality Settings

Realtime: Onset Method = Complex, Window Size = 1024 (both defaults; 1024 was validated by a measured onset-F1 window sweep). Batch: Onset Method = Complex, Rhythm Method = Degara, FFT 2048, Hop 512 (all defaults). For both modes: set the narrowest BPM range that covers your material (e.g., 100–140 for house, 80–160 for pop). Use SuperFlux for music with soft/gradual onsets.

Batch BPM and Sample Rate

RhythmExtractor2013 requires 44.1 kHz input. Non-44.1 kHz audio is resampled internally (libsamplerate) before BPM extraction, verified against a native 44.1 kHz analysis of the same material, agreeing to two decimals. Onset detection is frame-based and rate-aware, so it never needed resampling. If you have batch BPM numbers from v1.1.8: that release shipped without the resampler, so BPM on non-44.1 kHz files came out scaled by roughly rate/44100 (about 9% high on 48 kHz files). Expect v2.0 to report different, correct values on the same files.

Analysis Window vs Cook Rate

Realtime Rhythm analyzes the latest Window Size samples once per cook (512–4096, default 1024). Two measured facts guide the choice: 1024 is the validated default (onset F1 0.461 on a 262-onset reference; 4096 collapses to 0.028 because long windows smear onset timing), and if the per-cook audio chunk exceeds the window, the excess is never analyzed: at 30 fps / 48 kHz a 1024 window skips 36% of the audio and detection degrades sharply. The operator warns whenever window < sampleRate / fps and the fix is either direction: raise Window Size or raise TD’s fps.

Output channels

ChannelRangeWhat it does & how to use it
onset0 or 1Transient trigger. Fires on detected attacks: drum hits, plucks, consonants. Use for particle bursts, flash/strobe triggers, camera cuts, step-sequenced events, or instantiating geometry.
onset_strength0+Continuous onset function. The raw detection signal before thresholding. Scale burst intensity or particle count by strength for velocity-sensitive triggers. Smooth with a Lag CHOP for an onset envelope.
bpmBPM min – maxEstimated tempo. Sync LFO rates, animation cycle durations, or generative pattern timing to the music. Divide by 60 to get beats-per-second for direct use in Speed parameters.
beat0 or 1Beat trigger. Fires on the estimated beat grid. Use for rhythmic pulsing, beat-locked scene transitions, synchronized step-sequencing, or quantized color cycling.
beat_phase0 – 1 (sawtooth)Position within beat. Ramps linearly from 0 to 1 between beats. Feed into easing curves (Lookup CHOP) for smooth beat-synced animation: bouncing, breathing, pendulum swings, or rhythmic camera motion.
beat_confidence0 – 1Beat tracking reliability. Low during ambient or arrhythmic passages, high during steady beats. Use to fade in beat-synced effects only when tracking is stable, or crossfade to an onset-only mode when confidence drops.
low mid high0 – 1Drum band levels (Enable Drums). Energy in 30–150 Hz, 200–2500 Hz and 10–20 kHz, each relative to its own recent peak, so they stay usable across quiet and loud material. Drive bass-heavy scaling, mid-band pulses or a high-end shimmer layer without building your own band filters.
kick0 or 1Kick trigger (Enable Drums). Fires for one cook on a jump in the low band. Camera kicks, floor pulses, bass-drop flashes. On full mixes a bass line can fire it too.
perc0 or 1Mid-band hit trigger (Enable Drums). Snares, claps, rims and toms. Backbeat accents, strobes on 2 and 4. The least reliable of the three on full mixes, where guitars and vocals share the band.
hat0 or 1High-band hit trigger (Enable Drums). Hi-hats, shakers, cymbals. Sparkles, particle spawns, fast rhythmic detail.
Operator

Essentia Loudness

METERING

EBU R128 loudness metering with momentary, short-term, and integrated measurements, plus RMS energy and zero-crossing rate. Supports Realtime (per-frame via ring buffer) and Batch (full-file offline analysis).

  • InputAudio / File In CHOP (raw audio, both modes)
  • OutputRealtime: 1 sample per channel | Batch: N samples per channel. 7 channels: loudness (instantaneous dB), loudness_momentary, loudness_shortterm, loudness_integrated, dynamic_range, rms, zcr
  • ProcessingAudio → ring buffer → frame dispatch → Essentia Loudness → dB conversion → sliding windows (momentary 400ms, short-term 3s) → EBU R128 two-pass gating for integrated. RMS and ZCR computed directly from the audio frame.

Parameters

ParameterTypeDefaultDescription
ModeMenuRealtimeRealtime (per-frame via ring buffer) / Batch (full-file offline)
Compute (Batch)Pulse–Trigger batch analysis
Auto Compute (Batch)ToggleOnRe-run when the input audio changes (parameter changes only raise a stale warning)
Frame SizeMenu1024512 / 1024 / 2048
ZCR ThresholdFloat0Dead-band around zero for ZCR (0–0.1). Increase to filter noise-floor chatter on quiet signals

Output channels

ChannelRangeWhat it does & how to use it
loudnessdB (−144 floor to ~0)Instantaneous perceived loudness. The fastest-responding level signal. Use for frame-level reactive scaling: VU-meter needle, geometry size, or opacity. Responds immediately to transients.
loudness_momentarydBEBU R128 momentary (400 ms). A smoothed level that tracks phrase-level dynamics without jitter. Use for dynamic scaling of visual elements, responsive but stable brightness, or gain-riding effects.
loudness_shorttermdBEBU R128 short-term (3 s). Captures the current "energy zone" of the track. Use for scene-level intensity: ambient lighting adjustments, macro-level background color, or fog density that follows the song's structure.
loudness_integrateddBRunning gated average. EBU R128 integrated loudness with absolute + relative gating. In Batch it covers the file up to each frame. In Realtime it is a session meter: it accumulates everything heard since the operator last reconfigured (changing Frame Size resets it), so compare it to a per-file LUFS figure only after a reset. Use for overall show level monitoring, normalization reference, or auto-gain to keep visuals consistent across tracks of different loudness.
dynamic_rangedB (0+)Loudness swing over 3 s. Peak-to-valley of the short-term window. Detects builds and drops: high values mean the audio is moving between quiet and loud. Drive contrast-based transitions, tension/release mapping, or drop-triggered explosions.
rms0 – 1Root mean square energy. The classic "audio reactive" signal: simple, linear amplitude. Map directly to geometry scale, opacity, displacement amount, or any parameter that should pulse with the music's energy.
zcr0 – 1Zero-crossing rate. Measures how "noisy" vs. "tonal" the audio is. High ZCR = noise, percussion, sibilance. Low ZCR = clean pitched tones. Use to drive grain/static effects, distinguish drums from melody, or control visual roughness.