Sound Analysis, Synthesis and Processing
Scientific areas of pertinence: ING-INF/03 and ING-INF/05
This course provides a comprehensive collection of fundamentals tools for audio synthesis and rendering; and for audio analysis and processing. A wide range of applications will be covered, including musical sound synthesis, 3D audio, restoration of historical recordings, sound classification and music information retrieval, sound localization and tracking, etc.
Part 1 - Digital Audio Analysis and Processing (DAAP)
In this second part we cover relevan applications of digital audio signal processing for various applications, ranging from audio restoration to adaptive audio processing and array processing.
- Sinusoidal modeling - resolution of periodic signals and windowing, spectral interpolation, sinusoidal peak tracking and continuation. Speech modeling (synthesis through analysis). Channel and phase vocoder design.
- Statistical audio processing - Wiener filtering: problem statement, orthogonality principle, Wiener-Hopf eqs., error performance surface. Typical applications of Wiener filtering: echo cancellation, noise cancellation, etc. Linear Prediction Coding: problem statement and application. Kalman filtering: problem statement, Bayesian model. Equalization: typical DSP structures for equalization.
- Audio restoration - noise modeling, removal of typical audio artifacts: clicks, crackles, background noise, hissing noise, wow and flutter, nonlinear distortions. Statistical filtering and equalization.
- Fundamentals of microphone array signal processing - data model, non parametric methods (beamforming and Minimum Variance Distortionless Beamforming). Parametric methods: MUSIC.
- Feature extraction and analysis: low-level descriptors (time-domain, spectral and timbral features), sound classification and similarity (dimensionality reduction, PCA classification methods.
Part 2 - Sound Synthesis and Spatial Processing (SSSP)
In this first part we introduce techniques for sound synthesis and spatialization/reverberation; and for a high-end rendering of audio signals. Using numerical methods and signal processing tools, we discuss traditional techniques for musical sound synthesis such as wavetable, additive, subtractive, granular and nonlinear synthesis techniques; more advanced sound synthesis solutions such as modal synthesis and sound synthesis through physical modelling with particular reference to finite elements, digital waveguide networks, wave digital filters, etc. We also introduce techniques for digital audio enhancement/improvement such as sound effects, reverberation and spatialization. Finally, we discuss advanced sound rendering techniques based on multi-channel processing and spatial (array) processing such as ambisonics, acoustic beamforming/projection and wavefield synthesis.
- Sound synthesis: additive and subtractive synthesis, granular synthesis, wavetable synthesis, nonlinear distortion (ring modulation, phase/frequency modulation), fractal synthesis.
- Physical sound modelling for timbral synthesis: modal synthesis and functional transformation method, finite elements, Cordis-Anima, digital waveguides, wave digital structures, object-based modelling.
- High-end sound effects and sound processing: modulated digital delay lines, sound effects, reverberation and spatialization algorithms (DSP solutions, geometric methods, physical modelling methods, perceptual techniques), binauralization, head-related transfer function, multi-channel processing, ambisonics.
- Space-time processing and array processing: uniform and non-uniform speaker arrays, beamforming and acoustic projection, polar emission shaping, wavefield synthesis.