This is an elective discipline within the "Applied Data Analysis and Artificial Intelligence" Bachelor's program. The course is designed to provide students with a solid foundation in the analysis, processing, and generation of audio and speech signals. It focuses on bridging the gap between classical signal processing and modern deep learning approaches to build state-of-the-art AI systems for voice interaction and sound analysis.
Learning Objectives
The course establishes a foundation in the analysis, processing, and generation of audio and speech signals, bridging classical digital signal processing with contemporary deep learning approaches. Students develop the capacity to design, implement, and evaluate AI-driven systems for voice interaction and sound analysis, including automatic speech recognition, text-to-speech synthesis, and audio event detection.
Expected Learning Outcomes
Characterizes acoustic properties of phoneme classes and explains the source-filter model of speech production.
Computes spectrograms and extracts time-frequency representations from raw audio signals.
Applies sampling, quantization, and Fourier analysis techniques to speech and audio data.
Extracts MFCCs and spectral features from speech signals using Python libraries.
Implements a voice activity detector and evaluates its performance on noisy audio.
Constructs HMM-GMM acoustic models and interprets their parameters for speech recognition tasks.
Builds a neural speech recognition model using PyTorch on a conversational speech dataset.
Fine-tunes a pre-trained ASR model on non-English or domain-specific speech.
Evaluates ASR systems using word error rate and analyzes error patterns.
Compares concatenative, parametric, and neural TTS paradigms regarding naturalness and flexibility.
Implements a neural TTS pipeline that converts text to waveform audio.
Assesses synthesized speech quality using objective and subjective evaluation methods.
Classifies environmental sounds and music genres using convolutional and recurrent architectures.
Implements a sound event detection system that localizes events in time.
Applies data augmentation techniques to improve audio classification robustness.
Explains the challenges of speech processing in complex acoustic environments.
Implements a deep learning-based speech enhancement system for noise reduction.
Evaluates speech separation systems using signal-to-distortion ratio and related metrics.
Course Contents
Topic 1: Foundations of Speech and Audio Signals
Topic 2: Feature Extraction and Acoustic Modeling
Topic 3: Deep Learning for Automatic Speech Recognition
Topic 4: Text-to-Speech Synthesis
Topic 5: Audio Event Detection and Music Information Retrieval
Topic 6: Speech Enhancement and Separation
Assessment Elements
Project
Test
Exam
Interim Assessment
2026/2027 1st module
1 * Test
2026/2027 2nd module
0.6 * Project + 0.4 * Exam
Bibliography
Recommended Core Bibliography
The Handbook of Computational Linguistics and Natural Language Processing [Электронный ресурс] / ed. by Alexander Clark, Chris Fox, Shalom Lappin; DB ebrary. – Chichester: John Wiley & Sons, 2013. – 203 p. – Режим доступа: https://ebookcentral.proquest.com/lib/hselibrary-ebooks/reader.action?docID=4035461&query=computational+linguistics
Unpingco, J. Python for Signal Processing. – Springer International Publishing, 2014. – 128 pp.
Recommended Additional Bibliography
Kao, A., Poteet S. Natural Language Processing and Text Mining. - Springer, 2007. - ЭБС Books 24x7.
Shlomo Engelberg, Digital Signal Processing / Shlomo Engelberg, Ph.D. - Springer Verlag London Ltd, 2008 - URL: https://link.springer.com/book/10.1007%2F978-1-84800-119-0
Преподаватель
Иванько Денис
Course Syllabus
Abstract
Learning Objectives
Expected Learning Outcomes
Course Contents
Assessment Elements
Interim Assessment
Bibliography
Recommended Core Bibliography
Recommended Additional Bibliography
Authors