B8375 - MACHINE LEARNING PER L'ELABORAZIONE DEI SEGNALI LM

Learning outcomes

At the end of the course, students will possess the skills required to design and implement machine learning solutions in engineering contexts, with particular focus on signal processing and telecommunications. In particular, students will be able to:

• Apply machine learning models for signal compression, filtering, and prediction;
• Design and use machine learning algorithms to enhance the performance of signal processing and communication systems;
• Integrate machine learning techniques with traditional signal processing methods.
  

Course contents

Review of discrete-time signals: energy, power, autocorrelation, DTFT, and power spectrum. DFT, IDFT, matrix representation of the DFT, and FFT. Nonparametric spectral estimation: periodogram, zero-padding, spectral resolution, averaged periodogram, Welch’s method, and windowing. Time-frequency analysis using the STFT and spectrogram; trade-off between time resolution and frequency resolution.

Z-transform: region of convergence, properties, and inverse Z-transform. Discrete-time LTI systems: causality, BIBO stability, impulse response, poles and zeros. Digital IIR and FIR filters: properties, design of IIR and FIR filters, linear-phase FIR filters, equiripple method, and frequency-sampling method. Efficient implementation of filtering using the FFT: fast convolution, overlap-and-add, and fast correlation.

Adaptive filtering: Wiener filter, MMSE criterion, and LMS algorithm.

Introduction to machine learning for signal processing. From linear processing to nonlinear processing learned from data. Shallow and deep neural networks, activation functions, linear regions, and expressive power. Loss functions derived from the maximum likelihood principle: regression, binary classification, and multiclass classification. Cross-entropy and overview of the Kullback-Leibler divergence. Training via gradient descent, stochastic gradient descent, momentum, Adam, and backpropagation.

Regularization, overfitting, generalization, early stopping, dropout, data augmentation, Tikhonov regularization, LASSO, and Elastic-net. Bayesian interpretation of regularization via the MAP criterion.

Applications: channel equalization, echo cancellation, nonlinear classification, nonlinear regression, anomaly detection via autoencoders, and nonlinear time-varying filtering for neural denoising of audio signals using STFT and learned time-frequency masks.

 

Readings/Bibliography

• Simon J. D. Prince, Understanding Deep Learning, The MIT Press, 2023.
• D. J. C. MacKay, Information Theory, Inference and Learning Algorithms. Cambridge University Press, 2003.
• S. Theodoridis, Machine Learning: A Bayesian and Optimization Perspective, 2nd ed. Academic Press, 2020.
• C. M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
• T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. Springer, 2009.
• J. Watt, R. Borhani, and A. K. Katsaggelos, Machine Learning Refined: Foundations, Algorithms, and Applications, 2nd ed. Cambridge University Press, 2020.

Teaching methods

The course includes classroom lectures, in-depth seminars, and software-based demonstration sessions. During the lectures, the fundamental theoretical concepts are introduced and, where appropriate, complemented by numerical examples developed in Matlab, with the aim of connecting the mathematical treatment to the practical implementation of the algorithms.

The examples cover both classical techniques for numerical signal processing and machine learning techniques for signal processing. The application topics include: spectral estimation and time-frequency analysis, design and implementation of digital filters, efficient filtering using the FFT, adaptive filtering for channel equalization and echo cancellation, nonlinear classification and regression using neural networks, regularization, anomaly detection via autoencoders, and neural denoising of audio signals using STFT and learned time-frequency masks.

Teaching material, including Matlab examples and demonstration scripts, is made available to the students as support for individual study and for reproducing the experiments shown during the lectures.

Assessment methods

Assessment is based on an examination divided into two parts:

1. completion of an exercise related to the topics covered in the course;
2. oral examination, including discussion of the solution to the exercise and questions on the theoretical and methodological tools acquired during the course.

The exercise is intended to assess the student’s ability to apply the methods studied to the solution of problems in signal processing and machine learning for signal processing. The oral examination aims to verify the understanding of the fundamental concepts, the mastery of the mathematical and algorithmic tools presented in the course, and the ability to connect theoretical aspects with the applications discussed during the lectures.

The final grade, expressed on a scale of thirty, takes into account the overall performance in both parts of the examination.

Teaching tools

Lecture notes, slides, exercises, and code examples will be made available on the Virtuale platform.

Office hours

See the website of Andrea Giorgetti