91255 - Statistical and Mathematical Methods for Artificial Intelligence

Learning outcomes

At the end of the course, the student masters the basic mathematical and statistical methods needed to acquire skills in artificial intelligence foundations, theories and applications.

Course contents

The course is organised into four modules. It introduces the mathematical and statistical tools required to analyse multivariate data, quantify uncertainty and construct predictive models. Theoretical concepts are accompanied by exercises, numerical examples and applications using R.

Module 1 – Basics of Linear Algebra

• Vectors and matrices: notation and basic operations
• Matrix multiplication and transposition
• Systems of linear equations
• Matrix inverse, determinant and trace
• Vector norms, distances and angles
• Linear combinations, span and linear independence
• Eigenvalues and eigenvectors
• Spectral decomposition of symmetric matrices
• Orthogonal projections and positive semi-definite matrices
• Quadratic forms and basic optimisation concepts

Module 2 – Principal Component Analysis

• Multivariate data and data matrices
• Centring and standardisation of variables
• Sample variance, covariance and correlation
• Covariance and correlation matrices
• Geometric interpretation of principal components
• Principal directions, loadings and scores
• Spectral decomposition of the covariance matrix
• Singular value decomposition
• Explained variance and selection of the number of components
• Graphical representations and interpretation of PCA results
• Projection onto a lower-dimensional space
• Low-rank reconstruction of the original data
• Implementation and interpretation of PCA in R

Module 3 – Introduction to Statistics

• Populations, samples and random sampling
• Events, probability rules and conditional probability
• Independence, partitions and Bayes’ theorem
• Random variables, probability mass functions, density functions and cumulative distribution functions
• Expectation, variance, covariance and correlation
• Quantiles and transformations of random variables
• Bernoulli, binomial, uniform, exponential and Gaussian distributions
• Multivariate random variables and the multivariate Gaussian distribution
• Empirical distributions and descriptive statistics
• Parameters, estimators and observed estimates
• Plug-in estimation and maximum likelihood estimation
• Properties of estimators: bias, variance, mean squared error and consistency
• Sampling distributions and asymptotic normality
• Monte Carlo simulation and evaluation of statistical procedures
• Exact, asymptotic and bootstrap confidence intervals

Module 4 – Regression Models

• Statistical relationships between variables
• Simple and multiple linear regression
• The linear regression model and its assumptions
• Least-squares estimation
• Matrix formulation of linear regression
• Interpretation of regression coefficients
• Fitted values, residuals and goodness of fit
• Statistical inference for regression coefficients
• Confidence intervals and prediction intervals
• Residual analysis and graphical diagnostics
• Leverage, influential observations and Cook’s distance
• Training error, test error and model assessment
• Cross-validation
• The bias–variance trade-off
• Model selection
• Regularised regression: ridge regression and lasso
• Fitting, evaluating and interpreting regression models in R

Readings/Bibliography

The class material is partly based on the book:

Deisenroth, M. P., Faisal, A. A., and Ong, C. S. (2020). Mathematics for Machine Learning. Cambridge University Press. ISBN: 978-1-108-45514-5.

The book is freely available online at: https://mml-book.github.io/book/mml-book.pdf

Teaching methods

Teaching is delivered through face-to-face, ex-cathedra lectures supported by HTML slides produced using R Markdown and made available on Virtuale.

Lectures combine explanations of the theoretical concepts with exercises and practical applications. The exercises are made available on Virtuale, discussed and solved in class, and their solutions are subsequently uploaded to Virtuale. Examples and applications using R are also presented during the lectures. The corresponding R scripts are provided so that students can reproduce the analyses independently.

One hour per week is devoted to an in-person test, followed by a discussion and correction of the proposed questions.

Assessment methods

Assessment is based on written, closed-book examinations. The use of books, notes or other supporting materials is not permitted. The questions and exercises are similar in type and level of difficulty to those discussed and solved during the course.

The final grade may be obtained by taking the written examination at the end of the semester or during either of the two subsequent resit sessions.

Students who wish to do so may replace the end-of-semester examination with continuous assessment based on the weekly tests taken during the semester. To select this option, students must submit a request in advance, towards the end of the course. The continuous-assessment grade is calculated as the average of the weekly test results, after excluding the single lowest result. A test missed because of absence is assigned a grade of zero before the lowest result is excluded.

Teaching tools

The teaching materials include HTML slides prepared with R Markdown, exercise sheets and their solutions, datasets, and R scripts used for the statistical examples and simulations presented in class.

R and RStudio are used for statistical computing, data analysis and graphical representations. All teaching materials are progressively made available to students through the Virtuale platform.

Office hours

See the website of Maria Pia Victoria Feser

See the website of Davide Evangelista