85197 - MODERN STATISTICS AND BIG DATA ANALYTICS

Learning outcomes

By the end of the course the student gains an understanding of theory and computing of modern statistical methods, with particular emphasis on methods for analysing large amounts of data (big data). More specifically, the student acquires knowledge on the most important methods of statistical learning and prediction and the skills required to solve real-world and decision-making problems.

Course contents

Cluster analysis: k-means, construction of distances, hierarchical clustering, partitioning around medoids, average silhouette width, mixture models, with algorithms, R-coding, theory, applications and in-depth discussion

Robust statistics: robustness concepts - outliers, influence function and breakdown point, robust estimation of mean and variance, outlier identification, robust regression, robust cluster analysis, with algorithms, R-coding, theory, applications and in-depth discussion

Dimension reduction: principal component analysis, variable selection in regression, cross-validation, Lasso, with algorithms, R-coding, theory, applications and in-depth discussion

Readings/Bibliography

Everitt, B. S., Landau, S., Leese, M., Cluster Analysis (fourth edition), E. Arnold 2001

Hennig, C., Meila, M., Murtagh, F., and Rocci, R., Handbook of Cluster Analysis, Taylor & Francis 2016.

Maronna, R. A., Martin R. D., Victor J. Yohai, Robust statistics : theory and methods, Wiley 2006.

Huber, P. J., Ronchetti, E. M., Robust Statistics (second edition), Wiley 2011.

Hastie, T., Tibshirani, R., Friedman, J., The Elements of Statistical Learning (second edition), Springer 2009.

Lecture Notes

Teaching methods

Classroom lessons, tutorials, computer workshop

Assessment methods

2 hours written exam. 5/30 marks can be earned from homework activity.

Teaching tools

Lecture Notes, supporting material provided on the web

Office hours

See the website of Christian Martin Hennig

See the website of Ivan Lorusso

See the website of Leonardo Bruni