95661 - Advanced Machine Learning

Learning outcomes

The Advanced ML course is geared towards state of the art application of neural network to pricing and market risk problem. The studend will acquire a sound knowledge of the principles underlying Neural Networks and will be guided in a tour of the relevant literature concerning the exploitation of machine learning for pricing of highly exotic products and applications to market risk managment. Altough the approach demands very large scale computing facilities, impossible to be provided to the students, nonetheless students will learn how to design solutions to this type of problem and will gain hands on experience of the methodology on simpler and smaller toy models.

Course contents

Al termine del corso, gli studenti saranno in grado di:

comprendere come il testo possa essere rappresentato numericamente, dalle rappresentazioni sparse (bag-of-words, TF-IDF) fino agli embedding distribuiti e contestuali;

acquisire le basi minime di reti neurali necessarie per interpretare gli embedding, i modelli transformer e i large language model;

distinguere tra pipeline NLP classiche e workflow moderni basati su LLM, riconoscendo quando un metodo semplice resta appropriato e quando un sistema richiede effettivamente embedding, retrieval o orchestrazione agentica;

comprendere la logica della ricerca semantica, del retrieval denso, del reranking e della retrieval-augmented generation, inclusi i rischi di allucinazione e le strategie di grounding;

analizzare l'evoluzione dell'estrazione di informazioni verso l'estrazione strutturata guidata da schema e la document intelligence, con l'uso di strumenti e workflow agentici;

sviluppare una prospettiva critica su valutazione, affidabilità, limiti e casi d'uso appropriati dei sistemi linguistici in ambito finanziario quantitativo.

Readings/Bibliography

Here's a concise bibliography organized around the main threads of the course.

Foundations of NLP and Text Representation

Jurafsky, D. and Martin, J.H. (2024). Speech and Language Processing, 3rd edition (draft). Stanford University. Available at https://web.stanford.edu/~jurafsky/slp3/. The standard reference for classical and modern NLP, covering tokenization, n-grams, TF-IDF, naive Bayes, logistic regression, embeddings, transformers, and language models in a single coherent treatment.

Manning, C.D., Raghavan, P. and Schütze, H. (2008). Introduction to Information Retrieval. Cambridge University Press. Remains the definitive treatment of term weighting, cosine similarity, and vector-space models that underpin Lesson 1.

Embeddings and Neural Foundations

Mikolov, T., Chen, K., Corrado, G. and Dean, J. (2013). "Efficient Estimation of Word Representations in Vector Space." arXiv:1301.3781. The original Word2Vec paper.

Pennington, J., Socher, R. and Manning, C.D. (2014). "GloVe: Global Vectors for Word Representation." Proceedings of EMNLP, 1532–1543. Introduces the GloVe embedding model.

Goodfellow, I., Bengio, Y. and Courville, A. (2016). Deep Learning. MIT Press. Chapters 6–10 provide the neural-network background assumed in Lesson 2.

Transformers and Large Language Models

Vaswani, A. et al. (2017). "Attention Is All You Need." Advances in Neural Information Processing Systems 30, 5998–6008. The foundational transformer paper.

Devlin, J., Chang, M.-W., Lee, K. and Toutanova, K. (2019). "BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding." Proceedings of NAACL-HLT, 4171–4186.

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D. and Sutskever, I. (2019). "Language Models are Unsupervised Multitask Learners." OpenAI Technical Report. The GPT-2 paper that established the prompting paradigm.

Zhao, W.X. et al. (2023). "A Survey of Large Language Models." arXiv:2303.18223. A comprehensive survey covering architectures, training, alignment, and emergent capabilities.

Retrieval-Augmented Generation and Semantic Search

Lewis, P. et al. (2020). "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks." Advances in Neural Information Processing Systems 33, 9459–9474. The original RAG paper.

Gao, Y. et al. (2024). "Retrieval-Augmented Generation for Large Language Models: A Survey." arXiv:2312.10997. A thorough review of chunking, retrieval, reranking, and grounding strategies.

Agentic AI and Tool Use

Yao, S. et al. (2023). "ReAct: Synergizing Reasoning and Acting in Language Models." Proceedings of ICLR 2023. The key reference for reasoning-plus-action workflows.

Schick, T. et al. (2023). "Toolformer: Language Models Can Teach Themselves to Use Tools." arXiv:2302.04761. Introduces the concept of LLMs learning autonomous tool use.

NLP and LLMs in Finance

Xie, Q., Han, W., Zhang, X., Lai, Y. and Peng, M. (2023). "A Survey of Large Language Models for Financial Applications." arXiv:2311.02929. Covers sentiment analysis, information extraction, summarization, and financial reasoning with LLMs.

Loughran, T. and McDonald, B. (2011). "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks." The Journal of Finance, 66(1), 35–65. The classic paper on domain-specific sentiment in financial text, still relevant as a baseline reference.

Teaching methods

The course employs a blended learning approach that combines theoretical instruction with practical hands-on experience. The didactical methods can be summarized as follows:

1. Theoretical Lectures:
- Each topic begins with a comprehensive lecture that provides the necessary theoretical background. These sessions introduce key concepts, algorithms, and their relevance to finance, ensuring that students understand the principles behind machine learning models and techniques.
- Theoretical lectures cover both foundational topics (e.g., machine learning basics, financial data structures) and advanced subjects (e.g., deep learning, reinforcement learning, and NLP).

2. Practical Examples Using Python:
- Following each theoretical session, students engage in practical exercises using Python. These hands-on sessions allow students to apply the concepts learned in real-world financial contexts, such as stock price prediction, sentiment analysis, and trading strategies.
- The exercises use popular Python libraries like Pandas, Scikit-learn, TensorFlow, and Keras to demonstrate data processing, model building, and evaluation techniques.
- The integration of Python into the learning process is essential, as it provides students with practical coding experience and deepens their understanding of how to implement machine learning models effectively.

The combination of theory and practice ensures that students not only grasp the academic aspects of machine learning but also gain the technical skills necessary to apply these methods in their financial careers.

Assessment methods

The course concludes with a final practical project, which plays a key role in assessing the students' ability to apply the learned concepts. Students are required to develop a machine learning project related to finance, which will be discussed on the day of the examination.

Key details regarding the project:

- Project Approval: The project must be proposed and approved by the instructor at least one week before the examination date. This ensures that the chosen topic aligns with the course objectives and allows for timely feedback.

- Examination Discussion: During the final exam, students will present and discuss their project, demonstrating their understanding of the applied techniques, the financial data used, and the results obtained.

This approach integrates both theoretical knowledge and practical skills, encouraging students to independently explore machine learning applications in finance.

Teaching tools

The course utilizes a variety of teaching tools designed to enhance both theoretical understanding and practical application of machine learning techniques in finance. The main tools used throughout the course include:

1. Python Programs:
- Python is the primary programming language used for all practical exercises. Students will work with key libraries such as Pandas, Scikit-Learn, TensorFlow, and Keras to implement machine learning models. Python enables students to handle financial data, build predictive models, and apply advanced machine learning techniques in real-world financial scenarios.

2. Jupyter Notebooks:
- All practical sessions are conducted using Jupyter Notebooks, an interactive environment that allows students to write Python code alongside explanatory text and visualizations. This tool is essential for experimenting with different machine learning models, performing data analysis, and documenting results. It fosters an exploratory learning style where students can visualize the output of their code in real-time.

3. Lecture Notes:
- Detailed lecture notes accompany each theoretical session. These notes provide a structured overview of key concepts, algorithms, and case studies discussed during lectures. They are designed to serve as a reference for students as they work through both the theoretical and practical aspects of the course.

4. Data Files:
- All relevant data files required for practical sessions are provided in advance. These datasets cover various financial data types (e.g., time series, stock prices, sentiment analysis data) and are used to practice machine learning tasks such as regression, classification, and clustering.

Accessibility of Didactical Materials:
- All course materials, including lecture notes, data files, and Jupyter Notebooks, will be made available in advance through the university’s online platform for didactical materials.

Office hours

See the website of Giovanni Della Lunga