B8565 - AGENTIC AI SYSTEMS M

Learning outcomes

At the end of this course, the student will have a solid understanding of the state of the art and the key conceptual and practical aspects of the design, implementation, and evaluation of agentic AI systems. These are systems that can act autonomously, make decisions, and effectively interact with their environment to achieve goals. The student will gain in-depth knowledge of the latest advancements in agentic AI, including current research trends and emerging technologies, and develop practical skills and hands-on experience in designing such systems. The course will equip the student with the ability to critically assess the ethical, societal, and philosophical implications of these technologies, providing an interdisciplinary perspective to the field.

Course contents

Prerequisites

There are no formal prerequisites. However, there is an expectation that students will have an in-depth knowledge of Machine Learning (including Deep Learning).

In addition, students should have a working knowledge of a deep learning framework, such as PyTorch or JAX.

 

Syllabus

Introduction to the design of agentic AI systems: analysis of the current technological and geopolitical scenario, and general definitions. 

Introduction to Reinforcement Learning (RL): multi-armed bandits, Montecarlo methods, tabular methods, function approximation methods, and policy approximation methods.

Applications of RL to games, classic control theory problems, and robotics.

Introduction to algorithmic game theory for multi-agent learning systems: cooperation and coordination, social dilemmas, and Multi-Agent Reinforcement Learning.

Agents based on large language models/foundational models: pre-training, post-training using Reinforcement Learning from human-feedback (RLHF), Direct Preference Optimization (DPO), Reinforcement Learning from Verifiable Rewards (RLVR), and reasoning models.

Intelligent machines that create: Generative Learning, agentic AI systems & AI creativity.

Open problems and the future: safety, value alignment, super-intelligence, controllability, and self-awareness. Ethical and philosophical implications of agentic AI systems.

The course will include labs in which we will discuss implementation oriented aspects of the techniques and methodologies presented during the course.

Readings/Bibliography

During the course, the instructor will provide an extensive list of pointers (scientific papers, technical documentation, books, etc.) for each topic.

Useful textbooks for the course include the following:

• Christopher M. Bishop and Hugh Bishop. Deep Learning: Foundations and Concepts. Springer. 2023.
  
• Dario Floreano and Claudio Mattiussi. Bio-inspired Artificial Intelligence: Theories, Methods and Technologies. MIT Press. 2008.
• David Foster. Generative Deep Learning: Teaching Machines to Paint, Write, Compose and Play. Second Edition. O’Reilly. 2023.
  
• Peter Norvig and Stuart J. Russell. Artificial Intelligence: A Modern Approach. Fourth Edition. Pearson. 2020.
• Max Pumperla and Kevin Ferguson. Deep Learning and the Game of Go. Manning. 2019.
  
• Richard R. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press 2018.

Teaching methods

Lectures in the classroom and in the lab.

Assessment methods

Oral or written exam on the theoretical aspects of the course (70%), with a compulsory research project to be submitted before the exam (30%), and class participation (bonus of up to 2 points). Students will be invited to read papers about the state-of-the-art in agentic AI systems.

The papers will be discussed during the lectures. The project will include the design, implementation and evaluation of an AI algorithm/system (related to the topics of the module) and a written report with a structure similar to a research paper. The project will have to be submitted before the closing date for the registration to the exam on AlmaEsami.

The topic(s) of the project will be given by the instructors of the module (i.e., it is not an "open" research problem). Instructions for the submission of the project will be given during the module. The registration to the exam through AlmaEsami is compulsory.

Teaching tools

The instructor will provide the students with slides that will be made available ahead of the lectures.

Office hours

See the website of Mirco Musolesi