B8565 - AGENTIC AI SYSTEMS M

Learning outcomes

At the end of this course, the student will have a solid understanding of the state of the art and the key conceptual and practical aspects of the design, implementation, and evaluation of agentic AI systems. These are systems that can act autonomously, make decisions, and effectively interact with their environment to achieve goals. The student will gain in-depth knowledge of the latest advancements in agentic AI, including current research trends and emerging technologies, and develop practical skills and hands-on experience in designing such systems. The course will equip the student with the ability to critically assess the ethical, societal, and philosophical implications of these technologies, providing an interdisciplinary perspective to the field.

Course contents

Introduction to the design of agentic AI systems: intelligent agents and intelligent machines, automatic vs autonomous decision-making.

Introduction to Reinforcement Learning (RL): multi-armed bandits, Montecarlo methods, tabular methods, approximation function methods, and policy-based methods.

Applications of RL to games, classic control theory problems and robotics.

Introduction to algorithmic game theory for multi-agent learning systems: cooperation and coordination, social dilemmas, and Multi-Agent Reinforcement Learning.

Agents based on large language models/foundational models: training, post-training using Reinforcement Learning from human-feedback (RLHF), Direct Preference Optimization (DPO) and Reinforcement Learning from Verifiable Rewards (RLVR).

Intelligent machines that create: Generative Learning and AI creativity.

Open problems and the future: safety, value alignment, super-intelligence, controllability, and self-awareness.

Ethical and philosophical implications of agentic AI systems.

The course will include labs in which we will discuss implementation oriented aspects of the techniques and methodologies presented during the course.

Readings/Bibliography

During the course, the instructor will provide an extensive list of pointers (scientific papers, technical documentation, books, etc.) for each topic.

Useful textbooks for the course include the following:

• Christopher M. Bishop and Hugh Bishop. Deep Learning: Foundations and Concepts. Springer. 2023.
• Francois Chollet. Deep Learning with Python. Second Edition. Manning. 2022.
  
• Dario Floreano and Claudio Mattiussi. Bio-inspired Artificial Intelligence: Theories, Methods and Technologies. MIT Press. 2008.
• David Foster. Generative Deep Learning: Teaching Machines to Paint, Write, Compose and Play. Second Edition. O’Reilly. 2023.
• Robin R. Murphy. Introduction to AI Robotics. Second Edition. MIT Press. 2019.
• Peter Norvig and Stuart J. Russell. Artificial Intelligence: A Modern Approach. Fourth Edition. Pearson. 2020.
• Max Pumperla and Kevin Ferguson. Deep Learning and the Game of Go. Manning. 2019.
  
• Richard R. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press 2018.

Teaching methods

Lectures in the classroom and in the lab.

Assessment methods

The assessment methods of this course will be published and the presented to the students at the beginning of the module.

Teaching tools

The instructor will provide the students with slides that will be made available ahead of the lectures.

Office hours

See the website of Mirco Musolesi