92858 - Autonomous and Adaptive Systems M

Learning outcomes

At the end of this course student will have a solid understanding of the state of the art and the key conceptual and practical aspects of the design, implementation and evaluation of intelligent machines and autonomous systems that learn by interacting with their environment. The course also takes into consideration ethical, societal and philosophical aspects related to these technologies.

Course contents

Introduction to the design of adaptive and autonomous systems: intelligent agents and intelligent machines, automatic vs autonomous decision-making.

Introduction to Reinforcement Learning (RL): multi-armed bandits, Montecarlo methods, tabular methods, approximation function methods, policy-based methods.

Applications of RL to games, classic control theory problems and robotics.

RL and neuroscience.

Introduction to algorithmic game theory for multi-agent learning systems: cooperation and coordination, social dilemmas, Multi-Agent Reinforcement Learning.

Bio-inspired adaptive systems. 

 Intelligent machines that create: Generative Learning, AI creativity, AI and the Arts.

The "brave new world": transformers, agents based on foundational models, training using Reinforcement Learning from human-feedback (RLHF).

Open problems and the future: safety, value alignment, super-intelligence, controllability, self-awareness.

Ethical implications of AI and autonomous systems. 

The course will include labs in which we will discuss implementation oriented aspects of the techniques and methodologies presented during the course.

 

Readings/Bibliography

During the course, the instructor will provide an extensive list of pointers (scientific papers, technical documentation, books, etc.) for each topic.

Useful textbooks for the course include the following:

• Francois Chollet. Deep Learning with Python. Second Edition. Manning. 2022.
  
• Dario Floreano and Claudio Mattiussi. Bio-inspired Artificial Intelligence: Theories, Methods and Technologies. MIT Press. 2008.
• David Foster. Generative Deep Learning: Teaching Machines to Paint, Write, Compose and Play. Second Edition. O’Reilly. 2023.
• Robin R. Murphy. Introduction to AI Robotics. Second Edition. MIT Press. 2019.
• Peter Norvig and Stuart J. Russell. Artificial Intelligence: A Modern Approach. Fourth Edition. Pearson. 2020.
• Max Pumperla and Kevin Ferguson. Deep Learning and the Game of Go. Manning. 2019.
  
• Richard R. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press 2018.

Teaching methods

Lectures in the classroom and in the lab.

Assessment methods

Oral exam (90%), with a compulsory research project to be submitted before the exam, and class participation (10%).

Students will be invited to read papers about state-of-the-art solutions in the area of Reinforcement Learning, Machine Intelligence and Autonomous Systems. The papers will be discussed during the lectures.

The project will include the design, implementation and evaluation of an AI algorithm/system (related to the topics of the module) and a written report with a structure similar to a research paper.

The project will have to be submitted before the closing date for the registration to the exam on AlmaEsami. Instructions for the submission of the project will be given during the module. The registration to the exam through AlmaEsami is compulsory.

Teaching tools

The instructor will provide the students with slides that will be made available ahead of the lectures.

All the teaching material of the course can be found here.

Office hours

See the website of Mirco Musolesi