Paolo Baldini

Mitigating robots performance loss during the transition from simulation to reality through fine-tuning and online adaptation

Difficulty: medium

Scientific Impact: medium-low

This thesis proposal aims to address the problem of performance degradation that typically occurs when transitioning from a simulated environment to a real robot (a phenomenon known as the “reality gap”) through fine-tuning and online adaptation techniques. The methodology involves training a neural network to perform a specific mission in simulation, followed by a phase of behavior correction and optimization directly on the real system, based on an internal performance evaluation. The most critical aspect of the work will be identifying a mission in which both the performance degradation and the resulting margin for recovery achievable through adaptation are sufficiently significant to permit a robust validation of the proposed approach.

Automatic generation of reward functions for online adaptation

Difficulty: medium-high

Scientific Impact: medium-high

This thesis proposal aims to develop a method for the automatic generation of reward functions to be used in online adaptation / learning contexts, drawing inspiration from the principles of inverse reinforcement learning. The approach involves training a neural network capable of evaluating the state perceived by the robot and, potentially, the actions it takes in the environment, based on feedback provided by a system with complete knowledge of the state (oracle). The underlying idea of this project is to define a set of desirable and undesirable situations to be used as a reference for generating a system capable of providing feedback to the robot based on its behavior. The project is also expected to require a data collection phase using simulations. The main challenge of this proposal lies in the difficulty of predicting the critical issues that may arise during development, which is why the student is expected to commit more effort to this project than to other topics.

Behavioral Adaptation in Multi-Robot Systems Through Communication and Sharing of Information on Collective Performance

 Difficulty: medium-high

Scientific Impact: medium-high

The objective of this thesis proposal is to develop a communication mechanism among the members of a robot group, aimed at sharing information regarding the system’s collective performance, in order to enable behavioral adaptation even in cases where that overall performance cannot be estimated locally by each individual agent. In many collective robotics scenarios, in fact, a single robot has only a partial and local perception of the environment and the task at hand, which prevents it from autonomously assessing the overall effectiveness of the mission. The proposed methodology involves designing a communication protocol that allows individual agents to exchange relevant information (e.g., local performance estimates, perceived state, actions taken) in order to reconstruct, either in a distributed manner or through consensus, an approximate estimate of the group’s overall performance, to be used as a signal for the online adaptation of individual and/or collective behavior. The main challenges of this work lie in designing an effective and scalable communication mechanism that adapts as the number of robots changes, in managing partial, noisy, or potentially conflicting information, and in defining an adaptation strategy that is effective despite being based on an indirect and distributed estimate of overall performance, rather than on a centralized or directly observable assessment.