92994 - OPTIMAL CONTROL M

Learning outcomes

The course deals with optimization-based control of dynamical systems, namely it provides theoretical and numerical methods to compute control system trajectories that minimize a performance index, and focuses on their application to trajectory optimization and maneuvering of Autonomous Systems. At the end of the course students will know how to (i) set-up an optimal control problem and characterize optimality conditions, (ii) develop numerical optimization methods to compute optimal, feasible trajectories, and (iii) design optimization-based receding-horizon control schemes for nonlinear systems. To bridge the gap between theory and application, students will apply the proposed techniques to trajectory optimization and maneuvering of Autonomous Systems in a number of application domains including Autonomous Vehicles, Robotic Systems (e.g., Aerial Robots) and other Mechatronic Systems.

Course contents

Introduction to optimal control
Motivating application domains and tasks for the optimal control of dynamical systems: dynamics exploration, maneuvering and trajectory optimization of Autonomous Vehicles, Robotic Systems (e.g., Autonomous Mobile Robots) and other Mechatronic Systems. Optimal control problem formulation. Examples of optimal control problems in the presented application domains.

Nonlinear optimization
Unconstrained optimization: problem formulation and optimality conditions. Special problem classes: convex optimization and quadratic optimization. Optimization (possibly nonconvex) over a convex set: optimality conditions. Constrained optimization: Lagrangian function, Lagrange multipliers, Karush-Kuhn-Tucker (KKT) optimality conditions. Optimization algorithms: descent (line-search) methods, gradient and Newton methods, projected gradient methods, barrier function methods, Sequential Quadratic Programming (SQP). Software tools and coding on case studies for nonlinear optimization.

Optimality conditions for optimal control
Nonlinear programming reformulation of optimal control and KKT optimality conditions. Unconstrained optimal control: reformulation via shooting, derivation of the reduced-cost gradient, Hamiltonian definition, necessary conditions for optimal control.

Linear Quadratic (LQ) optimal control
Finite horizon: problem formulation, necessary and sufficient conditions for optimality via Riccati equation, feedback structure of the optimal control. Infinite-horizon optimal control. Trajectory tracking via optimal control: Linear Quadratic Regulator (LQR). Continuous-time version of the LQ optimal control. Case studies in autonomous systems and robots. Software tools and coding on case studies.

Dynamic programming
Principle of optimality, value function and Bellman equation. Discrete-time Minimum Principle for optimal control. LQ optimal control via dynamic programming. Introduction to Reinforcement Learning.

Numerical methods for optimal control
Gradient method for optimal control. Differential Dynamic Programming (DDP): value function approximation, successive iterations, DDP algorithm. Barrier function method for constrained optimal control. SQP for optimal control. Software tools and coding on case studies for optimal control of autonomous systems and robots.

Applications of optimization-based control
Optimal control methods for maneuvering and trajectory optimization of autonomous systems and robots. Model Predictive Control. Software tools and coding on case studies from autonomous systems and robots.

Readings/Bibliography

The course is based on the books
“D. Bertsekas, Nonlinear Programming”
"D. Bertsekas, Dynamic Programming and optimal Control"
"A. E. Bryson, Y. Ho, Applied Optimal Control"

and a set of articles/notes which will be made available throughout the term.

Teaching methods

Frontal lectures, slides and hands-on software exercising

Assessment methods

Oral exam and discussion of a course project

Teaching tools

"Virtuale" (course content and material, useful info). Software tools for the design and simulation of optimal control methods

Office hours

See the website of Giuseppe Notarstefano