B8562 - SISTEMI CONCORRENTI E PARALLELI M

Learning outcomes

At the end of the course, the student has an in-depth understanding of:
a) the organization of concurrent and parallel computing systems, in relation to the characteristics of the hardware used, with particular focus on multiprocessor architectures, multicomputers, and high-performance computing (HPC) systems;
b) the features and internal structure of operating systems that support the execution of concurrent and parallel applications;
c) the models and tools for process synchronization in both the shared memory model and the message-passing model;
d) the methodologies and programming languages for designing concurrent applications;
e) the methodologies and programming languages for designing parallel applications on high-performance computing (HPC) systems, with particular reference to the MIMD architectural model.

In particular, the student is able to solve synchronization problems among concurrent/parallel activities in both the shared memory and message-passing models, develop concurrent programs using high-level languages based on both shared variables and message passing, develop parallel programs on HPC systems—especially within the MIMD architectural model—and evaluate their performance.

Course contents

1. Introduction:

Concurrency and Parallelism

Models of interaction between concurrent/parallel processes:

• Shared memory

• Message passing

2. Architectures of Concurrent and Parallel Systems

• The Von Neumann model: features and limitations

• Extensions of the Von Neumann model: cache, ILP, and Hardware Multithreading

• Parallel architectures: multiprocessors, multicomputers, clusters, HPC systems, GPUs

• Flynn’s taxonomy

3. Concurrent Programming

• Introduction and definitions

• Algorithms and non-sequential processes

• Types of interaction

• Concurrent programming languages

• Properties of concurrent programs

3.1 Shared Memory Model

Insights on threads and synchronization:

• Thread characteristics and implementation

• Semaphores: properties, use cases, implementation

Use of concurrent programming languages in the shared memory model: the C/pthread library

3.2 Message Passing Model

• Recap on channels and primitives

• Communication primitives: semantics and implementation

• Guarded commands

• Rendezvous and remote procedure call

• Algorithms for synchronization in distributed environments

Use of concurrent programming languages in the message passing model: Google Go, Ada

4. Software Support for Concurrency

Virtualization:

• Hardware virtualization: goals and solutions

• Implementation of virtual machine monitors: virtualization and paravirtualization

• Analysis and experimentation with technologies: the Xen case

Kernels of multithreaded/multitask systems:

• Implementation of thread management and synchronization mechanisms within the kernel of a uniprocessor system

• Extension to the multiprocessor case: SMP model, loosely-coupled model.

5. HPC Systems and Parallel Programming

HW/SW features of HPC systems:

• Examples of real architectures

• Overview of operating system features for HPC infrastructures

• Tools for workload management in HPC systems

Methodologies and metrics for evaluating the performance of parallel applications:

• Speedup and efficiency

• Strong and weak scalability

• Amdahl’s and Gustafson’s laws

Parallel application development:

• Message passing: the MPI library

• Shared memory: OpenMP

• Hybrid parallel programming

Readings/Bibliography

• G.R.Andrews: "Foundations of Multithreaded, Parallel and Distributed Programming", Addison-Wesley
• Peter Pacheco, An Introduction to Parallel Programming, Morgan Kauffmann, 2011
• P.Ancilotti, M.Boari: 'Programmazione concorrente e distribuita', McGraw-Hill
• P.Ancilotti, M.Boari A.Ciampolini,G.Lipari: ' Sistemi Operativi', seconda edizione, McGraw-Hill
• S. Tanenbaum: 'I Moderni Sistemi Operativi', Jackson Libri
• A.Silberschatz, P.Galvin, G.Gagne: 'Applied Operating System Concepts' I edizione, John Wiley & Son, Inc.

Teaching methods

Lectures will be integrated with laboratory activities that will take place in the laboratories of the School of Engineering and Architecture.

In consideration of the teaching methods adopted, the attendance of this course requires the prior participation of all students in the training modules 1 and 2 on safety in the study places [https://elearning-sicurezza.unibo.it/], in e-learning mode.

Assessment methods

The exam consists of two mandatory parts:

• Laboratory test: development of concurrent/parallel applications.

• Oral examination.

The oral examination can only be taken after passing the laboratory test.

Teaching tools

Classroom lectures are presented with the aid of slides and a video projector.
The slides presented during lectures are made available to all students through the "Virtuale" platform.
Each student enrolled in the course is assigned a virtual machine to be used for self training at home.

Office hours

See the website of Anna Ciampolini