58437 - Parallel Calculus

Learning outcomes

At the end of the course, students know the main high performance computing architectures; can design and analyze parallel algorithms for different models of parallel computation; can implement parallel algorithms on shared and distributed memory architectures using the most suitable languages, tools and technologies

Course contents

Syllabus

• Parallel algorithms and parallel models of computation.
  
• Introduction to HPC architectures: Flynn taxonomy, shared memory and distributed memory architectures; GPGPU
• Parallel programming patterns (embarassingly parallel; stencil; work farm; scan; reduce)
• Programming shared-memory architectures with C/OpenMP
• Programming distributed-memory architectures with C/MPI
• GPGPU programming
• Brief introduction to SIMD programming using compiler intrinsics or auto-vectorization (GCC compiler)
• Performance evaluation of parallel programs: measuring and understanding speedup and efficiency

Prerequisites

This course assumes the following prerequisites:

• Programming in C.
• Knowledge of the Unix/Linux environment (how to use the shell, how to write a basic Makefile, ecc.).
• Basic knowledge of computer architectures.

Readings/Bibliography

Selected parts from the following textbooks:

• Joseph Jaja, An Introduction to Parallel Algorithms, ‎Addison-Wesley, 1st edition, 1992, ISBN 978-0201548563.
• Peter Pacheco and Matthew Malensek, An Introduction to Parallel Programming, Morgan Kauìffman, 2nd edition, 2021, ISBN 9780128046050; the first edition is fine, too: Peter Pacheco, An Introduction to Parallel Programming, Morgan Kaufmann, 2011, ISBN 978-0123742605
• Jason Sanders, Edward Kandrot, CUDA by Example: An Introduction to General-Purpose GPU Programming [https://developer.nvidia.com/cuda-example], Addison-Wesley, 2010, ISBN 978-0131387683
• Georg Hager, Gerhard Wellein, Introduction to High Performance Computing for Scientists and Engineers, CRC Press; 1 edition (July 2, 2010), ISBN 978-1439811924

Additional teaching material (including lecture slides and programming exercises) will be made available on the course web site.

Teaching methods

The lectures introduce the foundations of parallel programming and several parallel programming costructs. Then, the same concepts are applied during practice sessions (BYOD-Bring Your Own Device) through a set of small programming assignments under the supervision of the instructor. Solutions are provided after the lab sessions, order to support self-study.

Due to the the type of activity and teaching methods adopted, participation to this course requires that students attend the training modules 1 and 2 on safety in the study places, through the e-learning platform https://elearning-sicurezza.unibo.it/ .

Assessment methods

Written exam and programming project. The two parts are independent and can be given in any order.

The written exam consists of a few questions on the topics addressed during the lectures. The exam is closed-book and consists of open questions and/or multiple-choice quizzes. The maximum number of points granted by each correct answer is indicated in the exam sheet.

The programming project requires the implementation of a parallel program according to the specifications provided by the instructor; the programming project also requires the preparation of a written report. The project evaluation will consider the clarity, correctness and efficiency of the submitted program, and the quality of the accompanying report.

The exam is passed with a grade equal to or greater than 18 on both the written and programming assignment. The final grade is computed as the weighted average of the grades of the written part (weight 0.4) and the programming assignment (weight 0.6). Honors ("lode") will be awarded by the instructor to exams of outstanding quality.

Teaching tools

Lectures: slides and practical programming demos will often be used, integrated with exercises and Q&A sessions at the whiteboard.

Practice: all programming activities will use the development tools provided by the Linux environment. Students will be given access to a compute server. All tools are freely available, in order to support self-study activities on own hardware, if available.

Links to further information

https://www.moreno.marzolla.name/teaching/CalPar/

Office hours

See the website of Moreno Marzolla