72778 - High-Performance Computing

Learning outcomes

At the end of the course, students will be acquainted with the fundamental programming techniques for high performance computer architectures. Students will be able to design, implement and benchmark parallel programs on shared-memory and distributed-memory systems.

Course contents

Syllabus

• Parallel algorithms and parallel models of computation.
  
• Introduction to HPC architectures: Flynn taxonomy, shared memory and distributed memory architectures; GPGPU
• Parallel programming patterns (embarassingly parallel; stencil; work farm; scan; reduce)
• Programming shared-memory architectures with C/OpenMP
• Programming distributed-memory architectures with C/MPI
• GPGPU programming
• Brief introduction to SIMD programming using compiler intrinsics or auto-vectorization (GCC compiler)
• Performance evaluation of parallel programs: measuring and understanding speedup and efficiency

Prerequisites

The course requires good programming skills with the C language in a Unix/Linux environment, and basic knowledge of computer architectures (at the level of what provided by the Computer Architectures course).

Readings/Bibliography

Selected parts from the following textbooks:

• Peter Pacheco, An Introduction to Parallel Programming, Morgan Kaufmann, 2011, ISBN 978-0123742605
• I. Foster. Designing and Building Parallel Programs. Addison-Wesley, 1995, ISBN 978-0201575941; Available online.
• Jason Sanders, Edward Kandrot, CUDA by Example: An Introduction to General-Purpose GPU Programming, Addison-Wesley, 2010, ISBN 978-0131387683
• Georg Hager, Gerhard Wellein, Introduction to High Performance Computing for Scientists and Engineers, CRC Press; 1 edition (July 2, 2010), ISBN 978-1439811924

Teaching methods

The lectures introduce the foundations of parallel programming and several parallel programming costructs. Then, the same concepts are applied in the lab through a set of small programming assignments under the supervision of the instructor. The solution of every programming exercise is provided after the lab sessions, in order to support self-study.

Due to the the type of activity and teaching methods adopted, participation to this course requires that students attend the training modules 1 and 2 on safety in the study places, through the e-learning platform https://elearning-sicurezza.unibo.it/.

Assessment methods

Written exam and programming project.The two parts are independent and can be given in any order.

The written exam consists of a set of questions (usually, four) on the topics addressed during the lectures. The exam is closed-book and consists of open questions and/or multiple-choice quizzes. The maximum number of points granted by each correct answer is indicated in the exam sheet.

The programming project requires the implementation of a parallel program according to  the specifications provided by the instructor; the programming project also requires the preparation of a written report. The project evaluation will consider the clarity, correctness and efficiency of the submitted program, and the quality of the accompanying report.

The exam is passed with a grade equal to or greater than 18 on both the written and programming assignment. The final grade is computed as the weighted average of the grades of the written part (weight 4) and the programming assignment (weight 6). Honors ("lode") will be awarded by the instructor to exams of outstanding quality.

Teaching tools

Lectures: slides and practical programming demos will often be used, integrated with exercises and Q&A sessions at the blackboard.

Lab: all programming activities will use the development tools provided by the Linux environment. Students will be given access to a compute server that can be accessed remotely. All tools are freely available, in order to support self-study activities on own hardware, if available.

Links to further information

https://www.moreno.marzolla.name/teaching/HPC/

Office hours

See the website of Moreno Marzolla