72778 - High-Performance Computing

Course Unit Page

Academic Year 2019/2020

Learning outcomes

At the end of the course, students will be acquainted with the fundamental programming techniques for high performance computer architectures. Students will be able to design, implement and benchmark parallel programs on shared-memory and distributed-memory systems.

Course contents

Syllabus

  • Parallel algorithms and parallel models of computation.
  • Introduction to HPC architectures: Flynn taxonomy, shared memory and distributed memory architectures; GPGPU
  • Parallel programming patterns (embarassingly parallel; stencil; work farm; scan; reduce)
  • Programming shared-memory architectures with C/OpenMP
  • Programming distributed-memory architectures with C/MPI
  • GPU programming with CUDA
  • SIMD programming using compiler intrinsics or auto-vectorization (GCC compiler)
  • Performance evaluation of parallel programs: measuring and understanding speedup and efficiency

Prerequisites

The course requires good programming skills with the C language in a Unix/Linux environment, and basic knowledge of computer architectures (at the level of what provided by the Computer Architectures course).

Readings/Bibliography

Selected parts from the following textbooks:

Teaching methods

The lectures introduce the foundations of parallel programming and several parallel programming costructs. Then, the same concepts are applied in the lab through a set of small programming assignments under the supervision of the instructor. The solution of every programming exercise is provided after the lab sessions, in order to support self-study.

Assessment methods

Written exam and programming project. The written exam consists of a set of questions (usually, four) on the topics addressed during the lectures. The written exam is closed-book and consists of open questions and/or multiple-choice quizzes. The maximum number of points granted by each correct answer is indicated in the exam sheet. The programming project requires the implementation of a parallel program according to  the specifications provided by the instructor; the programming project also requires the preparation of a written report. The project evaluation will consider the clarity, correctness and efficiency of the submitted program, and the quality of the accompanying report. The exam is passed with a grade equal to or greater than 18 on both the written and programming assignment. The final grade is computed as the weighted average of the grades of the written part (weight 4) and the programming assignment (weight 6).

Teaching tools

Lectures: slides and practical programming demos will often be used, integrated with exercises and Q&A sessions at the blackboard.

Lab: all programming activities will use the gcc compiler, the MPI library, and the CUDA/C compiler. All tools are freely available, in order to support self-study activities on own hardware, if available.

Links to further information

https://www.moreno.marzolla.name/teaching/HPC/

Office hours

See the website of Moreno Marzolla