Home » CS at DA­IICT » Courses » Semester V » High Performance Computing

High Performance Computing

 

High Performance Computing Course format

3 hours lecture per week and 3 hours lab per week.

Course Placement: HPC is a core course offered to third year students of B.Tech Hons. ICT (Minor in Computational Science) program.

Curriculum Details This course is an introduction to parallel computing and aims at teaching basic models of parallel programming including the principles of parallel algorithm design, parallel computer architectures, performance considerations, programming models for shared and distributed-memory systems, message passing programming models used for cluster computing along with some important algorithms for parallel systems. Finally a brief overview of applications of HPC and future trends.

Major part of the course includes Lab Component for actual implementation after learning the basics of Parallel programming and HPC.

Course details summarized in Appendix.

 

Appendix: Detailed Course Contents
Topic Name Content (includes lab)  
Introduction to parallel programming. Overview of latest parallel machines and architecture. Introduction to high performance computing. Parallel Programming concepts. Need for Parallel Computing. Limitations. Parallel programming languages. Parallel libraries. Amdahl’s law, speed up. Basics of parallelization.
Optimization and Performance considerations. Improving performance on a single processor: basic optimization techniques for serial code. Measuring performance, parallel-serial problem breakdown, bandwidth measures, thread synchronization, Memory structures and bandwidth optimization, performance improvements. Optimization, performance analyzer tools, debugging.

Amdahl’s law, Gustafson’s Law, Karp–Flatt metric, isoefficiency metric.

Shared Memory parallel programming Multi-threading model using OpenMP.

OpenMP: parallel do, private variables, nested loops, reductions, loop dependencies, thread-safe functions, parallel sections, and barriers.

Distributed memory parallel programming Message Passing Programming (important implementations using MPI.)  MPI send and receive, MPI communicators, broadcast, reduce. Performance properties.
Case Studies Several Important Parallel Algorithms and implementation strategies from different class of problems such as

Integration using trapezoidal rule.

Calculation of PI using monte carlo method.

Matrix operations. Inclusive and exclusive scan. Fibonacci series, Image processing. Cellular automata. Sorting algorithms. Solution of Differential Eqns using Finite Difference etc.

Hybrid parallelization with MPI and OpenMP.

 

Assessment method

  • Two mid-semester examinations,
  • 8 lab assignments,
  • Final project and presentation.

HPC Projects (Compulsory): Individual/ Group

 Lectures will be supplemented through some ppt slides for the important concepts.

References:

  1. An Introduction to Parallel Programming; Elsevier; by Peter S. Pacheco.
  2. Scientific Parallel Computing; Princeton University Press ; by Babak Bagheri Terry Clark L Ridgway Scott Bagheri Clark Scott
  3. PARALLEL PROGRAMMING; Barry Wilkinson, Pearson Education.
  4. Introduction to High Performance Computing for Scientists and Engineers; G. Hager & G. Wellein. CRC Press.
  5. Algorithms Sequential & Parallel: A Unified Approach,  by Millers Russ; Cengage, ISBN 9788131525050
  6. Parallel Programming in C with MPI and OpenMP; by Michael J. Quinn ; McGraw-Hill Higher Education
  7. Parallel Computing Theory and Practice. By Michael J. Quinn; McGraw Hill Education (India).