CS433A - Parallel Programming
Prerequisites:
3-0-0-9
Course Contents
- Introduction: Why parallel computing; Ubiquity of parallel hardware/multi cores; Processes and threads; Programming models: shared memory and message passing; Speedup and efficiency; Amdahl’s Law.
- Introduction to parallel hardware: Multi cores and multiprocessors; shared memory and message passing architectures; cache hierarchy and coherence; sequential consistency.
- Introduction to parallel software: Steps involved in developing a parallel program; Dependence analysis; Domain decomposition; Task assignment: static and dynamic; Performance issues: 4C cache misses, inherent and artifactual communication, false sharing, computation to communication ratio as a guiding metric for decomposition, hot spots and staggered communication.
- Shared memory parallel programming: Synchronization: Locks and barriers; Hardware primitives for efficient lock implementation; Lock algorithms; Relaxed consistency models; High level language memory models (such Java and/or C++); Memory fences. Developing parallel programs with UNIX fork model: !PC with shared memory and message passing; UNIX semaphore and its aliornone semantic. Example case studies (see note below for some details). Developing parallel programs with POSIX thread library: Thread creation; Thread join; Mutex; Condition variables. Example case studies (see note below for some details). Developing parallel programs with Open MP directives: Parallel for; Parallel section; Static, dynamic, guided, and runtime scheduling; Critical sections and atomic operations; Barriers; Reduction. Example case studies (see note below for some details).
- Message passing programming: Distributed memory model; Introduction to message passing interface (MPI); Synchronization as Send/Recvpair; Synchronous and asynchronous Send fRecv; Collective communication: Reduce, Broadcast, Data distribution, Scatter, Gather; MPI derived data types. Example case studies (see note below for some details).
Topics
Current Course Information
Instructor(s):
Number of sections:
Tutors for each section:
Schedule for Lectures:
Schedule for Tutorial:
Schedule for Labs: