CISC 874: Introduction to Parallel Computing

Catalog Description:
Overview of high performance machine architectures, parallel program performance measurement, different paradigms for achieving parallelism including automatic parallelization and vectorization, fine grain parallelism for high performance workstations, data parallel programming, shared memory and message passing programming, and SPMD versus MIMD; task scheduling.

Current Text:
Designing and Building Parallel Programs
Ian Foster
Addison Wesley, 1995.

Goals:
This course is intended as an introduction and overview of the SOFTWARE aspects of parallel computing. The course will concentrate on the issues involved in effectively exploiting the potential parallelism in high performance computing systems through parallel programming. The course will contain hands-on experience programming and analyzing the performance of parallel programs on a cluster of workstations. The course will not cover the details of automatic parallelizers; this material is covered in CIS 872. The course will only briefly discuss parallel architectures; this material is covered in depth in CIS 662. The course will not concentrate on developing and analyzing parallel algorithms; this material is covered in depth in CIS 821.

Content:

• Introduction and Overview
• Parallel Architectures and Paradigms of Parallelism
• Basic Steps of Designing Parallel Programs
• Performance Evaluation
• Distributed Memory, Message-Passing Paradigm and MPI
• Data Parallel Programming and HPF
• Shared Memory MIMD Parallel Paradigm
• Distributed Shared Memory Systems
• Task and Loop Scheduling
• Debugging Parallel Programs
• Alternate Parallel Programming Paradigms

Required Background: CISC 672 or CISC 471 (or equivalent).

Recommended Background: CISC 662 (Computer Architecture).