Early Adopter

Introduction to Parallel Computing: Research Intensive University:

4th Year Elective

Bo HongElectrical and Computer Engineering

Georgia Institute of Technology

2 NSF/IEEE-TCPP PDC Curriculum

Syllabus – Overview (1)

• Format– A 4th year elective course on parallel computing.

• Target Audience– students who have background in computer

organization and programming, and would like to learn parallel computing.

3 NSF/IEEE-TCPP PDC Curriculum

Syllabus – Overview (2)

• Topical Coverage:– Parallel computing systems– Parallel programming– Parallel algorithms

• Expected Outcomes:– Upon completion of the course, the students are

expected to (1) understand parallel computing systems, and (2) be able to design algorithms and implement programs for such systems.

4 NSF/IEEE-TCPP PDC Curriculum

Syllabus – Topical Coverage (1)

Topics BLOOM# Hours Source

Parallel Architectures

Stacks and thread execution K 1 Addition to the proposed curriculum

Multi-threading on shared memory computers

C 1 Addition to the proposed curriculum

Memory Hierarchy and cache coherency

K 1 NSF-TCPP Curriculum Architecture (C)

Snoopy based cache coherency

C 2 NSF-TCPP Curriculum Architecture (N)

Directory based cache coherency

A 2 NSF-TCPP Curriculum Architecture (N)

Interconnect, topologies and routing

A 1 NSF-TCPP Curriculum Architecture (C/K)

Massive multi-threading and GPU

K 2 Addition to the proposed curriculum

5 NSF/IEEE-TCPP PDC Curriculum

Syllabus – Topical Coverage (2)

Topics BLOOM# Hours Source

Parallel Programming

SIMD K 1 NSF-TCPP Curriculum Programming (K)

Shared Memory K 2 NSF-TCPP Curriculum Programming (A)

Message Passing K 2 NSF-TCPP Curriculum Programming (C)

Synchronization A 1 NSF-TCPP Curriculum Programming (A)

Pthread Library K 3 NSF-TCPP Curriculum Programming (C)

MPI K 3 Briefly mentioned in the proposed curriculum

CUDA K 3 Briefly mentioned in the proposed curriculum

6 NSF/IEEE-TCPP PDC Curriculum

Syllabus – Topical Coverage (3)

Topics BLOOM# Hours Source

Parallel Algorithms

Asymptotics K 1 NSF-TCPP Curriculum Algorithm (C)

Speedup, efficiency, cost, and scalability

K 2 NSF-TCPP Curriculum Algorithm (C/K/N)

Broadcast, reduction, scatter and gather

C 3 NSF-TCPP Curriculum Algorithm (C/A)

Matrix multiplication K 3 NSF-TCPP Curriculum Algorithm (optional)

Solving linear systems K 2 NSF-TCPP Curriculum Algorithm (optional)

Sorting (merge, bitonic, quick) K/C 3 NSF-TCPP Curriculum Algorithm (C)

Graphs (minimum spanning tree)

C 2 Addition to the proposed curriculum

FFT K/C 3 Addition to the proposed curriculum

7 NSF/IEEE-TCPP PDC Curriculum

Teaching Experience: Melding with Old Topics• Students knew uni-thread execution mechanisms

before coming to the course.• The lectures analyzed the role of stacks and address

spaces for uni-threads, and then extended the analysis to multiple threads.

• Students found it easy to understand the execution of multi-threaded programs, which further extends to other parallel computing models.

8 NSF/IEEE-TCPP PDC Curriculum

Evaluation (1)

• Pedagogy

1. Gradual transition from sequential experience to parallel thinking

2. Explanation of abstract concepts via concrete examples (e.g. cache coherency)

• Student feedbacks– Item 1 is important for accepting parallel thinking– Item 2 is helpful for understanding of the materials

9 NSF/IEEE-TCPP PDC Curriculum

Evaluation (2)

• Learning outcome– Evaluated through programming projects

(pthread/MPI/CUDA intro, finding prime numbers, solving linear systems, sorting) and a final survey on current topics

• With the course, the students– can approach a problem with parallel thinking– understand the mechanisms of different parallel

architectures– can design and analysis simple parallel algorithms– can implement and evaluate parallel algorithms using

pthread, MPI, and CUDA

10 NSF/IEEE-TCPP PDC Curriculum

Evaluation (3)

• Appropriateness of topical overages– Students find that

• The topics are comprehensive, and it is appreciated• Some topics are complicated (cache coherency, CUDA

programming, synchronization, etc)– Students suggest that

• Add some brief discussions on the current state-of-the-art of parallel computing

• Include more hands-on programming projects to help understand the concepts

11 NSF/IEEE-TCPP PDC Curriculum

Deviation from the Proposed Curriculum and the Impact• Additions to the curriculum

– Thread execution mechanisms• facilitates the sequential-to-parallel transition

– Massive multi-threading and GPU• helps cover the emerging field of GPU computing

– Minimum spanning tree and FFT• discusses two useful applications of parallel computing

• Expected level of understanding of the topics– Slight adjusted to fit the comprehensive introductory

course

12 NSF/IEEE-TCPP PDC Curriculum

Summary

• With the introductory course, the students achieved the expected learning outcome.

• The comprehensiveness and level of difficulty of the course are appreciated by the students.