parallel systems welcome to cs380prossbach/cs380p/lectures/01... · 2019. 12. 2. · outline for...
TRANSCRIPT
-
Parallel SystemsWelcome to cs380p
Chris Rossbach + Calvin Lin
CS380p
Introduction 1CS380P
-
Outline for Today
• Course Overview
• Course Details and Logistics
• Concurrency & Parallelism Basics• Motivation• Problem Decomposition
Acknowledgments: some materials in this lecture borrowed from or built on materials from:
• Emmett Witchel, who borrowed them from: Kathryn McKinley, Ron Rockhold, Tom Anderson, John Carter, Mike Dahlin, Jim Kurose, Hank Levy, Harrick Vin, Thomas Narten, and Emery Berger
• Mark Silberstein, who borrowed them from: Blaise Barney, Kunle Olukoton, Gupta
Introduction 2CS380P
-
Course Details
Course Name: CS380P – Parallel Systems
Lectures: Online
Class Web Page: http://www.cs.utexas.edu/users/rossbach/cs380p
Instructors: Chris Rossbach + Calvin Lin
Text: Principles of Parallel Programming (ISBN-10: 0321487907)
Please read the syllabus!
Introduction 3CS380P
http://www.cs.utexas.edu/users/rossbach/cs378hhttp://www.cs.utexas.edu/users/rossbachhttp://www.cs.utexas.edu/users/lin/https://www.universitycoop.com/search?keywords=Lin%20Snyder%20Parallel%20Programming
-
Why you should take this course
Introduction 4CS380P
-
Why you should take this course
• Parallelism is super-cool and super-important
Introduction 4CS380P
-
Why you should take this course
• Parallelism is super-cool and super-important
• You’ll learn important concepts and background
Introduction 4CS380P
-
Why you should take this course
• Parallelism is super-cool and super-important
• You’ll learn important concepts and background
• Have fun programming cool systems• GPUs! Multi-core!
• Modern infrastructure and programming languages
• Interesting synchronization primitives (not just about locks!)
Introduction 4CS380P
-
Why you should take this course
• Parallelism is super-cool and super-important
• You’ll learn important concepts and background
• Have fun programming cool systems• GPUs! Multi-core!
• Modern infrastructure and programming languages
• Interesting synchronization primitives (not just about locks!)
Two perspectives:• The “just eat your kale and quinoa” argument• The “it’s going to be fun” argument
Introduction 4CS380P
-
My first computer
Introduction 5CS380P
-
My first computer
Introduction 5CS380P
-
My first computer
CPU
Introduction 5CS380P
-
My first computer
CPU
Storage
Introduction 5CS380P
-
My first computer
CPU
Storage
Tape drive!(also good for playing heavy metal music)
Introduction 5CS380P
-
My first computer
CPU
Storage
screen
Tape drive!(also good for playing heavy metal music)
Introduction 5CS380P
-
My first computer
Wires + gobbledygook
CPU
Storage
screen
Tape drive!(also good for playing heavy metal music)
Introduction 5CS380P
-
My current computer
Introduction 6CS380P
-
My current computer
Too boring…
Introduction 6CS380P
-
Another of my current computers
Introduction 7CS380P
-
Another of my current computers
Introduction 7CS380P
-
Another of my current computers
GPU
Image DSP
Crypto
CPU
…
CPU
Introduction 7CS380P
-
Another of my current computers
GPU
Image DSP
Crypto
CPU
…
CPU
Introduction 7CS380P
-
Another of my current computers
GPU
Image DSP
Crypto
CPU
…
CPU
A lot has changed but…the common theme is…??
Introduction 7CS380P
-
Modern Technology Stack
Introduction 8CS380P
-
Modern Technology Stack
Introduction
CPU I/O dev DISK NIC
HW
8CS380P
-
Modern Technology Stack
Introduction
CPU I/O dev DISK NIC
HW
8CS380P
-
Modern Technology Stack
Introduction
CPU I/O dev DISK NIC
ApplicationsApplications
HW
8CS380P
-
Modern Technology Stack
Introduction
driver driver driver
CPU I/O dev DISK NIC
process files pipes
LIBC/CLR
process files pipes
ApplicationsApplications
user
kernel
OS-levelabstractions
HAL
user-modeRuntimes/libs
HW
8CS380P
-
Modern Technology Stack
Introduction
GPU
HW FPGA ASIC
DSPNVM CRYPT
8CS380P
-
Modern Technology Stack
Introduction
Vendor-specific
driver
GPU
ioctl
Runtime
deviceAPIs
ApplicationsApplications
user
kernel
HW
Runtimesupportmmap
FPGA ASIC
DSPNVM CRYPT
Runtime
deviceAPIs
Runtime
deviceAPIs
Runtime
deviceAPIs
8CS380P
-
Concurrency and Parallelism are Everywhere
Introduction
CPU DISKGPU
FPGA
ASIC
DSPNVM CRYPT
driver driver driver
ApplicationsApplications
Vendor-specific
driver
ioctl
Runtime
deviceAPIs
ApplicationsApplications
mmap
Runtime
deviceAPIs
Runtime
deviceAPIs
Runtime
deviceAPIs
9CS380P
-
HYPERVISORHYPERVISOR
vCPU vDISKvGPU
vFPGA
vASIC
vDSPvNVM vCRYPT
Concurrency and Parallelism are Everywhere
Introduction
CPU DISKGPU
FPGA
ASIC
DSPNVM CRYPT
driver driver driver
ApplicationsApplications
Vendor-specific
driver
ioctl
Runtime
deviceAPIs
ApplicationsApplications
mmap
Runtime
deviceAPIs
Runtime
deviceAPIs
Runtime
deviceAPIs
9CS380P
-
ApplicationsApplications
driver driver driver
vCPU vDISK
ApplicationsApplications
Vendor-specific
d r i v e r
vGPU
ioctl
Runtime
deviceAPIs
mmap
vFPGA
vASIC
vDSPvNVM vCRPT
Runtime
deviceAPIs
Runtime
deviceAPIs
Runtime
deviceAPIs
CPU DISKGPU
FPGA
ASIC
DSPNVM CRYPT
HYPERVISORHYPERVISOR
Introduction
Concurrency and Parallelism are Everywhere
10CS380P
-
Introduction
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
Concurrency and Parallelism are Everywhere
10CS380P
-
Introduction
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
Concurrency and Parallelism are Everywhere
10CS380P
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
-
Introduction
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
Concurrency and Parallelism are Everywhere
10CS380P
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
-
Introduction
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
Concurrency and Parallelism are Everywhere
10CS380P
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
-
Introduction
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
Concurrency and Parallelism are Everywhere
10CS380P
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
-
Introduction
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
Concurrency and Parallelism are Everywhere
10CS380P
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
-
Introduction
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
Concurrency and Parallelism are Everywhere
10CS380P
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
-
Introduction
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
Cluster OSCluster OS
Concurrency and Parallelism are Everywhere
10CS380P
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
-
Introduction
ApplicationsApplications
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
Cluster OSCluster OS
Concurrency and Parallelism are Everywhere
10CS380P
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
-
Introduction
ApplicationsApplications
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
Cluster OSCluster OS
Concurrency and Parallelism are Everywhere
10CS380P
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
driver driver driver
vCPUvDIS
K
Vendor-specific
d r i v e rvGP
U
ioctl
Runtime
deviceAPI
s
mmap
vFPGA
vASIC
vDSP
vNVM
vCRPT
Runtime
deviceAPI
s
Runtime
deviceAPI
s
Runtime
deviceAPI
s
CPU DISKGPU
FPGA
ASIC
DSPNVMCRYPT
HYPERVISORHYPERVISOR
-
Concurrency and Parallelism are everywhere
GPU
Image DSP
Crypto
CPU(s)
…
GPUCrypto
FPGADSP
Introduction 11CS380P
-
Concurrency and Parallelism are everywhere
GPU
Image DSP
Crypto
CPU(s)
…
GPUCrypto
FPGADSP
Introduction 11CS380P
-
Concurrency and Parallelism are everywhere
GPU
Image DSP
Crypto
CPU(s)
…
GPUCrypto
FPGADSP
Key concerns:
Introduction 11CS380P
-
Concurrency and Parallelism are everywhere
GPU
Image DSP
Crypto
CPU(s)
…
GPUCrypto
FPGADSP
Key concerns:• Concurrency/parallelism can’t be avoided
anymore (want a job?)• A program or two playing with locks and threads
isn’t enough
Course goal is to expose you to lots of ways of programming systems like these
Introduction 11CS380P
-
Goal: Make Parallelism Your Close FriendMethod: Use Many Different Approaches
Abstract Concrete
Locks and Shared Memory Synchronization Basic Locking
Prefix sum – pthreads
Language SupportGo lab: condition variables, channels, go routines
Rust lab: type-safety, 2PC
Parallel Architectures GPU Programming Lab
HPC MPI: Barnes-Hut lab
Modern/Advanced Topics
• Specialized Runtimes / Programming Models
• Auto-parallelization
• Race Detection
Introduction 12CS380P
-
Serial vs. Parallel Program
Introduction 13CS380P
-
Serial vs. Parallel Program
Introduction 13CS380P
-
Serial vs. Parallel Program
Introduction 13CS380P
-
Serial vs. Parallel ProgramKey concerns:
Introduction 13CS380P
-
Serial vs. Parallel ProgramKey concerns:• Programming model
Introduction 13CS380P
-
Serial vs. Parallel ProgramKey concerns:• Programming model• Execution Model
Introduction 13CS380P
-
Serial vs. Parallel ProgramKey concerns:• Programming model• Execution Model• Performance/Efficiency
Introduction 13CS380P
-
Serial vs. Parallel ProgramKey concerns:• Programming model• Execution Model• Performance/Efficiency• Exposing parallelism
Introduction 13CS380P
-
Technology Trends
Introduction 14CS380P
-
Introduction 15CS380P
-
Execution Models: Flynn’s Taxonomy
Introduction 16CS380P
-
Execution Models: Flynn’s Taxonomy
Introduction 16CS380P
-
Execution Models: Flynn’s Taxonomy
Normal Serial program
Introduction 16CS380P
-
Execution Models: Flynn’s Taxonomy
Uncommon architecture:Fault – tolerancePipeline parallelism Introduction 16CS380P
-
Execution Models: Flynn’s TaxonomyOur main focus
Introduction 16CS380P
-
SIMD
Introduction 17CS380P
-
SIMD
Introduction 17CS380P
-
• Example: vector operations (e.g., Intel SSE/AVX, GPU)
SIMD
Introduction 17CS380P
-
• Example: vector operations (e.g., Intel SSE/AVX, GPU)
SIMD
Introduction 17CS380P
-
MIMD
Introduction 18CS380P
-
MIMD
• Example: multi-core CPU
Introduction 18CS380P
-
MIMD
• Example: multi-core CPU
Introduction 18CS380P
-
Problem Partitioning
Introduction 19CS380P
-
Problem Partitioning
• Decomposition: Domain v. Functional
Introduction 19CS380P
-
Problem Partitioning
• Decomposition: Domain v. Functional
• Domain Decomposition (Data Parallel)• SPMD
• Input domain
• Output Domain
• Both
Introduction 19CS380P
-
Problem Partitioning
• Decomposition: Domain v. Functional
• Domain Decomposition (Data Parallel)• SPMD
• Input domain
• Output Domain
• Both
Introduction 19CS380P
-
Problem Partitioning
• Decomposition: Domain v. Functional
• Domain Decomposition (Data Parallel)• SPMD
• Input domain
• Output Domain
• Both
• Functional Decomposition (Task Parallel)• MPMD
• Independent Tasks
• Pipelining
Introduction 19CS380P
-
Problem Partitioning
• Decomposition: Domain v. Functional
• Domain Decomposition (Data Parallel)• SPMD
• Input domain
• Output Domain
• Both
• Functional Decomposition (Task Parallel)• MPMD
• Independent Tasks
• Pipelining
Introduction 19CS380P
-
Game of Life
Introduction 20CS380P
-
Game of Life
• Given a 2D Grid:
Introduction 20CS380P
-
Game of Life
• Given a 2D Grid:
What decomposition fits “best”?
• Domain (data parallel)
• Functional (task parallel)
Introduction 20CS380P
-
Domain decomposition
Introduction 21CS380P
-
Domain decomposition
Each CPU gets part of the input
Introduction 21CS380P
-
Domain decomposition
Each CPU gets part of the input
CPU 0
CPU 1
Introduction 21CS380P
-
Domain decomposition
Each CPU gets part of the input
CPU 0
CPU 1
For next time:
Introduction 21CS380P
-
Domain decomposition
Each CPU gets part of the input
CPU 0
CPU 1
For next time: • What issues/challenges might arise with this solution?
Introduction 21CS380P
-
Domain decomposition
Each CPU gets part of the input
CPU 0
CPU 1
For next time: • What issues/challenges might arise with this solution?• How could we do a functional decomposition?
Introduction 21CS380P