multi-core programming: basic concepts. copyright © 2006, intel corporation. all rights reserved....

Multi-core Programming: Basic Concepts

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.

2


Objectives

After completion of this module you will be familiar with the basic concepts of:

• threads • multithreaded programming

Note: • Threads will be “agents” doing work• No code examples are used



3


Agenda

Basics

Design concepts

Correctness concepts

Performance concepts



4


Agenda

Basics

• Processes and threads

• Parallelism and concurrency

Design concepts





5


Why Use Threads

Benefits• Increased performance

• Easy method to take advantage of multi-core

• Better resource utilization• Reduce latency (even on single processor systems)

• Efficient data sharing • Sharing data through memory more efficient than message-passing

Risks• Increases complexity of application• Difficult to debug (data races, deadlocks, etc.)

Processes & Threads



6


Processes and Threads

Modern operating systems load programs as processes

• Resource holder• Execution

A process starts executing at its entry point as a thread

Threads can create other threads within the process• Each thread gets its own stack

All threads within a process share code & data segments

Processes & Threads

Code segment

Data segment

thread

main()

…thread thread

Stack Stack

Stack



7


Concurrency vs. Parallelism

• Concurrency: two or more threads are in progress at the same time:

• Parallelism: two or more threads are executing at the same time

• Multiple cores needed

Thread 1Thread 1

Thread 2Thread 2

Thread 1Thread 1

Thread 2Thread 2

Concurrency vs. Parallelism



8


Agenda

Basics

Design concepts• Threading for functionality or performance? • Threading for throughput or turnaround?• Decomposing the work





9


Threading for Functionality

Assign threads to separate functions done by application• Easiest method since overlap is unlikely

Example: Building a houseBricklayer, carpenter, roofer, plumber,…

Threading for Functionality or Performance?



10


Threading for Performance

Increase the performance of computations

Thread in order to improve turnaround or throughput

Examples

• Automobile assembly line• Each worker does an assigned function

• Searching for pieces of Skylab• Divide up area to be searched

• US Postal Service• Post office branches, mail sorters, delivery




11


Turnaround

Complete single task in the smallest amount of time

Example: Setting a dinner table

• One to put down plates

• One to fold and place napkins

• One to place utensils• Spoons, knives, forks

• One to place glasses

Threading for Throughput or Turnaround?



12


Throughput

Complete the most tasks in a fixed amount of time

Example: Setting up banquet tables• Multiple waiters each do separate tables• Specialized waiters for plates, glasses, utensils, etc.




13


Task Decomposition

Divide computation based on natural set of independent tasks• Assign data for each task as needed

Example: Paint-by-Numbers• Painting a single color is a single task

• Number of tasks = number of colors

• Two artists: one does even, other odd

1

1 12

2

3

33

3

3

33

3 3

3

4

4

4

4

5 5 5 5 5

55

5

5

5 5 5 5 5

36

6

79

8

3

8

3

3

88

9

1

107

6

11

Task Decomposition



14


Data Decomposition

Large data sets whose elements can be computed independently

• Divide data and associated computation among threads

Example: Grading test papers• Multiple graders with same key

What if different keys are needed?

Data Decomposition



15


Agenda

Basics

Design concepts

Correctness concepts• Race Conditions and Synchronization• Deadlock




16


Race Conditions

Threads “race” against each other for resources• Execution order is assumed but cannot be guaranteed

Storage conflict is most common• Concurrent access of same memory location by multiple threads

• At least one thread is writing

Example: Musical Chairs

Race Conditions and Synchronization



17


Mutual Exclusion

Critical Region• Portion of code that accesses (reads & writes) shared variables

Mutual Exclusion• Program logic to enforce single thread access to critical region

• Enables correct programming structures for avoiding race conditions

Example: Safe Deposit box• Attendants ensure mutual exclusion




18


Synchronization

Synchronization objects used to enforce mutual exclusion

• Lock, semaphore, critical section, event, condition variable, atomic

• One thread “holds” sync. object; other threads must wait

• When done, holding thread releases object; some waiting thread given object

Example: Library book

• One patron has book checked out

• Others must wait for book to return




19


9

Barrier Synchronization

Threads pause at execution point• Threads waiting are idle; overhead

When all threads arrive, all are released

Example: Race starting line




20


Deadlock

Threads wait for some event or condition that will never happen

Example:• Traffic jam at intersection• Cars unable to turn or back up

What is Livelock?• Threads change state in

response to each other

Example:

• Robin Hood and Little John on log bridge

Deadlock



21


Agenda

Basics

Design concepts


Performance concepts• Speedup and Efficiency• Granularity and load balance



22


Speedup (Simple)

Measure of how much faster the computation executes versus the best serial code

• Serial time divided by parallel time

Example: Painting a picket fence• 30 minutes of preparation (serial)• One minute to paint a single picket• 30 minutes of cleanup (serial)

Thus, 300 pickets takes 360 minutes (serial time)

Speedup and Efficiency



23


Computing Speedup

What if fence owner uses spray gun to paint 300 pickets in one hour?• Better serial algorithm

• If no spray guns are available for multiple workers, what is maximum parallel speedup?

Number of painters

Time Speedup

1 30 + 300 + 30 = 360 1.0X

2 30 + 150 + 30 = 210 1.7X

10 30 + 30 + 30 = 90 4.0X

100 30 + 3 + 30 = 63 5.7X

Infinite 30 + 0 + 30 = 60 6.0X

Illustrates Amdahl’s Law

Potential speedup is restricted by serial portion




24


Efficiency

Measure of how effectively computation resources (threads) are kept busy

• Speedup divided by number of threads• Expressed as average percentage of non-idle time

Number of painters

Time Speedup Efficiency

1 360 1.0X 100%

2 30 + 150 + 30 = 210 1.7X 85%

10 30 + 30 + 30 = 90 4.0X 40%

100 30 + 3 + 30 = 63 5.7X 5.7%

Infinite 30 + 0 + 30 = 60 6.0X very low




25


Granularity

Loosely defined as the ratio of computation to synchronization

Be sure there is enough work to merit parallel computation

Example: Two farmers divide a field. How many more farmers can be added?

Granularity and Load Balance



26


Load Balance

Most effective distribution is to have equal amounts of work per thread

• Threads that finish first sit idle• Threads should finish close to same time

Example: Busing banquet tables• Better to assign same number of

tables to each bus person

Granularity and Load Balance



27


Multi-core Programming: Basic Concepts What’s Been CoveredProcesses and threads

Parallelism and concurrency

Design concepts• Threading for throughput or turnaround?• Threading for functionality or performance? • Decomposing the work

Correctness concepts• Race Conditions and Synchronization• Deadlock

Performance concepts• Speedup and Efficiency• Granularity and load balance



28




29


Alternative Slides



30


Turnaround


Example: Decorating a cake

• Frosting• One to frost top, one to frost sides

• Decoration• One to do borders• One to place flowers• One to write “Happy Birthday”




31


Throughput


Example: Decorating cakes

• Multiple bakers doing entire decoration

• Assembly line of frosters and decorators




32


Turnaround


Example: Parents to arrive in one hour

• Hugues cleans living room

• Otto cleans bathroom

• Thomas cleans kitchen




33


Throughput


Example: Produce pins “One man draws out the wire, another straights it, a third cuts it, a fourth points it, a fifth grinds it at the top for receiving the head;…” (A. Smith 1776)




34


Threading for Functionality

Assign threads to separate functions done by application• Easiest method since overlap is unlikely

Example: Coaching a football team• Head coach, position coaches, special teams coach




35


Deadlock

Threads wait for some event or condition that will never happen

Example:• Traffic jam at intersection• Cars unable to turn or back up

What is Livelock?• Threads change state in

response to each other

Example:

• Robin Hood and Little John on log bridge

Deadlock

multi-core programming: basic concepts. copyright © 2006, intel corporation. all rights reserved....

Documents

intel logo

basic conceptscopyright

basic conceptsprocesses

basic conceptsconcurrency

respective owners

united states

thread threads

stackall threads