parallel programming 0024 mergesort spring semester 2010
TRANSCRIPT
![Page 1: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/1.jpg)
Parallel ProgrammingParallel Programming00240024
MergesortMergesort
Spring Semester 2010Spring Semester 2010
![Page 2: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/2.jpg)
2
Outline
Discussion of last assignment
Presentation of new assignment Introduction to Merge-Sort Code Skeletons (see homepage) Issues on Parallelizing Merge-Sort Performance measurements
Questions/Comments?
![Page 3: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/3.jpg)
3
Discussion of Homework 3
![Page 4: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/4.jpg)
4
Part2 – First question
Why is it not sufficient to add the 'synchronized' keyword to the read() and write() methods to guarantee the specified behavior of the producer/consumer problem?
![Page 5: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/5.jpg)
5
Part2 – First question
Why is it not sufficient to add the 'synchronized' keyword to the read() and write() methods to guarantee the specified behavior of the producer/consumer problem?
Solution: Synchronization ensures that the producer and the consumer can not access the buffer at the same time. But it does not prevent the consumer to read a value more than one time or the producer to overwrite a value that was not read.
![Page 6: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/6.jpg)
6
Part2 – Second Question
Would it be safe to use a boolean variable as a "guard" within the read() and write() methods instead of using the synchronized keyword?
![Page 7: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/7.jpg)
7
Part2 – Second Question
Would it be safe to use a boolean variable as a "guard" within the read() and write() methods instead of using the synchronized keyword?
Solution: No, reading and writing a value is not atomic! – Can you tell me why, e.g., i++ is not atomic?
![Page 8: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/8.jpg)
8
Part3 – First Question
Would it suffice to use a simple synchronized(this) within the run()-method of each, the producer and the consumer to guard the updating of the buffer?
![Page 9: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/9.jpg)
9
Part3 – First Question
Would it suffice to use a simple synchronized(this) within the run()-method of each the producer and the consumer to guard the updating of the buffer?
No, since Producer and Consumer are different objects with different locks no mutual exclusion guaranteed
![Page 10: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/10.jpg)
10
Part3 – Second Question
What is the object that should be used as the shared monitor and (the object upon which the threads are synchronized())?
Solution: The shared instance of UnsafeBuffer.
Question: What could you have used instead?
![Page 11: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/11.jpg)
11
Part 3 – Third Question
What are the potential advantages/disadvantages of synchronizing the producer/consumer over synchronizing the buffer?
![Page 12: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/12.jpg)
12
Part 3 – Third Question
What are the potential advantages/disadvantages of synchronizing the producer/consumer over synchronizing the buffer?
Advantages: You can use arbitrary (also unsafe!) buffers You can do things in the Producer/Consumer that need to be done
before the other thread can use the buffer. (For example print something to the console).
Disadvantages: More work to do :-) More error-prone
![Page 13: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/13.jpg)
13
Presentation of Homework 4
![Page 14: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/14.jpg)
14
MergeSort
Problem: Sort a given list 'l' of 'n' numbers
Example: Input: 9 8 7 6 5 4 3 2 1 0 Output: 0 1 2 3 4 5 6 7 8 9
Algorithm: Divide l into two sublists of size n/2 Sort each sublist recursively by re-applying MergeSort Merge the two sublists back into one sorted list
End of recursion: Size of the sublist becomes 1 If size of a sublist > 1 => other sorting needed
![Page 15: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/15.jpg)
15
Example: Divide into sublists
5, 4, 3, 2, 1, 0
5, 4, 3 2, 1, 0
5, 4 3
5 4
2, 1 0
2 1
![Page 16: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/16.jpg)
16
Merging
Combine two sorted lists into one sorted list
Example: List 1: 0, 5 List 2: 3, 4, 45 Output: 0, 3, 4, 5, 45
Merging example: Create a list Output of size 5 0, 5 and 3, 4, 45 0 < 3 → insert 0 in Output 0, 5 and 3, 4, 45 3 < 5 → insert 3 in Output 0, 5 and 3, 4, 45 4 < 5 → insert 4 in Output 0, 5 and 3, 4, 45 5 < 45 → insert 5 in Output Finally, insert 45 in Output
![Page 17: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/17.jpg)
17
Example: Merging Sorted Sublists
0, 1, 2, 3, 4, 5
3, 4, 5 0, 1, 2
4, 5 3
5 4
1, 2 0
2 1
![Page 18: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/18.jpg)
18
The Code Skeletons(Eclipse)
![Page 19: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/19.jpg)
19
A Parallel MergeSort
Which operations can be done in parallel?
![Page 20: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/20.jpg)
20
A Parallel MergeSort
Which operations can be done in parallel? Sorting
Each sub-list can be sorted by a separate thread
![Page 21: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/21.jpg)
21
A Parallel MergeSort
Which operations can be done in parallel? Sorting
Each sub-list can be sorted by a separate thread
MergingTwo ordered sub-lists can be merged by a thread
![Page 22: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/22.jpg)
22
A Parallel MergeSort
Which operations can be done in parallel? Sorting
Each sub-list can be sorted by a separate thread
MergingTwo ordered sub-lists can be merged by a thread
Synchronization issues Limitations in parallelization?
![Page 23: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/23.jpg)
23
A Parallel MergeSort
Which operations can be done in parallel? Sorting
Each sub-list can be sorted by a separate thread
MergingTwo ordered sub-lists can be merged by a thread
Synchronization issues Limitations in parallelization?
Merge can only happen if two sublists are sorted
Performance issues Number of threads? Size of array to sort?
![Page 24: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/24.jpg)
24
Load balancing
• What if: size of array % numThreads != 0?
• Simple (proposed) solution– Assign remaining elements to one thread
• Balanced (more complicated) solution– Distribute remaining elements to more threads
![Page 25: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/25.jpg)
25
Performance Measurement
# of threads/array size
1 2 4 8 16 32 64 … 1024?
100,000 x
500,000 x
…
10,000,000?
![Page 26: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/26.jpg)
26
How to Measure Time?
• System.currentTimeMillis() might not be exact• Granularity might be higher than a millisecond
• Might be slightly inaccurate
• System.nanoTime()• Nanosecond precision, but not nanosecond accuracy
• For our measurements System.currentMillis() is good enough
![Page 27: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/27.jpg)
27
How to Measure Time?
long start, end;
start = System.currentMillis();
// some action
end = System.currentMillis();
System.out.println("Time elapsed: “
+ (end - start));
![Page 28: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/28.jpg)
28
Questions to be answered
• Is the parallel version faster?
• How many threads give the best performance?
• What is the influence of the CPU model/CPU frequency?
![Page 29: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/29.jpg)
29
The Harsh Realities of Parallelization Ideally
upgrading from uniprocessor to n-way multiprocessor should provide an n-fold increase in computational power
Real world most computations cannot be efficiently parallelized
• Sequential code, synchronization, communication
Speedup
– time(single processor) / time(n concurrent processors)
![Page 30: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/30.jpg)
Mein Tipp
Thread t = …
t.start();
…
try {
t.join(); // Warten bis t fertig ist
} catch (InterruptedException e) {
e.printStackTrace();
}
![Page 31: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/31.jpg)
Mein Tipp
int[] array
System.out.println(Arrays.toString(array));
Oder für die ersten paar Einträge
System.out.println(Arrays.toString(array).substring(0,20));
StringBuffer und StringBuilder sind schneller als String.
![Page 32: Parallel Programming 0024 Mergesort Spring Semester 2010](https://reader033.vdocuments.us/reader033/viewer/2022051401/5697bf701a28abf838c7dc46/html5/thumbnails/32.jpg)
32
Any Questions?