computer science and engineering copyright by hesham el-rewini advanced computer architecture

Computer Science and EngineeringCopyright by Hesham El-Rewini

Advanced Computer Advanced Computer ArchitectureArchitecture


Grosch’s Law Moore’s Law Von Neumann’s Bottlneck Parallelism Speedup Amdahl’s Law The Gustafson-Barsis Law Benchmarks

Performance Evaluation


Grosch’s Law (1960s)

“To sell a computer for twice as much, it must be four times as fast”

Vendors skip small speed improvements in favor of waiting for large ones

Buyers of expensive machines would wait for a twofold improvement in performance for the same price.


Moore’s Law

Gordon Moore (cofounder of Intel) Processor performance would double

every 18 months This prediction has held for several

decades Unlikely that single-processor

performance continues to increase indefinitely


Von Neumann’s bottleneck

Great mathematician of the 1940s and 1950s

Single control unit connecting a memory to a processing unit

Instructions and data are fetched one at a time from memory and fed to processing unit

Speed is limited by the rate at which instructions and data are transferred from memory to the processing unit.


Problem

Assume that a switching component such as a transistor can switch in zero time. We propose to construct a disk-shaped computer chip with such a component. The only limitation is the time it takes to send electronic signals from one edge of the chip to the other. Make the simplifying assumption that electronic signals travel 300,000 kilometers per second. What must be the diameter of a round chip so that it can switch 109 times per second? What would the diameter be if the switching requirements were 1012 time per second?


Parallelism

Multiple CPUs

Within the CPU One Pipeline Multiple pipelines


Superscalar Parallelism

Scheduling


Past Trends in Parallel Architecture (inside the box)

Completely custom designed components (processors, memory, interconnects, I/O) Longer R&D time (2-3 years) Expensive systems Quickly becoming outdated

Bankrupt companies!!


New Trends in Parallel Architecture (outside the box)

Advances in commodity processors and network technology

Network of PCs and workstations connected via LAN or WAN forms a Parallel System

Network Computing Compete favorably (cost/performance) Utilize unused cycles of systems

sitting idle


Speedup

S = Speed(new) / Speed(old)

S = Work/time(new) / Work/time(old)

S = time(old) / time(new)

S = time(before improvement) / time(after improvement)


Speedup

Time (one CPU): T(1)

Time (n CPUs): T(n)

Speedup: S

S = T(1)/T(n)


Amdahl’s Law

The performance improvement to be gained from using some faster mode of execution is limited by the fraction of the time the faster mode can be used


20 hours

200 miles

A B

Walk 4 miles /hourBike 10 miles / hourCar-1 50 miles / hourCar-2 120 miles / hourCar-3 600 miles /hour

must walk

Example


20 hours

200 miles

A B

Walk 4 miles /hour 50 + 20 = 70 hours S = 1Bike 10 miles / hour 20 + 20 = 40 hours S = 1.8Car-1 50 miles / hour 4 + 20 = 24 hours S = 2.9Car-2 120 miles / hour 1.67 + 20 = 21.67 hours S = 3.2Car-3 600 miles /hour 0.33 + 20 = 20.33 hours S = 3.4

must walk

Example


Amdahl’s Law (1967)

: The fraction of the program that is naturally serial

(1- ): The fraction of the program that is naturally parallel


S = T(1)/T(N)

T(N) = T(1) + T(1)(1- )

N

S = 1

+ (1- )

N

=N

N + (1- )


Amdahl’s Law


Gustafson-Barsis Law

N & are not independent from each other

T(N) = 1

T(1) = + (1- ) N

S = N – (N-1)

: The fraction of the program that is naturally serial


Gustafson-Barsis Law


Distributed Computing Performance

Single Program Performance

Multiple Program Performance


Benchmark Performance

Serial Benchmarks Parallel Benchmarks PERFECT Benchmarks NAS Kernel The SLALOM The Golden Bell Prize WebSTONE for the Web Performance Comparisons

computer science and engineering copyright by hesham el-rewini advanced computer architecture

Documents