chapter 6 warehouse-scale computers to exploit request-level …harmanani.github.io › classes ›...
TRANSCRIPT
The University of Adelaide, School of Computer Science 22 November 2018
Chapter 2 — Instructions: Language of the Computer 1
1Copyright © 2019, Elsevier Inc. All rights Reserved
Chapter 6
Warehouse-Scale Computers to Exploit Request-Level and Data-Level Parallelism
Computer ArchitectureA Quantitative Approach, Sixth Edition
2Copyright © 2019, Elsevier Inc. All rights Reserved
Introductionn Warehouse-scale computer (WSC)
n Provides Internet servicesn Search, social networking, online maps, video sharing, online
shopping, email, cloud computing, etc.n Differences with HPC “clusters”:
n Clusters have higher performance processors and networkn Clusters emphasize thread-level parallelism, WSCs
emphasize request-level parallelismn Differences with datacenters:
n Datacenters consolidate different machines and software into one location
n Datacenters emphasize virtual machines and hardware heterogeneity in order to serve varied customers
Introduction
The University of Adelaide, School of Computer Science 22 November 2018
Chapter 2 — Instructions: Language of the Computer 2
3Copyright © 2019, Elsevier Inc. All rights Reserved
Introductionn Important design factors for WSC:
n Cost-performancen Small savings add up
n Energy efficiencyn Affects power distribution and coolingn Work per joule
n Dependability via redundancyn Network I/On Interactive and batch processing workloads
Introduction
4Copyright © 2019, Elsevier Inc. All rights Reserved
Introductionn Ample computational parallelism is not important
n Most jobs are totally independentn “Request-level parallelism”
n Operational costs countn Power consumption is a primary, not secondary, constraint when
designing systemn Scale and its opportunities and problems
n Can afford to build customized systems since WSC require volume purchase
n Location countsn Real estate, power cost; Internet, end-user, and workforce availability
n Computing efficiently at low utilizationn Scale and the opportunities/problems associated with scale
n Unique challenges: custom hardware, failuresn Unique opportunities: bulk discounts
Introduction
The University of Adelaide, School of Computer Science 22 November 2018
Chapter 2 — Instructions: Language of the Computer 3
5Copyright © 2019, Elsevier Inc. All rights Reserved
Efficiency and Cost of WSC
n Location of WSCn Proximity to Internet backbones, electricity cost,
property tax rates, low risk from earthquakes, floods, and hurricanes
n Power distribution
Efficiency and Cost of W
SC
6Copyright © 2019, Elsevier Inc. All rights Reserved
Prgrm’g Models and Workloadsn Batch processing framework: MapReduce
n Map: applies a programmer-supplied function to each logical input record
n Runs on thousands of computersn Provides new set of key-value pairs as intermediate values
n Reduce: collapses values using another programmer-supplied function
Programm
ing Models and W
orkloads for WSC
s
The University of Adelaide, School of Computer Science 22 November 2018
Chapter 2 — Instructions: Language of the Computer 4
7Copyright © 2019, Elsevier Inc. All rights Reserved
Prgrm’g Models and Workloads
n Example:n map (String key, String value):
n // key: document namen // value: document contentsn for each word w in value
n EmitIntermediate(w,”1”); // Produce list of all words
n reduce (String key, Iterator values):n // key: a wordn // value: a list of countsn int result = 0;n for each v in values:
n result += ParseInt(v); // get integer from key-value pairn Emit(AsString(result));
Programm
ing Models and W
orkloads for WSC
s
8Copyright © 2019, Elsevier Inc. All rights Reserved
Prgrm’g Models and Workloads
n Availability:n Use replicas of data across different serversn Use relaxed consistency:
n No need for all replicas to always agree
n File systems: GFS and Colossusn Databases: Dynamo and BigTable
Programm
ing Models and W
orkloads for WSC
s
The University of Adelaide, School of Computer Science 22 November 2018
Chapter 2 — Instructions: Language of the Computer 5
9Copyright © 2019, Elsevier Inc. All rights Reserved
Prgrm’g Models and Workloadsn MapReduce runtime environment
schedules map and reduce task to WSC nodesn Workload demands often vary considerablyn Scheduler assigns tasks based on completion of
prior tasksn Tail latency/execution time variability: single
slow task can hold up large MapReduce jobn Runtime libraries replicate tasks near end of job
Programm
ing Models and W
orkloads for WSC
s
10Copyright © 2019, Elsevier Inc. All rights Reserved
Prgrm’g Models and Workloads
Programm
ing Models and W
orkloads for WSC
s
The University of Adelaide, School of Computer Science 22 November 2018
Chapter 2 — Instructions: Language of the Computer 6
11Copyright © 2019, Elsevier Inc. All rights Reserved
Computer Architecture of WSC
n WSC often use a hierarchy of networks for interconnection
n Each 19” rack holds 48 1U servers connected to a rack switch
n Rack switches are uplinked to switch higher in hierarchyn Uplink has 6-24X times lower bandwidthGoal is to
maximize locality of communication relative to the rack
Com
puter Ar4chitecture of WSC
12Copyright © 2019, Elsevier Inc. All rights Reserved
Storagen Storage options:
n Use disks inside the servers, orn Network attached storage through Infiniband
n WSCs generally rely on local disksn Google File System (GFS) uses local disks and
maintains at least three relicas
Com
puter Ar4chitecture of WSC
The University of Adelaide, School of Computer Science 22 November 2018
Chapter 2 — Instructions: Language of the Computer 7
13Copyright © 2019, Elsevier Inc. All rights Reserved
Array Switch
n Switch that connects an array of racksn Array switch should have 10 X the bisection
bandwidth of rack switchn Cost of n-port switch grows as n2
n Often utilize content addressible memory chips and FPGAs
Com
puter Ar4chitecture of WSC
14Copyright © 2019, Elsevier Inc. All rights Reserved
WSC Memory Hierarchyn Servers can access DRAM and disks on other
servers using a NUMA-style interface
Com
puter Ar4chitecture of WSC
The University of Adelaide, School of Computer Science 22 November 2018
Chapter 2 — Instructions: Language of the Computer 8
15Copyright © 2019, Elsevier Inc. All rights Reserved
WSC Memory HierarchyC
omputer Ar4chitecture of W
SC
16Copyright © 2019, Elsevier Inc. All rights Reserved
WSC Memory Hierarchy
Com
puter Ar4chitecture of WSC
The University of Adelaide, School of Computer Science 22 November 2018
Chapter 2 — Instructions: Language of the Computer 9
17Copyright © 2019, Elsevier Inc. All rights Reserved
Infrastructure and Costs of WSCn Cooling
n Air conditioning used to cool server roomn 64 F – 71 F
n Keep temperature higher (closer to 71 F)n Cooling towers can also be used
n Minimum temperature is “wet bulb temperature”
PhyscicalInfrastrcutureand C
osts of WSC
18Copyright © 2019, Elsevier Inc. All rights Reserved
Infrastructure and Costs of WSCn Cooling system also uses water (evaporation and
spills)n E.g. 70,000 to 200,000 gallons per day for an 8 MW facility
n Power cost breakdown:n Chillers: 30-50% of the power used by the IT equipmentn Air conditioning: 10-20% of the IT power, mostly due to fans
n How man servers can a WSC support?n Each server:
n “Nameplate power rating” gives maximum power consumptionn To get actual, measure power under actual workloads
n Oversubscribe cumulative server power by 40%, but monitor power closely
PhyscicalInfrastrcutureand C
osts of WSC
The University of Adelaide, School of Computer Science 22 November 2018
Chapter 2 — Instructions: Language of the Computer 10
19Copyright © 2019, Elsevier Inc. All rights Reserved
Infrastructure and Costs of WSCn Determining the maximum server capacity
n Nameplate power rating: maximum power that a server can draw
n Better approach: measure under various workloadsn Oversubscribe by 40%
n Typical power usage by component:n Processors: 42%n DRAM: 12%n Disks: 14%n Networking: 5%n Cooling: 15%n Power overhead: 8%n Miscellaneous: 4%
PhyscicalInfrastrcutureand C
osts of WSC
20Copyright © 2019, Elsevier Inc. All rights Reserved
Measuring Efficiency of a WSCn Power Utilization Effectiveness (PEU)
n = Total facility power / IT equipment powern Median PUE on 2006 study was 1.69
n Performancen Latency is important metric because it is seen by
usersn Bing study: users will use search less as
response time increasesn Service Level Objectives (SLOs)/Service Level
Agreements (SLAs)n E.g. 99% of requests be below 100 ms
PhyscicalInfrastrcutureand C
osts of WSC
The University of Adelaide, School of Computer Science 22 November 2018
Chapter 2 — Instructions: Language of the Computer 11
21Copyright © 2019, Elsevier Inc. All rights Reserved
Measuring Efficiency of a WSCPhyscicalInfrastrcuture
and Costs of W
SC
22Copyright © 2019, Elsevier Inc. All rights Reserved
Cost of a WSCn Capital expenditures (CAPEX)
n Cost to build a WSCn $9 to 13/watt
n Operational expenditures (OPEX)n Cost to operate a WSC
PhyscicalInfrastrcutureand C
osts of WSC
The University of Adelaide, School of Computer Science 22 November 2018
Chapter 2 — Instructions: Language of the Computer 12
23Copyright © 2019, Elsevier Inc. All rights Reserved
Cloud Computingn Amazon Web Services
n Virtual Machines: Linux/Xenn Low costn Open source softwaren Initially no guarantee of servicen No contract
Cloud C
omputing
24Copyright © 2019, Elsevier Inc. All rights Reserved
Cloud Computingn Cloud Computing Growth
Cloud C
omputing
The University of Adelaide, School of Computer Science 22 November 2018
Chapter 2 — Instructions: Language of the Computer 13
25
Fallacies and Pitfallsn Cloud computing providers are losing money
n AWS has a margin of 25%, Amazon retail 3%
n Focusing on average performance instead of 99th percentile performance
n Using too wimpy a processor when trying to improve WSC cost-performance
n Inconsistent Measure of PUE by different companies
n Capital costs of the WSC facility are higher than for the servers that it houses
Copyright © 2019, Elsevier Inc. All rights Reserved
Fallcies and Pitfalls
26
Fallacies and Pitfallsn Trying to save power with inactive low power
modes versus active low power modesn Given improvements in DRAM dependability and
the fault tolerance of WSC systems software, there is no need to spend extra for ECC memory in a WSC
n Coping effectively with microsecond (e.g. Flash and 100 GbE) delays as opposed to nansecond or millisecond delays
n Turning off hardware during periods of low activity improves the cost-performance of a WSC
Copyright © 2019, Elsevier Inc. All rights Reserved
Fallcies and Pitfalls