greensoftware : managing datacenters powered by renewable energy
DESCRIPTION
GreenSoftware : Managing Datacenters Powered by Renewable Energy. Íñigo Goiri, William Katsak, Md E Haque , Kien Le, Ryan Beauchea, Jordi Guitart, Jordi Torres, Thu D. Nguyen, Ricardo Bianchini Department of Computer Science . Motivation. Datacenters consume large amounts of energy - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/1.jpg)
GreenSoftware:Managing Datacenters Powered by Renewable
Energy
Íñigo Goiri, William Katsak, Md E Haque, Kien Le,Ryan Beauchea, Jordi Guitart, Jordi Torres,
Thu D. Nguyen, Ricardo Bianchini
Department of Computer Science
![Page 2: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/2.jpg)
2
Motivation• Datacenters consume large amounts of energy• High energy cost and carbon footprint– Brown electricity: coal and natural gas
• Connect datacenters to green sources: solar, wind
Apple DC in Maiden, NC 40MW solar farm
Green datacenter
![Page 3: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/3.jpg)
3
Challenges and opportunities
• Scheduling workload/energy sources– Lower costs: brown energy, peak brown power, capital
• Study opportunities in green datacenters– Build hardware/software
Power
Time
Load
VariableSolar power
Workload
![Page 4: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/4.jpg)
4
GreenSoftware
How to build software for green datacenters?1. Malleable energy demand– Idle nodes → Turn off/Sleep (S3) [COLP’01]– Reduce frequency (DVFS) → Lower quality
2. Move computation under renewables– Weather forecast → Green energy forecast– Delay computation or degrade quality– Leverage energy storage
![Page 5: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/5.jpg)
5
Outline
• Motivation• GreenSoftware– GreenSlot– GreenHadoop– GreenSwitch– GreenCassandra– … and others
• Conclusion
![Page 6: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/6.jpg)
6
GreenSlot [SC’11]
• Batch jobs on SLURM (& Hadoop)• Send idle nodes to S3• Predict solar availability• Delay jobs within deadlines– Known jobs characteristics (length, deadline, size…)– Heuristic
Time
Job 1 Power
Deadline
Job 2
Job 3
Job 4
![Page 7: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/7.jpg)
7
GreenSlot [SC’11]
• Batch jobs on SLURM (& Hadoop)• Send idle nodes to S3• Predict solar availability• Delay jobs within deadlines– Known jobs characteristics (length, deadline, size…)– Heuristic
Time
Job 1 Power
Deadline
Job 2 Job 3
Job 4
![Page 8: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/8.jpg)
8
GreenHadoop [Eurosys’12]
• Batch jobs on Hadoop• Send idle nodes to S3• Make required data available– Move data blocks
• Predict solar availability• Delay jobs within deadlines– Predict global jobs energy consumption– Heuristic
Map1Map2Map3Map4Map5
Reduce
Reduce 6
7
Shuffle
![Page 9: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/9.jpg)
9
Covering subset
GreenHadoop: Data management• Deactivate servers to save energy
– Some data might become unavailable• Prior solution: covering subset [Leverich’09]
– Set of servers always running has ALL data
7
3
45
21 6
8
7 1
4 5
6
3
2
8 1
7 3
• Our approach• Only required data has to be available• We usually require fewer active servers
Server
Block
![Page 10: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/10.jpg)
10
GreenHadoop: Data managementServer 1
1 72Active
Decommission
Down
Server 24
356
Server 3
46
Required fileNon-required file
Server 42
3 84
Server 5
3 67
JobA 4
JobB 5
JobC 1
6
Running queue:
![Page 11: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/11.jpg)
11
GreenHadoop: Data management
Server 42
3 84
Server 5
3 67
Active
Decommission
Down
GreenHadoop (computation) requires only 2 servers
Server 1
1 72
Server 1
1 72
Server 24
356
Server 3
46
Required fileNon-required file JobA 4
JobB 5
JobC 1
6
Running queue:
![Page 12: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/12.jpg)
12
GreenHadoop: Data management
Active
Decommission
Down
Move required files to Active servers
Server 1
1 72
Server 24
356
Server 3
46
1
Server 42
3 84
Server 5
3 67
ReplicateJobA 4
JobB 5
JobC 1
6
Running queue:
![Page 13: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/13.jpg)
13
Server 1
1 72
GreenHadoop: Data management
Active
Decommission
Down
Decommissioned server can be sent to Down
Server 1
1 72
Server 24
356
Server 3
46
Required fileNon-required file
1
Server 42
3 84
Server 5
3 67
JobA 4
JobB 5
JobC 1
6
Running queue:
![Page 14: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/14.jpg)
14
Server 1
1 72
GreenHadoop: Data management
Active
Decommission
Down
Jobs to be executed change → Required files change
Server 24
356
Server 3
46
Non-required file
1
Server 42
3 84
Server 5
3 67
JobA 4
JobB 5
JobC 1
6
JobD 8
Required file
646
4
648
Required file
Running queue:
![Page 15: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/15.jpg)
15
Server 42
3 84
Server 1
1 72
GreenHadoop: Data management
Active
Decommission
Down
Make missing data available
Server 24
356
Server 42
3 84
Server 5
3 67
Server 3
46
1
Required file
Non-required file
JobB 5
JobC 1
JobD 8
Required fileRunning queue:
![Page 16: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/16.jpg)
16
Server 42
3 84
Server 1
1 72
GreenHadoop: Data management
Active
Decommission
Down
Server 24
356
Server 42
3 84
Server 5
3 67
GreenHadoop (computation) requires 3 servers
Server 3
46
1
Non-required file
JobB 5
JobC 1
JobD 8
Required fileRunning queue:
![Page 17: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/17.jpg)
17
GreenSwitch [ASPLOS’13]
• Batch jobs on Hadoop• Similar to GreenHadoop• Energy storage– Battery– Net metering
• Schedule workload and energy sources– Optimization
• Evaluation on Parasol(Presented on Monday by Thu)
![Page 18: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/18.jpg)
18
GreenCassandra
• Distributed DB/storage on Cassandra• Add an optional ring
• Degrade quality when no green
1
4
6 2
35
DHT Ring
A
A
A
1
4
35
6 2
DoubleDHT Ring
Optional
AA
Server
Data
![Page 19: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/19.jpg)
19
GreenSoftware summaryType Malleable energy Green adaptability
GreenSlot Batch jobs Delay jobsSleep servers Delay until green
GreenHadoop Batch jobsDelay jobsSleep serversData management
Delay until green
GreenSwitch Batch/interactive jobs Delay jobsSleep servers
Delay until greenEnergy storage
GreenCassandra Distributed storage Optional ring Degrade quality
GreenSLA VMs Migrate VMsSleep servers
Route green energyto racks
GreenPar MPI jobs Change parallelismSleep servers
Greater parallelismon green
GreenScale Non-deferrable jobs CPU and mem DVFS Faster on green
GreenNebula Geo distributed VMs Migrate VMs “Follow the renewables”
![Page 20: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/20.jpg)
20
Conclusions
• Green datacenters– Challenges & opportunities– Hardware/software solution
• GreenSoftware– Adapt software to green datacenters– Malleable energy demand– Match computation and renewables
![Page 21: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/21.jpg)
GreenSoftware:Managing Datacenters Powered by Renewable
Energy
Íñigo Goiri, William Katsak, Md E Haque, Kien Le,Ryan Beauchea, Jordi Guitart, Jordi Torres,
Thu D. Nguyen, Ricardo Bianchini
Department of Computer Science
![Page 22: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/22.jpg)
22
Other GreenSoftware• GreenSLA [IGCC’13]
– Bringing green energy to users– New hardware to route green energy
• GreenPar– MPI jobs with sub linear speedup– Use “Free” green energy
• GreenNebula– VMs in multiple geo distributed datacenters– Follow the sun
• GreenScale– Change frequency (DVFS)
![Page 23: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/23.jpg)
27
Parasol without GreenSwitch
Green use
Green available
Net metering
Brown use
IT load
![Page 24: GreenSoftware : Managing Datacenters Powered by Renewable Energy](https://reader035.vdocuments.us/reader035/viewer/2022062222/56816668550346895dd9fd2a/html5/thumbnails/24.jpg)
28
GreenSwitch: deferrable workload
Battery discharge
Battery charge
IT load
Net meteringGreen available
Green use