serverless computing: achilleas tzenetopoulos, ph.d
TRANSCRIPT
Serverless Computing: Micromanagement on the Clouds
1
Microlab’s Fridays Tech Talks, #3
November 6, 2020
Achilleas Tzenetopoulos, Ph.D. Student
Overview1. Introduction - Going through the Post-PC era
2. Defining Serverless
3. Limitations & Challenges
4. Resource Management related opportunities
5. Other opportunities for computer architects
2
Introduction - Going through the “Post-PC” era
3
Physical Servers / Bare-metal
Introduction - Going through the “Post-PC” era
4
Physical Servers / Bare-metal
Introduction - Going through the “Post-PC” era
5
Physical Servers / Bare-metal
Infrastructure as a Service
Introduction - Going through the “Post-PC” era
6
Physical Servers / Bare-metal
Infrastructure as a Service
Container as a Service
Introduction - Going through the “Post-PC” eraPhysical Servers / Bare-metal
Infrastructure as a Service
Container as a Service
Function as a Service
7
Defining Serverless Computing (1/2) - Benefits
8
Jonas, Eric, et al. "Cloud programming simplified: A berkeley view on serverless computing." arXiv preprint arXiv:1902.03383 (2019).
Serverless computing is a platform that hides server
usage from developers and runs code on-demand
automatically scaled and billed only for the time
the code is running.
1
Defining Serverless Computing (2/2) Serverless enables the deployment of microservices in a more scalable and cost
efficient way.
9
Yu Gan, et. al 2019. An Open-Source Benchmark Suite for Microservices and Their Hardware-Software Implications for Cloud & Edge Systems. In Proceedings of 2019 Architectural Support for Programming Languages and Operating Systems (ASPLOS’19). ACM, New York, NY, USA, 16 pages. https: //doi.org/10.1145/3297858.3304013
Defining Serverless Computing (2/2) Serverless enables the deployment of microservices in a more scalable and cost
efficient way.
10
Yu Gan, et. al 2019. An Open-Source Benchmark Suite for Microservices and Their Hardware-Software Implications for Cloud & Edge Systems. In Proceedings of 2019 Architectural Support for Programming Languages and Operating Systems (ASPLOS’19). ACM, New York, NY, USA, 16 pages. https: //doi.org/10.1145/3297858.3304013
Serviceful
Limitations & Challenges (1/3) - Excessive Data Movement
Rapid scaling
11
(Num)PyWren GEMM read and write amplification
A
B
C
D
E F
G
Fan-out
Fan-in
Benjamin Carver et.al. 2020. Wukong: a scalable and locality-enhanced framework for serverless parallel computing. In Proceedings of the 11th ACM Symposium on Cloud Computing (SoCC '20). Association for Computing Machinery, New York, NY, USA, 1–15. DOI:https://doi.org/10.1145/3419111.3421286
Limitations & Challenges (1/3) - Excessive Data Movement
Rapid scaling
12
(Num)PyWren GEMM read and write amplification
A
B
C
D
E F
G
Fan-out
Fan-in
Benjamin Carver et.al. 2020. Wukong: a scalable and locality-enhanced framework for serverless parallel computing. In Proceedings of the 11th ACM Symposium on Cloud Computing (SoCC '20). Association for Computing Machinery, New York, NY, USA, 1–15. DOI:https://doi.org/10.1145/3419111.3421286
Rapid scaling
13
(Num)PyWren GEMM read and write amplification
A
B
C
D
E F
G
Fan-out
Fan-in
Limitations & Challenges (1/3) - Excessive Data Movement
Benjamin Carver et.al. 2020. Wukong: a scalable and locality-enhanced framework for serverless parallel computing. In Proceedings of the 11th ACM Symposium on Cloud Computing (SoCC '20). Association for Computing Machinery, New York, NY, USA, 1–15. DOI:https://doi.org/10.1145/3419111.3421286
14
(Num)PyWren GEMM read and write amplification
A
B
C
D
E F
G
Fan-out
Fan-in
Limitations & Challenges (1/3) - Excessive Data Movement
Benjamin Carver et.al. 2020. Wukong: a scalable and locality-enhanced framework for serverless parallel computing. In Proceedings of the 11th ACM Symposium on Cloud Computing (SoCC '20). Association for Computing Machinery, New York, NY, USA, 1–15. DOI:https://doi.org/10.1145/3419111.3421286
AWS and Azure fail to provide proper
performance isolation between coresident
instances, and so contention can cause
considerable performance degradation.
1
15
1.Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. 2018. Peeking behind the curtains of serverless platforms. In Proceedings of the 2018 USENIX Conference on Usenix Annual Technical Conference (USENIX ATC '18). USENIX Association, USA, 133–145.2.Tianyi Yu, Qingyuan Liu, Dong Du, Yubin Xia, Binyu Zang, Ziqian Lu, Pingchao Yang, Chenggang Qin, and Haibo Chen. 2020. Characterizing Serverless Platforms with ServerlessBench. In ACM Symposium on Cloud Computing (SoCC ’20), October 19–21, 2020, Virtual Event, USA. ACM, New York, NY, USA, 15 pages. https: //doi.org/10.1145/3419111.3421280
Limitations & Challenges (2/3) - Performance isolation
AWS and Azure fail to provide proper
performance isolation between coresident
instances, and so contention can cause
considerable performance degradation.
1
16
1. Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. 2018. Peeking behind the curtains of serverless platforms. In Proceedings of the 2018 USENIX Conference on Usenix Annual Technical Conference (USENIX ATC '18). USENIX Association, USA, 133–145.2.Tianyi Yu, Qingyuan Liu, Dong Du, Yubin Xia, Binyu Zang, Ziqian Lu, Pingchao Yang, Chenggang Qin, and Haibo Chen. 2020. Characterizing Serverless Platforms with ServerlessBench. In ACM Symposium on Cloud Computing (SoCC ’20), October 19–21, 2020, Virtual Event, USA. ACM, New York, NY, USA, 15 pages. https: //doi.org/10.1145/3419111.3421280
Limitations & Challenges (2/3) - Performance isolation
A serverless function will contend
with others for memory bandwidth,
which implies the platform should
provide an essential isolation
mechanism to guarantee sufficient
bandwidth budgets.
2
17
Limitations & Challenges (2/3) - Performance isolationCloud providers charge customers MB(Ram)/s User is able to define only the memory usage of
her function
AWS tries to allocate a fixed amount of CPU
cycles to an instance based only on function
memory.
AWS employs bin packing techniques in order
to maximize Memory usage per VM instance
However, while this technique is optimal for revenue maximization, it results in resource
contention and prolonged execution time.
1. Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. 2018. Peeking behind the curtains of serverless platforms. In Proceedings of the 2018 USENIX Conference on Usenix Annual Technical Conference (USENIX ATC '18). USENIX Association, USA, 133–145.2.Tianyi Yu, Qingyuan Liu, Dong Du, Yubin Xia, Binyu Zang, Ziqian Lu, Pingchao Yang, Chenggang Qin, and Haibo Chen. 2020. Characterizing Serverless Platforms with ServerlessBench. In ACM Symposium on Cloud Computing (SoCC ’20), October 19–21, 2020, Virtual Event, USA. ACM, New York, NY, USA, 15 pages. https: //doi.org/10.1145/3419111.3421280
18
Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. 2018. Peeking behind the curtains of serverless platforms. In Proceedings of the 2018 USENIX Conference on Usenix Annual Technical Conference (USENIX ATC '18). USENIX Association, USA, 133–145.Tianyi Yu, Qingyuan Liu, Dong Du, Yubin Xia, Binyu Zang, Ziqian Lu, Pingchao Yang, Chenggang Qin, and Haibo Chen. 2020. Characterizing Serverless Platforms with ServerlessBench. In ACM Symposium on Cloud Computing (SoCC ’20), October 19–21, 2020, Virtual Event, USA. ACM, New York, NY, USA, 15 pages. https: //doi.org/10.1145/3419111.3421280
Limitations & Challenges (3/3) - Startup Latency● Οn-demand request execution.
● Startup overhead (including
sandbox preparation, function
loading and initialization)
● Overhead is considerable with
respect to the small and short-lived
execution unit
Resource Management opportunities (1/5)
19
Function compositions
A
B
C
D
E F
G
Fan-out
Fan-in
Data-locality benefits
Parallel execution
benefits
Resource Management opportunities (2/5)
20
Function compositions
Coldstart Latency B E F
A F
Resource Management opportunities (2/5)
21
Function compositions
Leverage hints of the expected arrival time of the next request
Coldstart Latency B E F
A F
Resource Management opportunities (3/5)Function compositions
Coldstart Latency
Interference-aware placement
L3 cacheL2 L1
C
VM
VM
RAM
RAM
VM
RAM
RAM
Cache Misses
Memory Bandwidth
Network Bandwidth
Resource Management opportunities (4/5)Function compositions
Coldstart Latency
Interference-aware placement
Resource fine-tuning on
runtime
A
B
C
D
E F
G
Resource Management opportunities (4/5)Function compositions
Coldstart Latency
Interference-aware placement
A
B
C
D
E F
G
Horizontal & Vertical scaling and runtime
resource management
Satisfy QoS guarantees by increasing
resource allocation of future tasks
Resource fine-tuning on
runtime
Resource Management opportunities (5/5)
System
monitoring
DAG generator
A
B
C
D
E F
GWorkload
partitioning
BEF
AQoS Controller
Application
QoS monitoring
25
Function Scheduling
Runtime resource
tuning
Cost minimization
High utilization
Optimization
Goals s.t QoS
Low power
consumption
26
Other opportunities for computer architects
● Static code analysis
● Leverage domain specific architectures (FPGAs, GPUs, TPUs, etc)
● Language-specific custom processors through hardware and software
co-design
Thank you!
27
Bonus: Serverless on Edge
28
Edge servers
Sensors
Computing power
Data movement cost
Function’s fine granularity enables an even more distributed placement
Efficient, interference-aware co-location on edge-devices with respect to QoS requirements can dodge unnecessary data movement