cross-layer scheduling in cloud systems hilfi alkaff, indranil gupta, luke leslie department of...
TRANSCRIPT
Cross-Layer Scheduling in Cloud Systems
Hilfi Alkaff, Indranil Gupta, Luke Leslie
Department of Computer Science
University of Illinois at Urbana-Champaign
1Distributed Protocols Research Group: http://dprg.cs.uiuc.edu
Inside a Datacenter: Networks Connecting Servers
Tree
Fat Tree[Leiserson 85]
Jellyfish [Singla 12]
Clos [Dally 04]
VL2 [Greenberg 09]
2
Tree
Fat Tree[Leiserson 85]
Jellyfish [Singla 12]
Clos [Dally 04]
VL2 [Greenberg 09]
Structured Networks Unstructured Networksand/or routing
Inside a Datacenter: Networks Connecting Servers
3
SDN• Software Defined Networking
• For any end-host pair, multiple routes available
• SDN Controller helps to choose one of these routes– Configures switches accordingly
• Which route is the “best”?
4
SDNs and Applications• Which route is the “best”?• Our approach
– Best network routes should really be decided based on the application that is using the network• To minimize interference (and thus congestion) and to optimize bandwidth use• Today: SDN routes selected application-agnostic way
– But the application itself can help, by placing tasks at servers• Today: Applications schedule tasks in network-agnostic way, leading to bad
bandwidth utilization– SDN Controller and Application Scheduler should coordinate with
each other• This is our cross-layer scheduling approach
5
Applications: Short Real-Time Analytics Jobs
Batch Processing: MapReduce, Hadoop
Stream Processing: Storm
6
Tasks
Storm
Tasks
Hadoop
7
Tasks and Flows
Storm
Tasks
Hadoop
Flows 8
Challenges• Two large state spaces to explore
1. Set of Possible Routes for each end-to-end flow
– Large numbers of flows and possible routes
2. Set of Possible Task to Server Placements
– Large numbers of servers and tasks
9
Our Strategy• To explore state space, use simulated annealing– At application level scheduler– And separately at routing (SDN) level
• Simulated Annealing– probabilistic approach – avoids getting stuck in local optima with some non-zero
probability of jumping away– probability of jumping away decreases quickly over time
(annealing process for steel)
10
Pre-computation• For all pairs of servers, pre-compute the k shortest paths
– Store it in a hash table, indexed by server pair
– Compact storage by merging overlapping routes (for a server pair) into a tree
• Small in size and Quick to compute– 1000 servers, k=10
– 50 M entries
– After compaction, 6 MB
– 3 minutes to generate
11
When a Job Arrives• Don’t change the allocations or routes of existing jobs
– Non-intrusive
– Reduces state space to explore
• Simulated Annealing is run offline, and the resultant schedule is used to schedule new job’s tasks and flows
• Primary Simulated Annealing (SA) runs at Application level– Calls Routing level SA
12
Simulated Annealing Steps• Start from an arbitrary state
– Tasks to servers, and routes to flows
• Generate next-state S’(At Application Level)
1. De-allocate one task• Prefer tasks that affect computation more, e.g., closer to beginning or end of
topology
2. Allocate this task to random server
3. Call Routing level SA
13
Simulated Annealing Steps (2)…
3.Call Routing level SA
4.(At Routing Level)
5.De-path one route• Select random server pair
• Remove its worst path
– Prefer higher number of hops, and break ties by lower bandwidth
6.Allocate Path: Change this route to a better path– Prefer lower number of hops, and break ties by higher bandwidth
14
Simulated Annealing Steps (3)• After generating next-state S’
– Calculate utility(S’)
– Utility function considers all jobs in cluster (not just new job)
– Utility function accounts for bottlenecked paths from source tasks to sink tasks
• If utility(S’) > utility(current state)– Transition from current state to S’
• If utility(S’) ≤ utility(current state)– Transition with probability e(utility(S’)-utility(current state))/t
– Non-zero probability of transitioning even if S’ is a worse state
– Probability decreases over time (t)
• Wait until convergence
• Re-run entire simulated annealing 5 times, and take best result
15
Experiments• Implemented into Apache Hadoop (YARN)
• Implemented into Apache Storm
• Deployment experiments on Emulab: up to 30 hosts– Emulated network using ZeroMQ and Thrift
– Emulated Fat-Tree and Jellyfish
• Larger scale simulation experiments – Upto 1000 hosts
16
Experimental Settings• 10 hosts, 100 Mbps, 5 links per router, #links selected via scaling rules
– 3 GHz, 2 GB RAM
• Hadoop cluster workload– Facebook’s SWIM benchmark
– Shuffle ranges from 100 B to 10 GB
– 1 job per second
• Storm cluster workload: Random tree topologies– Topologies constructed as randomly with number of children selected by Gaussian (mean = sd = 2)
– 100 B tuples
– Each source generate 1 MB – 100 MB of data
– 10 jobs per minute
• Each experimental run is 10 minutes
17
Tree
Fat Tree[Leiserson 85]
Jellyfish [Singla 12]
Clos [Dally 04]
VL2 [Greenberg 09]
Structured Networks Unstructured Networksand/or routing
Inside a Datacenter: Networks Connecting Servers
18
Storm on Jellyfish Topology
App+Routing SA: 34.1% improvement in throughput at 30 hosts
Application-only SA: 21.2%Routing-only SA: 23.2% Performance
improves with scale
19
Hadoop on Fat-Tree Topology
App+Routing SA: 26% improvement in throughput at 30 hosts
Application-only SA & Routing-only SASmaller than combining both
Performance improves with scale
20
Other Experimental Results• Similar results for other combinations
• Hadoop on Jellyfish– App+Routing SA: 31.9% improvement in throughput at 30 hosts
– Performance improves with scale
– Application-only SA: 18.8%
– Routing-only SA: 25.5%
• Storm on Fat-Tree– App+Routing SA: 30% improvement in throughput at 30 hosts
– Performance improves with scale
– Application-only SA: 21.1%
– Routing-only SA: 22.7%
21
Other Experimental Results (2)• Scheduling time is small
– Time to schedule a new job in a 1000 server cluster– Fat-Tree: 0.48 s (Hadoop) to 0.53 s (Storm)
– Jellyfish: 0.67 s (Hadoop) to 0.74 s (Storm)
• No starvation – Worst case degradation in completion time for any job is 20% in Hadoop, 30% in
Storm
– Outliers are large jobs (rare in real-time analytics with short jobs)
• Fault-recovery is fast– Upon failure, re-run simulated annealing once
– Recovery occurs within 0.35 s to 0.4 s
22
Takeaways• Today: Application schedulers and SDN scheduler are disjoint
– Leads to suboptimal placement and routing
• Our approach: coordinated cross-layer scheduling– Explore small state spaces
– Use simulated annealing
• At 30 hosts, gives between 26% to 34% improvement in Hadoop and Storm for both structured/unstructured networks – Other networks will fall between these two numbers
• Overheads are small, and improvement gets better with scale
23Distributed Protocols Research Group: http://dprg.cs.uiuc.edu
Ongoing/Future WorkOur work opens the door:
•Explore other heuristics, e.g., data affinity for tasks, congestion
•Explore other non-SA approaches
•Available bandwidth estimation
•OpenFlow integration
•Batching multiple jobs into scheduling
24Distributed Protocols Research Group: http://dprg.cs.uiuc.edu