planetlab: present and future steve muir 3rd august, 2005 (slides taken from larry peterson)
TRANSCRIPT
PlanetLab: Present and Future
Steve Muir
3rd August, 2005
(slides taken from Larry Peterson)
PlanetLab Today
• Global distributed systems infrastructure– platform for long-running services– testbed for network experiments
• 583 nodes around the world– 30 countries– 250+ institutions (universities, research labs, gov’t)
• Standard PC servers– 150–200 users per server– 30–40 active per hour, 5–10 at any given time– memory, CPU both heavily over-utilised
Node Software
• Linux Fedora Core 2– kernel being upgraded to FC4– always up-to-date with security-related patches
• VServer patches provide security– each user gets own VM (‘slice’)– limited root capabilities
• CKRM/VServer patches provide resource mgmt– proportional share CPU scheduling– hierarchical token bucket controls network Tx bandwidth– physical memory limits– disk quotas
Issues
• Multiple VM Types– Linux vservers, Xen domains
• Federation– EU, Japan, China
• Resource Allocation– Policy, markets
• Infrastructure Services– Delegation
Need to define the PlanetLab Architecture
Key Architectural Ideas
• Distributed virtualization– slice = set of virtual machines
• Unbundled management– infrastructure services run in their own slice
• Chain of responsibility– account for behavior of third-party software– manage trust relationships
N x N
Trust Relationships
PrincetonBerkeleyWashingtonMITBrownCMUNYUETHHarvardHP LabsIntelNEC LabsPurdueUCSDSICSCambridgeCornell…
princeton_codeennyu_dcornell_beehiveatt_mcashcmu_esmharvard_icehplabs_donutlabidsl_pseprirb_phiparis6_landmarksmit_dhtmcgill_cardhuji_enderarizona_storkucb_bambooucsd_shareumd_scriptroute…
TrustedIntermediary
(PLC)
Principals
• Node Owners– host one or more nodes (retain ultimate control)– selects an MA and approves of one or more SAs
• Service Providers (Developers)– implements and deploys network services– responsible for the service’s behavior
• Management Authority (MA)– installs an maintains software on nodes– creates VMs and monitors their behavior
• Slice Authority (SA)– registers service providers– creates slices and binds them to responsible provider
Trust Relationships(1) Owner trusts MA to map network
activity to responsible sliceMA
Owner Provider
SA
(2) Owner trusts SA to map slice to responsible providers
1
2
5
6
(3) Provider trusts SA to create VMs on its behalf
3
(4) Provider trusts MA to provide working VMs & not falsely accuse it
4
(5) SA trusts provider to deploy responsible services
(6) MA trusts owner to keep nodes physically secure
Architectural Elements
MA
NM +VMM
nodedatabase
NodeOwner
OwnerVM
SCS
SAslicedatabase
VM ServiceProvider
Narrow Waist
• Name space for slices< slice_authority, slice_name >
• Node Manager Interfacerspec = < vm_type = linux_vserver,
cpu_share = 32,
mem_limit - 128MB,
disk_quota = 5GB,
base_rate = 1Kbps,
burst_rate = 100Mbps,
sustained_rate = 1.5Mbps >
Node Boot/Install ProcessNode PLC Boot Server
1. Boots from BootCD (Linux loaded)
2. Hardware initialized
3. Read network config . from floppy
7. Node key read into memory from floppy
4. Contact PLC (MA)
6. Execute boot mgr
Boot Manager
8. Invoke Boot API
10. State = “install”, run installer
11. Update node state via Boot API
13. Chain-boot node (no restart)
14. Node booted
9. Verify node key, send current node state
12. Verify node key, change state to “boot”
5. Send boot manager
PlanetFlow
• Logs every outbound IP flow on every node– accesses ulogd via Proper– retrieves packet headers, timestamps, context ids (batched)– used to audit traffic
• Aggregated and archived at PLC
Chain of ResponsibilityJoin Request PI submits Consortium paperwork and requests to join
PI Activated PLC verifies PI, activates account, enables site (logged)
User Activated Users create accounts with keys, PI activates accounts (logged)
Nodes Added to Slices
Users add nodes to their slice (logged)
Slice Traffic Logged
Experiments run on nodes and generate traffic (logged by Netflow)
Traffic Logs Centrally Stored
PLC periodically pulls traffic logs from nodes
Slice Created PI creates slice and assigns users to it (logged)
Network Activity Slice Responsible Users & PI
Slice Creation
PLC(SA)
VMM
NM VM
PI SliceCreate( ) SliceUsersAdd( )
User SliceNodesAdd( ) SliceAttributeSet( ) SliceInstantiate( )
SliceGetAll( )
slices.xml VM VM…
.
.
.
.
.
.
Slice Creation
PLC(SA)
VMM
NM VM
PI SliceCreate( ) SliceUsersAdd( )
User SliceAttributeSet( ) SliceGetTicket( )
VM VM…
.
.
.
.
.
.
(distribute ticket to slice creation service)
SliverCreate(ticket)
Brokerage Service
PLC(SA)
VMM
NM VM
PI SliceCreate( ) SliceUsersAdd( )
Broker SliceAttributeSet( ) SliceGetTicket( )
VM VM…
.
.
.
.
.
.
(distribute ticket to brokerage service)
rcap = PoolCreate(ticket)
Brokerage Service (cont)
PLC(SA)
VMM
NM VM VM VM…
.
.
.
.
.
.
(broker contacts relevant nodes)
PoolSplit(rcap, slice, rspec)
VM
User BuyResources( ) Broker