Slide 1

Scalable Management for Networks and Services Rolf Stadler Laboratory for Communication Networks KTH Royal Institute of Technology Stockholm HP Laboratories, Palo Alto, March 31, 2003 Slide 2 node Management station Management station Manager-Agent based management node AAA M results download & execute P Management Program Manager Agent Centralized Control Management protocols: SNMP, CMIP Program runs on Management Station Decentralized Control Program runs on network nodes P The Shift of a Management Paradigm A P A A Slide 3 Router Execution Environment Management station Management Program navigation Code Server Architecture for Pattern-based Management Slide 4 Slide 5 WeaverA Testbed for pattern-based Management WAN A WAN B WAN C WAN D Management Station Router ARouter BRouter CRouter D FastEthernet Switch Slide 6 Slide 7 Slide 8 Slide 9 Simple Navigation Patterns Slide 10 Echo Pattern (expansion) Slide 11 d root =1 Echo Pattern (expansion) Slide 12 d root =2 Echo Pattern (expansion) Slide 13 d root =3 Echo Pattern (expansion) Slide 14 d root =4 Echo Pattern (expansion) Slide 15 Echo Pattern d root =5 Slide 16 d root =4 Echo Pattern (contraction) Slide 17 d root =3 Echo Pattern (contraction) Slide 18 d root =2 Echo Pattern (contraction) Slide 19 d root =1 Echo Pattern (contraction) Slide 20 Slide 21 Slide 22 The Echo Pattern Two phases of traversal expansion phase: explorers flood network with requests for local operations contraction phase: echoes return and aggregate results Properties Generates balanced traffic load Traffic load depends on network topology, not on speed of traversal Time complexity increases linearly with network diameter. Slide 23 Examples of Echo-based Management Get information on topology compute the current number of leaf nodes, the connectivity distribution discover current topology within 10 hops of node x Get information on network state identify 10 most congested links compute distribution of link utilization, queue lengths identify sub topologies with highly loaded links find a resource R closest to node x Slide 24 Pattern-based Management An Engineering Approach to Decentralized Management A management program consists of A navigation pattern (distr. graph traversal algorithm) An operation on nodes An aggregation function Relevance of this approach Provides a basis to analyze management operation for performance, scalability, robustness Supports concept of re-usable patterns, hides complexity Slide 25 Composing Management Programs Segall Echo Patterns Navigation Patterns Chang Skip WaitScopeMulti Echo Aggregators Res. Disc. Aggregators Leaf Count Load. Hist. Conn. Hist. CLIHTTP XML SNMP Local Operations Node Access Management Program Slide 26 Properties of Patterns Echo Aggregators Res. Disc. Aggregators Leaf CountLoad. Hist.Conn. Hist. CLIHTTP XML SNMP Node Access Management Program Segall Simple Echo Robust EchoOthers Echo Patterns Navigation Patterns Chang Skip Wait Scope Multi A pattern can be used for many management operations. A pattern can be chosen according to performance objectives. A pattern hides the complexity of a distributed operation. Network failures can be handled within patterns. Code mobility can be controlled. Slide 27 visited i : boolean init false; G i : set of integers init neighbors(); parent i : integer init -1; Echo(inmsg: bytes, from: integer) { G i := G i - from; if visited i = false { parent i := from; visited i := true; OnInitiate(inmsg, outmsg); if G i != empty dispatch(parent i, outmsg, i); } else OnAggregate(inmsg); if G i = empty { OnComplete(outmsg); if parent i >= 0 dispatch(parent i, outmsg, i); else OnTerminate(inmsg); } The Interface between Pattern and Aggregator OnBegin OnTerminate OnInitiate OnComplete OnAggregate av_load i := av_load; av_load := load(); n:=1; av_load := (av_load*n + av_load j )/(n+1); n:=n+1; Slide 28 SIMPSON: A SIMple Pattern Simulator fOr Large Networks Slide 29 Analyzing Management Operations Network Graph G=(V,E)Execution Graphs G=(V,E) Centralized ManagementDistributed Management Star PatternEcho Pattern Slide 30 Traffic Complexity of Management Operations Amount of traffic placed on the network during execution. Slide 31 Time Complexity of Management Operations Time needed from invocation until completion of a operation. Slide 32 Performing Echo-based Operations on the Entire Internet Purpose is illustrating the scalability of echo-based operations. What we needed: Complexity analysis of pattern Estimation of Internet topological properties diameter connectivity distribution number of nodes Slide 33 Estimated Performance of Echo-based Operation on the Internet Assumptions: Process-level transmission time: 5ms Network delay per hop: 4ms Message size: 1KB Local operation: 500ms per execution Diameter of Internet: 34 hops Slide 34 Active Node Manager Source Repository Binaries Repository Preprocessor Transport Access Point Execution Environment Management Operation Results Device Manager C++ Compiler Source, State SNMP sets Management commands Weaver Active Node Source code, Active node management Router Node State Local Program States Source, State Events SNMP gets/traps Source Code results Active Node Engine Management Station Slide 35 Suboperations in Weaver Node ANode B start Execution (T1) Serialization (T2) Dispatch (T3) Receiving (T4) Loading (T5) or Instantiation (T6) De-serialization (T7) Execution (T1) Serialization (T2) Dispatch (T3) Receiving (T4) Resolving (T8) end Time De-serialization (T7) Execution (T1) T C1 T C2 Slide 36 Measuring Execution Times on Weaver Duration in msPerformed by Module Execution (T1)1.57 ( = 0.48)Execution Environment Serialization (T2)3.46 ( = 0.71)Execution Environment Dispatch (T3)1.67 ( = 0.49)Transport Access Point Receiving (T4)0.62 ( = 0.30)Transport Access Point Loading (T5)23.42 ( = 0.70)Execution Environment Instantiation (T6)0.77 ( = 0.015)Execution Environment De-serialization (T7)2.04 ( = 0.49)Execution Environment Resolving (T8)0.15 ( = 0.001)Execution Environment Communications Delay (T C )4.04 ( = 0.10)--- Slide 37 Estimating Execution Times of Echo-based Operations on Weaver Slide 38 Skip Echo SkipEcho(inmsg: bytes from: integer) { if visited i = false { parent i := from; visited i := true; OnInitiate(inmsg, outmsg, i); G i = up_neighbors() - from; if G i != empty dispatch(parent i, outmsg, i); } else { G i = G i - from; OnAggregate(inmsg); } if complete i != true and G i = empty { OnComplete(outmsg); completed i := true; if parent i >= 0 dispatch(parent i, outmsg, i); else OnTerminate(inmsg); } alarm(type: {failure, recovery}, affected: integer){ if visited i = true { if type = failure { G i := G i - affected if complete i != true and G i = empty { complete i := true; OnComplete(outmsg); if parent i >= 0 dispatch(parent i, outmsg, i); else OnTerminate(inmsg); } Wait Echo SkipEcho(inmsg: bytes from: integer) { if visited i = false { parent i := from; visited i := true; OnInitiate(inmsg, outmsg, i); G i = up_neighbors() - from; if G i != empty dispatch(parent i, outmsg, i); } else { G i = G i - from; OnAggregate(inmsg); } if complete i != true and G i = empty { OnComplete(outmsg); completed i := true; if parent i >= 0 dispatch(parent i, outmsg, i); else OnTerminate(inmsg); } alarm(type: {failure, recovery}, affected: integer){ if visited i = true { if type == failure { G i = G i - affected B i = B i + affected if complete i != true and G i = empty { complete i := true; OnComplete(outmsg); if parent i >= 0 dispatch(parent i, outmsg, i) else OnTerminate(inmsg); } } else { if affected is in B i { B i = B i - affected G i = G i + affected } Designing Robust Patterns Plain Echo Echo(inmsg: bytes, from: integer) { Gi := Gi - from; if visitedi = false { parenti := from; visitedi := true; OnInitiate(inmsg, outmsg); if Gi != empty dispatch(parenti, outmsg, i); } else OnAggregate(inmsg); if Gi = empty { OnComplete(outmsg); if parenti >= 0 dispatch(parenti, outmsg, i); else OnTerminate(inmsg); } Slide 39 Network Coverage vs. Execution Time for Skip Echo MTTR=1 min MTTR=11 min MTTR 0 MTTR inf MTTF = 3.6 hrs MTTF = 7.3 hrs MTTF = 11.0 hrs MTTF = 14.7 hrs MTTF = 29.4 hrs MTTF = 73.6 hrs Slide 40 Current and Planned Work Self-organizing, adaptable Networks and Systems: Patterns for routing and dynamic construction of network control structures. (Constantin Adam) WQL: A table-based Network Query Language on Weaver. (Koon-Seng Lim) Policy-based Management: Patterns for distribution and dynamic re-computation of policies. (Alberto Gonzalez) Slide 41 Literature on this Work K.S. Lim, R. Stadler: WeaverRealizing a scalable management paradigm on commodity routers, Eighth IFIP/IEEE International Symposium on Integrated Network Management (IM 2003), Colorado Springs, Colorado, USA, March 24-28, 2003.WeaverRealizing a scalable management paradigm on commodity routers K.S. Lim and R. Stadler: "Developing pattern-based management programs," IFIP/IEEE International Conference on Management of Multimedia Networks and Services (MMNS 2001), Chicago, IL, October 29 - November 1, 2001.Developing pattern-based management programs K.S. Lim and R. Stadler: "A navigation pattern for scalable Internet management,"IFIP/IEEE International Symposium on Integrated Network Management (IM 2001), Seattle,Washington, 14-18 May, 2001.A navigation pattern for scalable Internet management R. Kawamura and R. Stadler: "A middleware architecture for active distributed management of IP networks, "IEEE/IFIP Network Operations and Management Symposium (NOMS 2000), Honolulu, Hawaii, April 10-14, 2000.A middleware architecture for active distributed management of IP networks