industrial issues in software performance aman shaikh at&t labs - research...
TRANSCRIPT
Industrial Issues in Software Performance
Aman ShaikhAT&T Labs - Research
June 24, 20087th International Workshop on Software and Performance
Princeton
Acknowledgement: Alan Glasser, Kobus van der Merwe, Jennifer Yates
Industrial Issues in Software Performance2
Introduction• Communication service providers
– Various services: voice, video, data– Various ways of connecting to customer
• Ethernet, Frame Relay, ATM, Wireless, …
– Trend towards supporting all communication over IP-based core network
• Today’s IP networks and services– Thousands of equipments and systems have to work
in tandem• Routers, switches, servers, line cards …• Provisioning, configuration, trouble-shooting, maintenance,
capacity planning, billing, …
– Huge amalgamation of hardware and software
Industrial Issues in Software Performance3
Challenges of Managing Networks• Lack of management support
– Absence of standardization in network management
– Solutions often develop in an organic fashion
• Scale– Hundreds of offices, thousands of routers, hundred thousands
of line cards and transport facilities
• Heterogeneity– Different kinds of equipments
• Example: routers, switches, servers, line cards, …
– Different vendors and models for same equipment• Example of router vendors: Cisco, Juniper, Alcatel, …
• Cisco router models: CRS, GSR, 7200 series, …
– Variety of service offerings with wide variety of features
Industrial Issues in Software Performance4
Challenges of Managing Networks • Constant churn
– Customers, services, technology
• Explosive growth– Internet traffic is predicted to double every two years
• http://newsroom.cisco.com/dlls/2008/ekits/Cisco_Visual_Networking_Index_061608.pdf
• More stringent requirements and expectations– Service delivery model of IP is best effort
• Works for original applications of the Internet: file transfer, e-mail and newsgroup
– Applications such as VoIP, IPTV, gaming require strong performance guarantees from the network• Deliver packets reliably within a certain time bound
• Security– Internet’s intrinsic model: “Trust every packet and do not
charge for it”
Industrial Issues in Software Performance5
Tier-1 Service Provider Network
PoP: Point-of-PresenceP: Backbone (core) RouterPE: Provider Edge RouterCE: Customer Edge Router
CPEAccess
Intercity
Metro
CECPE CE
EPEEPE
CPCP
Customer facing PE interfaces
CPCP
CPCP
CPCP
CPCP CPCP
CPCP CPCP
PoPEPEEPE
EPEEPE
EPEEPE
OC-48 or OC-192 DWDM
Rough Numbers•100s of offices•100s of Ps, 1000s of PEs, 10000s of CEs•100,000s of transport facilities
DWDM systems
LEC
PoP
PoPCPCP
CPCPEPEEPE
Industrial Issues in Software Performance6
The Software above it!
Database 1
Databases
Database 2 Database 3 abc def ghi
Ticket Management Mgt 1
xxx
Mgt 3 Mgt 4Mgt 2
xxx
abc
xxx
xyz
abc
xxx
abc
abc
xyz
xyz
abc
def
xyz
nms
ghi
ISE
ghi
abc
abcMgt sys xyz
CFKB
abc
def
abc
xyz
ghi
abc
abc
Transport
xyz
abc
abc
def
ghi
def
ghi
abc
abc
abc
abc
xyz
abc
xyz
def
xyz
abc
abc
xyz
ghi
xyz
abc
abc
def
xyz
abc
xyz
ghi
abc
xyz
xyz
xyz
xyz
xyz
xyz
xyz
xyz
abc
1
2
2
1
3
3
5
5
6
6
7
7
7
7
7
9
9
9 7
7
7
11
11
11
11
11
11
11
11
9
9
9
9 9
11
11 7
11
13
13
7
14
147
11 6
abc
abc
abc abc
xxx 1
19
20
19
20
11
abc
xyz
abc
def
xyz
xyz
abc
abc
abc
abc
xyz
xyz
xyz
xyz
DB3
DB4
ghiabc
DB2
DB1
abc abc
abc ghi
abc
19
20
abc ABC
def
abc
def
Platform for task xyz
Portal abc
abc abc
abc
abc NF-TA CAPRI def
abc
ghi
Platform xyz
abc
abc
xyz
xyz
abc
xyz
abc
xyz
abc
xyz
xxx
abc
7
21Portals
21
21
def
ghi
abct
abc
xxx!
xxx
xxxx
yyy
zzzzz
Platform for doing xyz
xyz
xyz
abce
xyz
abcabc
def
23
23
20
19
abc def
abc ghi
5
xyz abcabcabc
24
24
25
25
xyz
13
def13
7
6
abc
yyy
xyz
xxx
18
18
abc
xyz
gef
26
26
abc
7
7
System for xyz
27
27
Platform for abc
ghi
abc
Platform for xyz
abc
System X
platform for xyz
abce
26
def
def
dsagjag
abc
Dga;ljag;lkj
xyz
xyz
abc
yyz
abc
abc
abc
xyzr
abc1
abc5
abc2
abc3
abc4
xyz
abc
mno
def
ghi
abc
abc
To another sys
1
occ xxx
xyz
28
28
xyz 3
xyz
abc
29
29
xyzabc
29
1
Task xyzs
1 30
To abc
30
30
xyz
Taken from SIGCOMM 2005 Tutorial by Aman Shaikh & Albert Greenberg
Industrial Issues in Software Performance7
Implications for Software Requirement• Need to translate network requirements into
software requirements– Example: what does five-9’s of availability mean for
software on a router?
• Need to determine performance requirement for tomorrow, not today– Accommodate for growth that occurs during expected
life-time of a software
Industrial Issues in Software Performance8
Implications for Software Development• How to write reliable and robust software?
– Systems need to work 24 x 7• Under non-deterministic and unpredictable environment
– Systems should handle errors and overload gracefully– Systems should support hit-less upgrade
• How to write scalable and extensible software?– Software often lives longer than originally anticipated
• How to integrate software with existing systems?– Integration with legacy software often a challenge– Implication for performance
• Cater to lowest common denominator• Exploit opportunities available for parallelism
Industrial Issues in Software Performance9
Implications for Software Testing • How to generate network-like conditions during
testing?– Enumerating all possible conditions is infeasible
• However, “untested feature interaction” can lead to problems
– Creating network scale is nearly impossible• Simulators and emulators do exist, but do not always scale
to the extent of large real networks
Industrial Issues in Software Performance10
My World-View• Knowing performance requirements upfront is vital
– But in real world, this is often not possible– Or performance requirements change/evolve over time
• Due to growth• Longer than expected shelf-life of software
• How to cope with evolving performance requirements?– Analyze how software is likely to scale with input parameters
• Those Big-Oh and Big-Omega’s are important– Use tools to determine actual performance
• Humans not very good judge of performance and bottlenecks– Have a modular design
• Cleanly separate interface and implementation• (Re)-use existing libraries/modules
– Document, document, document, …• So that someone else can pick up the threads• Or you yourself don’t feel lost when you re-visit your code