tom levey (appdynamics) - the devops payrise
TRANSCRIPT
![Page 1: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/1.jpg)
Realising the true value of DevOpsThe DevOps Payrise
![Page 2: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/2.jpg)
Tom LeveyTech Evangelist
@tlevey
![Page 3: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/3.jpg)
DevOps?
![Page 4: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/4.jpg)
Developers working together with
Operations to get things done faster in an
automated and repeatable way
![Page 5: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/5.jpg)
DevOps Success?
![Page 6: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/6.jpg)
2005 Portal-services & handset testing
![Page 7: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/7.jpg)
my����������� ������������������ desk
![Page 8: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/8.jpg)
space
flashing����������� ������������������ lights
![Page 9: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/9.jpg)
IREGInternational Roaming
Expert Group
![Page 10: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/10.jpg)
International roaming
voice calls
video calls
signalingnetwork backbone
portal services
messaging
france network
spain network
italy network
germany network
norway network
![Page 11: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/11.jpg)
Runbook1. Call L2 ops team 2. Call infrastructure ops 3. Call signaling partner 4. Call radio ops 5. Call backbone provider 6. Call international L2 ops team 7. Wait for reply... 8. Repeat 1-7 until resolution
![Page 12: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/12.jpg)
![Page 13: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/13.jpg)
“Everything is fine from our end.”
OPS:
![Page 14: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/14.jpg)
Typical Day1. Open 30 new tickets 2. Make 200 phone calls 3. Attend executive P1 status update meeting 4. Argue about what a P1 and P2 really is 5. Reprioritise P2 tickets to P1 6. Reprioritise P3 tickets to P2 7. Close tickets as ‘Cannot reproduce’ or ‘Duplicate’
![Page 15: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/15.jpg)
What did I care about?
![Page 16: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/16.jpg)
P1’sSLA’s
What did I really care about?
![Page 17: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/17.jpg)
What did the Business care about?
£
![Page 18: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/18.jpg)
P1 = £Even though the business could never prove it.
![Page 19: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/19.jpg)
How the Business really viewed me
![Page 20: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/20.jpg)
How L2 & L3 Support viewed me
![Page 21: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/21.jpg)
How Ops viewed me
![Page 22: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/22.jpg)
False Alarms
Site is down
404 Errors
My search is slow
![Page 23: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/23.jpg)
2am Friday - #FFS
We have had an alert that the load on one of your staging servers is critical.
![Page 24: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/24.jpg)
How much time do false alarms waste?
Role Hours Per Week Cost Per Week Cost Per Year
Me 20 £400 £20,800
L2 10 £200 £10,400
L3 15 £300 £15,600
Hosting 6 £120 £6240
Network 6 £120 £6240
CMS 10 £200 £10,400
Total 55 £1,340 £69,680
Conservative estimates assuming £20/hour
![Page 25: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/25.jpg)
How much revenue did the business lose?
No idea
![Page 26: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/26.jpg)
Typical Day1. Open 30 new tickets 2. Make 300 phone calls 3. Attend executive P1 status update meeting 4. Argue about what a P1 and P2 really is 5. Reprioritize P2 tickets to P1 6. Reprioritize P3 tickets to P2 7. Close tickets as ‘Cannot reproduce’ or ‘Duplicate’
![Page 27: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/27.jpg)
Things I wish I’d had
1. Automation
2. Collaboration
3. Better Tooling
4. Business Metrics
![Page 28: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/28.jpg)
Things I wish I’d done
1. Baseline where I started
2. Measured my progress
3. Calculated Business Impact
4. Promote success not problems
5. Proved my value
![Page 29: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/29.jpg)
Present Day
![Page 30: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/30.jpg)
3 billion daily transactions
250 milliseconds
500+ updates/yr
![Page 31: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/31.jpg)
1 million+ servers
100 million GB
1,000 man years
1,500 miles
Konstantin Karpov
Users Expectations
![Page 32: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/32.jpg)
Web server 1
Internet FirewallLoad
Balancer
Web server 2
Database
![Page 33: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/33.jpg)
Napkin architecture…
![Page 34: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/34.jpg)
Key:
= bad
= not bad
![Page 35: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/35.jpg)
Pre$Produc)on+APM+–+“Non+Produc)on+Data”+
Development Operations
Dev Test Staging Live
Monitor & Manage Profile QA Load Test
Pre-Production Production
![Page 36: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/36.jpg)
Produc'on)APM)–)“Produc'on)Data”)
6
Development Operations
Dev Test Staging Live
Monitor & Manage
Pre-Production Production
Profile QA Load Test
![Page 37: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/37.jpg)
tools can be helpful
![Page 38: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/38.jpg)
right tools
right hands
right use
![Page 39: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/39.jpg)
How much time and £ do these tools save?
INFRASTRUCTURE AUTOMATION
![Page 40: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/40.jpg)
How much time and £ do these tools save?
DEPLOYMENT AUTOMATION
![Page 41: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/41.jpg)
How much time and £ do these tools save?
LOG AUTOMATION
LogStash
![Page 42: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/42.jpg)
Monitoring
How much time and $ do these tools save?
![Page 43: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/43.jpg)
![Page 44: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/44.jpg)
severe outage?
![Page 45: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/45.jpg)
PLAN FOR FAILURE!be stronger than the weakest link
![Page 46: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/46.jpg)
Traditional monitoring approach is limited
APPLICATION
BUSINESS TRANSACTION
Server
OS DB
MQ
Web
JVM
Silo’d domain visibility
EXISTING APPROACH
EXPANDED APPROACH
Business transaction
99.9% 99.9% 99.9%99.9%
END USER EXPERIENCE
![Page 47: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/47.jpg)
How many of you use performance
management tools?
![Page 48: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/48.jpg)
![Page 49: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/49.jpg)
Identify early !Troubleshoot fast !Resolve quickly !Quantify impact
x
![Page 50: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/50.jpg)
FOCUS
![Page 51: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/51.jpg)
![Page 52: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/52.jpg)
automatically map your application
![Page 53: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/53.jpg)
Who is using my app?
![Page 54: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/54.jpg)
device, browser, network, Carrier…
![Page 55: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/55.jpg)
DETAILED CRASH ANALYSIS
![Page 56: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/56.jpg)
device, carrier & browser performance
![Page 57: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/57.jpg)
§
Show me the daily deals in my local area…
47 nodes
![Page 58: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/58.jpg)
Search for a hotel in London…
215 nodes
![Page 59: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/59.jpg)
Executed code diagnostics
![Page 60: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/60.jpg)
Understand the impact of slow performance
* Screenshot from US e-Commerce AppDynamics Customer
Application Revenue
Application Errors
Application Response time
$64,499 per min
$11,987 per min
10.1 s
100 ms
![Page 61: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/61.jpg)
Understand the benefit of an application release
Application Revenue
Application Response time
code release 1
code release 2
code release 3
$44,499 per min
$58,237 per min
1.9 s3.1 sec
![Page 62: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/62.jpg)
Collaboration = ??
Productivity saves time Time is money.
![Page 63: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/63.jpg)
DevOps Tasks Annual Cost £
Annual Benefit £
Infrastructure Automation ? ?
Deployment Automation ? ?
Log Automation ? ?
Monitoring ? ?
Collaboration Pride ?
Total X Y
DevOps Value £ Y - X
![Page 64: Tom Levey (AppDynamics) - The DevOps Payrise](https://reader033.vdocuments.us/reader033/viewer/2022042522/55d516bebb61eb726b8b4738/html5/thumbnails/64.jpg)
!
Tell the Business how much you save them.
DevOps
Automation & collaboration Time Money= =