devops & security from an enterprise toolsmith's perspective
TRANSCRIPT
![Page 1: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/1.jpg)
Go Fast AND Be Secure?DevOps and Security from an Enterprise
Toolsmith’s Perspective
Alex Honor Damon Edwards
![Page 2: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/2.jpg)
@damonedwards
Damon Edwards Alex Honor
@alexhonor
![Page 3: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/3.jpg)
DevOps ConsultingAutomation Design
OperationsTools
![Page 4: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/4.jpg)
Business Demands
Our #1 priority is moving faster than our competitors!
![Page 5: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/5.jpg)
IT Responds
![Page 6: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/6.jpg)
IT Responds
![Page 7: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/7.jpg)
IT Responds
![Page 8: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/8.jpg)
… but what about security and compliance?
![Page 9: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/9.jpg)
Business Demands
Our #1 priority is moving faster than our competitors!
Our #1 priority is security and compliance! and
![Page 10: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/10.jpg)
IT Under Pressure
![Page 11: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/11.jpg)
Can we go faster and be more secure?
![Page 12: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/12.jpg)
Can we go faster and be more secure?
![Page 13: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/13.jpg)
What gets in the way?
![Page 14: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/14.jpg)
Everything is different
![Page 15: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/15.jpg)
Everything is different● Many servers hand built
![Page 16: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/16.jpg)
Everything is different● Many servers hand built● Custom is the rule
![Page 17: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/17.jpg)
Everything is different● Many servers hand built● Custom is the rule● Inconsistent access control
policy and rules
![Page 18: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/18.jpg)
Everything is different● Many servers hand built● Custom is the rule● Inconsistent access control
policy and rules● Network spaghetti topology
reflects snowflakes
![Page 19: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/19.jpg)
Everything is different● Many servers hand built● Custom is the rule● Inconsistent access control
policy and rules● Network spaghetti topology
reflects snowflakes● … it’s always a network
problem ;-)
![Page 20: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/20.jpg)
Multiplied by Datacenter● Geographically spread ● Generations of
hardware & software ● WAN latencies and
bandwidths ● Sometimes outsourced
![Page 21: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/21.jpg)
Culture clashes between silos
![Page 22: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/22.jpg)
Culture clashes between silos● “Too much change breaks
stuff” - Ops
![Page 23: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/23.jpg)
Culture clashes between silos● “Too much change breaks
stuff” - Ops● “Let me do it myself” - Dev
![Page 24: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/24.jpg)
Culture clashes between silos● “Too much change breaks
stuff” - Ops● “Let me do it myself” - Dev● “This is dangerous!” - Sec
![Page 25: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/25.jpg)
Culture clashes between silos● “Too much change breaks
stuff” - Ops● “Let me do it myself” - Dev● “This is dangerous!” - Sec● “It’s not ready” - QA
![Page 26: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/26.jpg)
Culture clashes between silos● “Too much change breaks
stuff” - Ops● “Let me do it myself” - Dev● “This is dangerous!” - Sec● “It’s not ready” - QA● Finger pointing - everyone
![Page 27: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/27.jpg)
Bureaucracy to get anything delivered“Have you got 27B-6?” - said a guy, in a downstream silo
“I’m a bit of a stickler for paperwork”
“All I need is a ACL/VIP/etc”
![Page 28: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/28.jpg)
It always ends up an escalation● Who yells loudest ● Cube driveby and
who you know ● Crisis at deadline
or outage ● Sometimes still a
rubber stamp
![Page 29: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/29.jpg)
Hard to see how delivery work gets done across the organization
![Page 30: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/30.jpg)
Process Islands Multiple Development teams out here somewhere
![Page 31: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/31.jpg)
Process Islands
“I know there are problems delivering, not sure where, but I know they are outside my island of control”
“We all have the best intentions from our perspective
![Page 32: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/32.jpg)
Process Islands
![Page 33: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/33.jpg)
Process Islands
![Page 34: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/34.jpg)
Process Islands
![Page 35: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/35.jpg)
Process IslandsI really wish to deploy multiple times daily
Friday evening
![Page 36: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/36.jpg)
Process Islands
Monday morning
![Page 37: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/37.jpg)
Process Islands
![Page 38: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/38.jpg)
Process Islands
Everybody on bridge call with the boss
![Page 39: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/39.jpg)
Complicated and self inflicted ● Left hand doesnt know
what the right hand doing
● “Bandaids” and “exception is the rule”
● Telephone and Tribal knowledge
● Low MTTD/MTTR
![Page 40: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/40.jpg)
How do we know when things are getting any better?
![Page 41: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/41.jpg)
You’ll know you are better when...
![Page 42: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/42.jpg)
You’ll know you are better when...● Security policy is applied reliably and consistently
![Page 43: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/43.jpg)
You’ll know you are better when...● Security policy is applied reliably and consistently● Security isn’t the bottleneck
![Page 44: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/44.jpg)
You’ll know you are better when...● Security policy is applied reliably and consistently● Security isn’t the bottleneck● An audit trail is easy to pull together
![Page 45: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/45.jpg)
You’ll know you are better when...● Security policy is applied reliably and consistently● Security isn’t the bottleneck● An audit trail is easy to pull together● Security engineers aren’t left out until the end of the party (or
never consulted)
![Page 46: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/46.jpg)
You’ll know you are better when...● Security policy is applied reliably and consistently● Security isn’t the bottleneck● An audit trail is easy to pull together● Security engineers aren’t left out until the end of the party (or
never consulted)● Everyone has the control they need (without root)
![Page 47: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/47.jpg)
You’ll know you are better when...● Security policy is applied reliably and consistently● Security isn’t the bottleneck● An audit trail is easy to pull together● Security engineers aren’t left out until the end of the party (or
never consulted)● Everyone has the control they need (without root)● Nobody feels like they are having the rug pulled out from
underneath them
![Page 48: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/48.jpg)
Shift left: Host OS SDLC
Collaborate with source code
Artifacts move through the “supply chain”
![Page 49: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/49.jpg)
Bastion host
![Page 50: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/50.jpg)
Bastion host
● centralized access point for authorized access
![Page 51: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/51.jpg)
Bastion host
● centralized access point for authorized access
● disallow home run connections
![Page 52: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/52.jpg)
Bastion host
● centralized access point for authorized access
● disallow home run connections
● dispatcher interfaces remote execution layer
![Page 53: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/53.jpg)
Bastion host
● centralized access point for authorized access
● disallow home run connections
● dispatcher interfaces remote execution layer
● hides network complexity like jump boxes per DC
![Page 54: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/54.jpg)
Bastion host
● centralized access point for authorized access
● disallow home run connections
● dispatcher interfaces remote execution layer
● hides network complexity like jump boxes per DC
![Page 55: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/55.jpg)
User traceability: Delegate account
● User logs in as himself to bastion host ● Remote commands and processes run
under a service account ● Eg, SSH keys used for delegate account
identity
![Page 56: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/56.jpg)
User traceability: End to end
● User logs in as himself to bastion host ● Remote commands executed using
same user account ● Eg., User may raise privilege via sudo
![Page 57: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/57.jpg)
White List and Wrapper
● No ad-hoc interactive logins. ● Use wrapper script and a white list ● Escalate privilege with sudo ● Not foolproof! SELinux still considered too hard for most
eg.: ssh forced command (~/.ssh/authorized_keys: command=wrapper.sh and $SSH_ORIGINAL_COMMAND)
![Page 58: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/58.jpg)
Leverage the toolchain to enforce policy
![Page 59: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/59.jpg)
Leverage the toolchain to enforce policyDesign and code reviews
![Page 60: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/60.jpg)
Leverage the toolchain to enforce policyDesign and code reviews
Code and binary scans
![Page 61: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/61.jpg)
Leverage the toolchain to enforce policyDesign and code reviews
Code and binary scans
“Bake” security tests into your “immune system”
![Page 62: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/62.jpg)
Leverage the toolchain to enforce policyDesign and code reviews
Code and binary scans
“Bake” security tests into your “immune system”
Component vulnerability and governance
![Page 63: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/63.jpg)
Leverage the toolchain to enforce policyDesign and code reviews
Code and binary scans
“Bake” security tests into your “immune system”
Component vulnerability and governance
Access policy and operational security checks
![Page 64: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/64.jpg)
Automate Evidence Collection for Audits
![Page 65: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/65.jpg)
Automate Evidence Collection for AuditsWhat’s the change?
![Page 66: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/66.jpg)
Automate Evidence Collection for AuditsWhat’s the change?
How did you validate the change?
![Page 67: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/67.jpg)
Automate Evidence Collection for AuditsWhat’s the change?
How did you validate the change?
How was the change distributed?
![Page 68: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/68.jpg)
Automate Evidence Collection for AuditsWhat’s the change?
How did you validate the change?
How was the change distributed?
Who did what when and where?
![Page 69: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/69.jpg)
Automate Evidence Collection for AuditsWhat’s the change?
How did you validate the change?
How was the change distributed?
Who did what when and where?
What executed on the node?
![Page 70: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/70.jpg)
Summary
● Shift left ● Bastion host ● User traceability ● White lists and wrappers ● Leverage the toolchain to enforce policy ● Automate evidence collection for audits
![Page 71: DevOps & Security from an Enterprise Toolsmith's Perspective](https://reader030.vdocuments.us/reader030/viewer/2022032421/55a6c0ed1a28ab36688b48d8/html5/thumbnails/71.jpg)
● Shift left ● Bastion host ● User traceability ● White lists and wrappers ● Leverage the toolchain to enforce policy ● Automate evidence collection for audits ● ?
Summary