![Page 1: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/1.jpg)
Pen Testing Is Broken
Zach Grace@ztgrace
![Page 2: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/2.jpg)
zach@bsides:~# id
• Lead Security Consultant at a Fortune 100
• Former Manager of Penetration testing at 403 Labs
• Wisconsin CCDC Red Team member
• @MilSec hacker herder
![Page 3: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/3.jpg)
A Better Approach
• Offense - provide more value to defense
• Defense - a different approach to securing your environment
![Page 4: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/4.jpg)
What is Pen Testing?
• Meant to be a simulated attack
• Proves if someone can break in
• Shows the impact once they’re inside
![Page 5: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/5.jpg)
![Page 6: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/6.jpg)
Pen Testing is the new AV
![Page 7: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/7.jpg)
What’s Broken With Pen Testing?
• Vulnerability focused
• Reporting doesn’t help defenders
• Lack of realistic threat modeling
![Page 8: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/8.jpg)
Vulnerability Focused
• Shutdown a single attack path• False sense of security
![Page 9: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/9.jpg)
Typical Report
![Page 10: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/10.jpg)
Who Ya Modeling?
![Page 11: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/11.jpg)
It’s not if, but when…
![Page 12: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/12.jpg)
Compromise
Detection
Containment
MTD - MTC = ∆
![Page 13: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/13.jpg)
∆ Force
![Page 14: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/14.jpg)
∆ Force Objectives• Perform threat simulations based on
threat modeling
• Breakdown attacks into stages
• Validate detection at each stage, and assist with correlation
• Provide attack use cases besides vulns in reports
![Page 15: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/15.jpg)
Threat Modeling TL;DR
• Who are you?
• What are you going after?
![Page 16: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/16.jpg)
Attack Paths
Target1
2 3
4
5
6
Shared DMZ1 DMZ2Users
![Page 17: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/17.jpg)
What About Defense?
• Delivering large capabilities is hard
• Let’s borrow some methodology from agile application development
![Page 18: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/18.jpg)
Scrum Sprint Cycle
http://en.wikipedia.org/wiki/Scrum_(software_development)
Security Backlog
Sprint Backlog Sprint New
Capabilities
24 h
2-4 weeks
![Page 19: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/19.jpg)
Start With A Backlog
Feature Priority Size
Exfiltration 1 Small
Malware Detection 2 Medium
Lateral Movement 3 Large
Privilege Escalation 4 X-Large
![Page 20: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/20.jpg)
User Story
As a userI would like to search for peopleso that I can find my friends.
![Page 21: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/21.jpg)
Security User Story
As a defenderI would like to monitor netflowso that I can detect lateral movement.
![Page 22: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/22.jpg)
Detailed Backlog
Feature Sec Story Task Size
Lateral Movement
NetFlow PTH 16 hours
Lateral Movement
NetFlow Port Scanning 8 hours
Privilege Escalation
HIDS Mimikatz 1 week
Exfiltration DLP > 50MB 2 weeks
![Page 23: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/23.jpg)
Scrum Sprint Cycle
http://en.wikipedia.org/wiki/Scrum_(software_development)
Security Backlog
Sprint Backlog Sprint New
Capabilities
24 h
2-4 weeks
![Page 24: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/24.jpg)
Putting It Together
![Page 25: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/25.jpg)
Threat Modeling
Targeted spear-phishing attack from an external actor going after the goods.
![Page 26: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/26.jpg)
Phishin’ fo the GoodsMail
The Goods
Victim Admin
![Page 27: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/27.jpg)
Bypass Mail ControlsMail
As a defenderI would like to detect and block malicious doc filesso that I can prevent patient zero.
![Page 28: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/28.jpg)
Executing Malicious Files
Victim
![Page 29: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/29.jpg)
Lateral MovementMail
Victim Admin
![Page 30: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/30.jpg)
Unauthorized Server Access
MailThe
Goods
Victim Admin
![Page 31: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/31.jpg)
Data ExfiltrationMail
The Goods
Victim Admin
![Page 32: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/32.jpg)
Do It AgainMail
The Goods
Victim Admin
![Page 33: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/33.jpg)
Simulation
• Each stage may be a sprint (2-4 weeks)
• Revalidate your controls
• Each technological gap is a business case opportunity
![Page 34: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/34.jpg)
Tips For Offense
• Be a sparring partner
• Incorporate use cases into reports
• Provide more data like PCAPs
• Provide artifacts to reproduce attacks
![Page 35: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/35.jpg)
Tips For Defense
• Build a backlog
• Require pen testing around your security stories
• Ask for more data from the tester
• Rotate your testing firms or rotate your testers
![Page 36: Pen Testing Is Broken - Zach Grace · What is Pen Testing? • Meant to be a simulated attack • Proves if someone can break in • Shows the impact once they’re inside](https://reader034.vdocuments.us/reader034/viewer/2022050513/5f9dad4c6b547a12e458c1f1/html5/thumbnails/36.jpg)
Thank You!@ztgrace