CHOOSING THE BEST WEB APP SECURITY SCANNER

WHO AM I ?Chirita Ionel

Application Security Analyst @

OWASP Chapter board member

WHAT DO WE WANT FROM A SCANNER? Wide Coverage Fast scans Low number of false positives Low number of false negatives Scalability Easy to use Permanent vulnerability database updates To be Cheap !?

W.A.S. EVALUATION CRITERIA Hardware Requirements & support Protocol support Authentication Session management Crawling Data Parsing Testing Command and control Reporting

HARDWARE REQUIREMENTS & SUPPORT

Thick client vs cloud

PROTOCOL SUPPORTTransport support

HTTP1.0 & HTTP1.1 SSL/TLS HTTP keep alive HTTP compression HTTP user agent configuration

Proxy support

HTTP1.0 & HTTP1.1 proxy Socks 4 proxy Socks 5 proxy PAC file support

AUTHENTICATION Basic Digest HTTP negotiate – NTLM & Kerberos Html form-based

Automated Scripted Non-automated

Single sign on Client SSL certificates Other

SESSION MANAGEMENT Session management capabilities

Start a new session Detect if the session is expired Reacquire session token

Session management token type support HTTP cookies HTTP parameters HTTP URL path

Session token detection Session token refresh policy

CRAWLING Define starting URL

Define additional hostname or exclusions for specific criteria

Support automated from submission

Detect error pages and custom 404 pages

Redirect support

DATA PARSING HTML JavaScript VBScript XML Plaintext ActiveX Objects Flash

TESTING

COMMAND AND CONTROL Schedule scans

Pause / resume

Real-time status of running scans

Run multiple scans simultaneously

GUI, CLI and web based interface

Extensibility & interoperability

REPORTING Executive summary

Technical detailed report

Delta reports

Compliance report

Customization

Report data file format

SO YOU SHOULD JUST USE THE BEST SCANNER, RIGHT? Why do you mean by “best” ?

Or the cheapest ?

By Larry Suto

WHAT ABOUT …

… running each vendor's scanner against each of the vendor's test sites and comparing the results

SUMMARY OF RESULTS

Acunetix

IBM Appscan

BurpSuite

Hailstorm

NTOSpider

Qualys

HP Webinspect

0 20 40 60 80 100 120

Falsely Reported and Missed Vulnerabilitites

False Negative False Positive

Acunetix

IBM Appscan

BurpSuite

Hailstorm

NTOSpider

Qualys

HP Webinspect

0 20 40 60 80 100 120 140 160

Vulnerability FindingsTrained Point & Shoot

SUMMARY OF RESULTS

Acuneti

x

IBM Appsca

n

BurpSu

ite

Hailstor

m

NTOSp

ider

Qualys

HP Web

inspe

ct0

20406080

100120140160

Vuln's Found Vuln's Missed FP's Reported

CASE STUDY

By Chirita Ionel

FP's Rported

0 1 2 3 4 5 6 7 8 9

FP's reportedIBM Qualys WebInspectVeracode Acunetix

Vuln's Found

0 1 2 3 4 5 6 7 8 9 10

Vuln's FoundIBM Qualys WebInspectVeracode Acunetix

Scan Time

0 1 2 3 4 5 6 7 8 9 10

Scan TimeIBM Qualys WebInspectVeracode Acunetix

Stability

0 1 2 3 4 5 6 7 8 9 10

Stability IBM Qualys WebInspectVeracode Acunetix

ON TOP OF ALL -> GARTNER MAGIC QUADRANT

SO ?