web application security testing automation.. copyright © 2008 deloitte touche tohmatsu. all rights...

Web Application Security Testing

Automation.

Copyright © 2008 Deloitte Touche Tohmatsu. All rights reserved. 2

What types of automated testing are there?What does web application security assessment comprise?How much can tools help?Where is it best to use these tools?

Agenda

What types of automated testing are there?

3


When can you test?

User Acceptance

Testing

User Acceptance

Testing

Project BasedDevelopmentProject BasedDevelopment

FunctionalTesting

FunctionalTesting

Non-Functional Testing

Non-Functional Testing

PilotPre ProductionProduction

Thank God its gone live party.

Performance & Volume TestingPerformance & Volume Testing

Feature requests

TEST

HERE?

TEST

HERE?

TEST

HERE?

TEST

HERE?

TEST

HERE?

TEST

HERE?

TEST

HERE?

TEST

HERE?

TEST

HERE?

TEST

HERE?

BAUdevelopment

BAUtesting


•Raw Source Code Review• Get the code and use software configured with

rules to find exceptions and investigate them


Source Code Rules

Analysis Raw Results

(means something to a developer)

Human review

Findings

(means something to

a project manager)

Source Secure Programming with Static Analysis Chess & West


•Integration into development environment

• Static analysis as you go• Write some code, push to webserver, do some

“black box testing”• Hmm what’s the first thing the developer will skip

when he is under pressure to ship code



•Integration into test software• Get some test data• Capture some UAT test scripts• Run those UAT scripts• Use those test scripts to do some “black box”

testing• Try and persuade a developer that the defect is a

defect • Try and find some project managers to agree who

is to pay to fix the defect • Don’t expect your UAT test team to do security

testing, they are usually lovely people, as they deal with the business



•Assessment from network• It’s ready to go, let’s do a final check..• With some test data walk the application logic, ALL

of the application logic • Scan away• Try and read the report before the project goes

live • Try and find a developer to educate?



•Fuzzing• Aka I’ve run out of ideas, lets just bash away until

something weird happens with input validation or business logic




Method Pros Cons

Source Code Review

Can be done at any time during/after development

Access to source code required, think contractuals

Development Environment Integrated

Can be leveraged by the developer to help educate them.

Can only be done during development

Test EnvironmentIntegrated

Testing is when most test data is hopefully available

Can only be done during testingIf you find a major input validation problem during test you will have to repeat UAT testing!

From Network Can be done at any time

Can cause a Denial of Service to the application

Fuzzing Application has to be operational

Can be slow over the internet

How much can tools help?

11


What does the testing comprise?

Web Application Security Assessment Typical Breakdown of Effort

13%

13%

25%

49%

AuthenticationSession ManagementInput ValidationBusiness Logic

There is a lot of manual testing involved in web application security testing

The majority of findings are related to poor implementation of role based access controls and “business logic flows”. Hence most effort is directed towards business logic testing.


What are we looking for?

0%

10%

20%

30%

40%

50%

60%

70%u

nva

lida

ted

pa

ram

ete

rs

bro

ken

acc

ess

con

tro

l

bro

ken

au

the

ntic

atio

n &

sess

ion

mg

mt

cro

ss-s

itesc

riptin

g

bu

ffer

ove

rflo

ws

com

ma

nd

inje

ctio

n

vuln

era

ble

err

or

ha

nd

ling

inse

cure

cryp

tog

rap

hy

rem

ote

ad

min

serv

er

mis

con

figu

ratio

n

Incidence of 10 common web application vulnerabilities in applications recently tested by Deloitte in the UK. (Vulnerability classifications defined by the Open Web Application Security Project – www.owasp.org)

% o

f te

sted

Web

App

licat

ions

sus

cep

tible

to

vuln

era

bilit

y


How web application scanners work

• Thankfully stolen from http://www.blackhat.com/presentations/win-usa-04/bh-win-04-grossman/bh-win-04-grossman-up.pdf

http://www.blackhat.com/presentations/win-usa-04/bh-win-04-grossman/bh-win-04-grossman-up.pdf

http://www.blackhat.com/presentations/win-usa-04/bh-win-04-grossman/bh-win-04-grossman-up.pdf


•Requirement for test data due to multi-page sequences•Dynamically produced content•Single Sign On/Identity Management/NTLM/Kerberos wackiness•Client side code (bad architect, bad architect!)•Non standard error messages (good developer!)•Denial of Service to application, email system, network monitoring etc. •Anti-automation

Challenges of automated scanning


How much can tools help?

Reduction in Effort

0

20

40

60

80

100

120

Authe

ntica

tion

Sessio

n Man

agem

ent

Inpu

t Valid

ation

Manual Automated

Some aspects of testing can be automated and reduce effort

Other aspects of testing from automation are improved by reduction in human errors


How good are they at finding defects?

A1 - XSS A2 - Injection Flaws

A3 - Malicious File Execution

A4 - Insecure Direct Object Reference

A5 - CSRF


How good are they at finding defects?

A6 - Information leakage and improper error handling

A7 - Broken Authentication and Session Management

A8 - Insecure cryptographic storage

A9 - Insecure communications

A10 - Failure to restrict URL access

Stolen with thanks from http://jeremiahgrossman.blogspot.com/2007/05/web-application-scan-o-meter.html

http://jeremiahgrossman.blogspot.com/2007/05/web-application-scan-o-meter.html

http://jeremiahgrossman.blogspot.com/2007/05/web-application-scan-o-meter.html

Where is it best to use these tools?

19


Where to use?

Method A Good situation to use?

Source Code Review

Outsourced project developmentBAU development

Development Environment Integrated

Education of BAU developers?

Test EnvironmentIntegrated

Hmmm?

From Network Scanning masses of brochure-ware sites for poor input validation and problems like XSS and SQL injection.

Fuzzing Vulnerability research


Manual and Automated Testing complement each other

Method Pros Cons

Manual Picks up business logic flaws.Flexible in the face of an unfinished/unreliable application or test environment

Sample based approach may miss instances of “low hanging fruit”.

Automated

Checks for boring vulnerabilities so you don’t have to (e.g. information disclosure, backups of files, XSS) can be done more efficiently and comprehensively

Doesn’t pick up the really important business logic flaws

Inflexible if the application is not completed.


•Of the automated tools, source code review tools are most flexible as they can be used at any point in the development cycle•Manual testing and automated testing complement each other

Conclusion

web application security testing automation.. copyright © 2008 deloitte touche tohmatsu. all rights...

Documents

deloitte touche tohmatsu

business logic testing

code copyright

business copyright

lot of manual testing

test datacapture

test softwareget

uat test scriptsrun