Copyright 2009 © The OWASP FoundationPermission is granted to copy, distribute and/or modify this document under the terms of the OWASP License.

The OWASP Foundation

OWASP

http://www.owasp.org

Threat analysis as methodology for deriving risk-based security tests of web application software

Marco MoranaOWASPmarco.m.morana@gmail.com

IMI 2009 Security Summit

OWASP

Presentation Agenda Security Testing 101 (In 10 steps)

Security requirements derivation Security testing in the SDLC Security testing workflows, techniques and tools Security testing guides Security test reporting and metrics

State of The Art Of Security Testing Effective testing, compliance driven testing, tool centric testing

Risk Based Security Testing Methodology Deriving test cases from cyber-intelligence, attack tree analysis,

attack vectors analysis, ATM/secure architecture analysis

Security Risk Testing Strategy Q&A

2

OWASP

Security Testing 101 (In 10 Steps)

33

OWASP 4

Why Security Testing?How Security Testing Is Different From

Traditional Quality Testing?Test that security controls function as designed

with “positive” testsTest that security controls cannot be abused by

exercising un-intended/abused functionality with “negative” tests

Test to find vulnerabilities in applications with an application scan/pen test

Test to find vulnerabilities in the source code with a source code analysis/secure code review

Test to find security flaws in design with application threat modeling/secure architecture review

OWASP

Testing For Software Security: Quality Assurance vs. Software Assurance

5

http://www.ddj.com/184405196

Security faults: test as built for anything unexpected

Quality faults: test as Designed for what we want to do

OWASP 6

Step 1: Derive Security Requirements

Functional requirements Validate security controls work as designed to function

(e.g. authentication, authorization, encryption etc) Non functional requirements

Validate there are no vulnerabilities and security flaws Security compliance requirements

Validate compliance with FFIEC, GLBA, PCI-DSS, SB 1386, InfoSec, AppSec standards

Threat mitigation requirementsValidate mitigation against threats : Spoofing user

identity, Tampering with the data, Repudiation, Information Disclosure, Denial of service, Elevation of privileges

The application need to implement input filtering and output encoding to mitigate XSS attacks

The authentication control should lock a user after 3 failed attempts

Do not store CC authentication data protect CCH data in storage and mask CCN on display

Mutually authenticate client and server to prevent repudiation attacks such as Man In the Middle (MiTM) attacks

OWASP

Derivation Of Security Requirements To Validate Compliance With Security Standards

7

[PCI-DSS] 6 Develop and Maintain Secure Systems and Applications All vulnerabilities must be corrected. The application must be re-evaluated after the

corrections. The application firewall must detect and prevent web

based attacks such as cross site scripting and SQL injection.

[PCI-DSS] 11 Regularly Test Security Systems and Processes

[PCI-DSS] 11.3.2 External application layer penetration test. For web applications, the tests should include, at a

minimum, the following vulnerabilities: OWASP T10

OWASP 8

Step 2: Use Security Requirement Engineering Techniques such as Use And Misuse Cases

User

Hacker/Malicious User

Brure ForceAuthentication

Enter Username andpassword

Validate PasswordMinimum Length and

ComplexityApplication/Server

Includes

Mitigates

User Authentication

Includes

Includes

Includes

Mitigates

Threatens

Show Generic ErrorMessage

Includes

Includes

Lock Account After N.Failed Login Attempts

Harverst (e.g. guess)Valid User Accounts

Dictionary Attack

Mitigates

Mitigates

OWASP 9

Step 3: Identify Security Testing Tollgates

Requirementsand use cases

Design Test plansCode

Testresults

Fieldfeedback

Securityrequirements

Threat and riskModeling

Risk-basedsecurity tests

Staticanalysis(tools)

Penetrationtesting

Secure Design Review

Iterative approach

Code Review

Risk = Threat x Vulnerability x Cost

What do we need to test,

And how Code review tools

OWASP

Step 4: Integrate Security Testing In Developer’s Workflows Developers Security Tests

Test security of functions, methods and classesSecurity unit/component testsSecure code reviews/security bugs testingDynamic analysis (e.g. application scans)

Main developer’s workflow test check-pointsSecure code reviews :validate compliance with

secure coding standards during coding interactions

Defect management: clean security bugs before check-in code in application builds

10

OWASP

Step 5: Integrate Security Testing in Tester’s Workflows Testers Security Tests

Test security in integrated application buildsValidate application security requirementsAssess and exploit vulnerabilitiesTest for secure configuration in operational environmentFuzz testing, reverse engineering

Tester Workflow Check PointsValidate application security requirements before

release to QA or production: security assuranceRemediate all HIGH RISK vulnerabilities before

release to production according to compliance requirements

11

OWASP

Step 6: Adopt Manual and Automated Security Testing Techniques

1212

Find Vulnerabilities Using the Running Application

Find Vulnerabilities Using the Source Code

AutomatedAutomatedVulnerabilityVulnerabilityScanningScanning

AutomatedAutomatedStatic CodeStatic Code

AnalysisAnalysis

ManualManualPenetrationPenetrationTestingTesting

ManualManualCodeCode

ReviewReview

Combining All Four Techniques is Most Effective

OWASP 13

Step 7: Decide Which Security Testing Tools To Use Vulnerability Scanning

ISS, Foundscan, Nessus, Nikto Penetration Testing (Black Box Testing)

Webinspect, Appscan, Hailstorm, Paros, Peach Binary Analysis/Reverse Engineering

IDA Pro, @stake SmartRisk Source Code Analysis

Fortify, Klockworks, Parasoft, Free Tools (e.g. FindBugs)

Threat ModelingMS TAM, TRIKE, PTA Technologies

Rootkit BackDoor Analysisootkits.org and rootkit.nl

OWASP 14

Step 8: Document Security Testing Cases Using Testing Guides

Information GatheringBusiness Logic TestingAuthentication TestingSession Management TestingData Validation TestingDenial of Service TestingWeb Services TestingAjax Testing

The OWASP Testing Guide

Testing PrinciplesTesting ProcessCustom Web Applications

Black Box TestingGrey Box Testing

Risk and ReportingAppendix: Testing ToolsAppendix: Fuzz Vectors

OWASP

Step 9: Report The Findings

15

http://www.owasp.org/index.php/How_to_write_the_report_of_the_testing_AoC

OWASP

What I Should Put in The Testing Report? The security threat that can potentially exploit

the vulnerability/defect The root cause of security issues (e.g., security

bugs, security flaw) The testing technique being used (e.g.

source code analysis, pen test, threat modeling, security test case)

The remediation of the vulnerability/defect (requirement, design, code configuration change)

The risk rating of the vulnerability (Critical, High, Medium, Low)

16

OWASP

Step 10: Collect Security Testing Metrics Measurement Goals:

Measure vulnerabilities found with pen-tests to validate that are closed before production release

Measure vulnerabilities found with source code analysis and correlate with the ones found in pen tests

Measure time it takes to fix vulnerabilities during coding and during validation tests

Security Metrics:No. of High, Mediums for each applicationNo. of vulnerabilities found in source code vs. the

applicationTime (man/hours) required to close security issues

17

OWASP

State of the Art Of Security Testing

1818

OWASP

Risk Based Security Testing

The main objective is to assert threat mitigation:Mitigate attacks targeting different exposure

factors : internet, extranet, intranetMitigate attacks targeting different threats:

fraud, confidential PII, credit card data, denial of service

Mitigate attacks that might exploit known vulnerabilities (e.g. OWASP T10)

Mitigate attacks that might abuse functionality to cause damage/business impact (e.g. design flaw that allow to reset password insecurely)

19

OWASP

Security Testing From Compliance Perspective Good reasons for compliance security testing:

Avoid fines : $500,000- 800,000 x breachDoing business with third party (e.g. Walmart)Minimal security assurance: CYANot convinced checklists will lead to better security

But not just rely on compliance security tests since data breaches occur even to PCI compliant companies:Heartland Payment Systems, 130 million credit cardsHannaford Bros, 4.2 million credit card

How aware are you about software security assurance ? Beware of bad examples…

20

According to a recent survey (*) “71 percent of companies DO NOT treat PCI as a strategic initiative and 79 percent have experienced a DATA BREACH” (*) http://www.imperva.com/docs/AR_Ponemon_2009_PCI_DSS_Compliance_Survey.pdf

OWASP 21

Security Testing From Security Tools Perspective

MITRE found that all application security tool vendors’ claims put together cover only 45% of the known vulnerability types (over 600 in CWE)

They found very little overlap between tools, so to get 45% you need them all (assuming their claims are true)

Beware of the silver bullet security mentality and false sense of security given by tools !

OWASP

Security Testing From The Application Business Logic Perspective

“76% of financial sites still have at least one critical design flaw that can be exploited to cause financial fraud, identify theft, reputation-brand damage, denial of service to customers” (University Of Michigan study)

“Information leakage, insufficient authentication and authorization, and abuse of the website’s functionality are prime money-makers” (WhiteHat Security Inc.)

22

OWASP

Risk Based Security Testing Methodology

2323

OWASP

Starting With Risk Based Security Testing

24

Basic Security Tests Are Prerequisite:Functional and non functional security

requirements are tested in compliance with standards and policies

Known vulnerabilities are assessed and mitigated

Expand Security Testing to:Real attack scenarios (e.g. cybercrime)Same methods, tools attack vectors and

vulnerability exploits used by the attackersPotential abuse/misuse of business logicAll entry points and access levelsProbing secure design/architecture

OWASP

Risk Based Security Testing Methodology1. Threat Intelligence: gather information about real

attacks to simulate the same attack scenarios2. Threat tree analysis: learn attacker goals and

methods to derive test cases3. Use and misuse cases to validate countermeasures

for potential abuse of functionality4. Attack vector analysis for testing attacks against

defense evasion techniques (e.g. data encoding, automation)

5. Secure architecture analysis for proving countermeasures at different layers: external entity, trust boundary, data flow, process and asset.

25

OWASP

Cyber-Attacks Intelligence Driven Security Tests Gather information from cybercrime

intelligence:Learn from public cyber-crime reports: IC3,

Symantec, McAfee AVERT Labs, Finjan softwareAnalyze the attacks: industry, geographic, local

market, overall business, branch Become your enemy: derive the attack scenarios,

analyze the attack vectors and vulnerabilities exploited by cybercriminals.

Probe your defenses:Try to exploit vulnerabilities using attack vectorsUse same botnet Trojans and PERL script tools to

test application resilience against automation attacks

26

OWASP

Cybercrime Threat Intelligence and Analysis:

27

ATTACK:Attack “xp_cmdshell on MSQL server to upload sniffers to capture CC transactions and ATM PINs from DB, HSM

TEST CASES AGAINST THIS THREAT:1.Test xp_cmdshell is disabled2.Test extended URLs are denied3.Test escape for special characters “”, comma4.Test store procedures run with minimum privileges5.Test SQL Server and IIS run under non-privilege6.Test appserver not use “sa” hardcoded7.Test account locking on mainframes against brute force8.Test access to DB server is internally restricted by IP9.Test a proxy server is used for internet access 10.Test firewall rules are updated11.Test HSM not use commands with PIN in the clear

OWASP

Threat Tree Analysis of Credit Card Compromise Attacks

28

Credit Card Data

Compromise

Man In The Middle/Browser

Attack

Automated SQL Injection Attack

To upload malware

Serve malicious IFRAME to

victim visiting the web site

Phishing Email/Social

Engineering

SQL Injection Exploit

Alter Query To Get CC

data

Exploit Weak Session

Management

Insecure Cryptographic

Storage/Transit

Impersonate user to get

access to CC data

Upload Sniffer To Get

CC data

Session Fixation to

get access to CC data

Attack User/Browser

Attack Web Application

Clickjacking

Serve Invisible Frame that runs

malware

Take Credentials and

CC data from user

Capture Non-Encrypted CC

Data

#2 Test for SQL injection and code injection (Frames) vulnerabilities

#4 Test for session fixation and hijacking

#3 Test encryption of sensitive CC data in storage and transit

#1 Test web application assuming browser compromise and/or automation attacks

OWASP

Security Testing With Attack Libraries

Derive a list of attack vectors that can be used for testing countermeasures such as filtering of encoded cross-site scripting and SQL injection commands.

Start with code injection attacks library:SQL injection attacks HTML (IFRAME) injection attacks Script injection (e.g. cross-site scripting) attacks Command shell injection attacks File injection attacks Server pages injection attacks (e.g. ASP, PHP) Cookie poisoning attacks XML poisoning attacks

29

OWASP

Attack Vectors For Testing Web Applications

30

OWASP

Security Testing Using Application Threat Modeling Analysis

DFD-secure architecture analysis provide testers information about the application scope for risk based testing such as: The location of the application entry points

that need to be tested The access levels that are required to access the

entry pointsThe vulnerabilities that can be exploited at

each layer of the architecture that need to be tested

The countermeasures that need to be tested for mitigation of identified threats.

31

OWASP

Users

Request

Responses

DM

Z (U

se

r/We

b S

erv

er B

ou

nd

ary

)

Message Call

Account/ TransactionQuery Calls

Web Server

ApplicationServer

Application Calls

Encryption +Authentication

Encryption + Authentication

Financial Server

Authentication Data

Re

stric

ted

Ne

two

rk(A

pp

& D

B S

erv

er/F

ina

nc

ial S

erv

er B

ou

nd

ary

)

DatabaseServer

Application Responses

FinancialData

Auth Data

MessageResponse

SQL Query Call

CustomerFinancial

Data

Inte

rna

l (We

b S

erv

er/ A

pp

& D

B S

erv

er B

ou

nd

ary

)

Application Threat Modeling

32

Access Level

External

Access Level

InternalAccess Level

Restricted

I. SpoofingII. Repudiati

on

I. TamperingII. RepudiationIII. Info

DisclosureIV. Denial OF

service

AuthN,EncryptionDigital,signatures,HMAC, TS

I. AuthN, Encryption

II. Digital signatures, HMAC, TS,AuthZ Audit

III. Encryption, AuthZ

IV. Filtering, AuthN

OWASP

Threat Modeling Driven Security Test Cases Spoofing Identity

Test attempts to impersonate a user by sniffing user credentials on the wire or in persistent storage

Test that security tokens (e.g. cookie) issued before authentication cannot be replayed to bypass authentication

Tampering with the data Test with a web proxy that is not possible to tamper

and replay the data Test strength of hashes against replay attacks (e.g.

use of salted hashes) Repudiation

Test mutual authentication and/or digital signatures to trust connection between client and server

Test all security events are audited and logged

33

OWASP

Threat Driven Security Test Cases (Con’t) Information Disclosure

Test that the access of non-public data requires authentication

Test that error messages and exceptions do not disclose useful information to an attacker

Test that processes do not cache or log passwords, session information or PII

Denial of Service (Dos) Test you cannot flood a process with so much data it stops

responding to valid requests. Test that malformed data cannot crash the process

Elevation of Privilege Test user cannot tamper client parameters or forceful

browsing to his elevate privileges Test that a process cannot be forced to load a command

shell, which in turn will execute with elevated privileges

34

OWASP

Threat Driven Security Testing Activities Throughout the SDLC

35

Threat Modeling in the SDLCD

esig

nD

esig

nD

evel

opm

ent

Dev

elop

men

tT

estin

gT

estin

gD

eplo

ymen

tD

eplo

ymen

tD

efin

ition

Def

initi

on Use and Abuse Cases

Security Requirements

Threat ModelingData Flow Diagrams

Attack Trees

Secure Code Reviews

Security Integrated Testing

Vulnerability Assessments

(User Acceptance Test)

Secure Architecture

Modeling

Secure Coding Standards

Secure Configuration &

Installation

Security Testing Guidelines

(System Tests)

Secure Requirements Engineering

Security Testing Guidelines (Unit

Tests)Security Unit

Testing

Vulnerability Assessments (Production)

Identify security requirements to be tested

Identify security flaws and countermeasures to be tested

Identify data flows to be tested

Identify attack scenarios, goals and methods to test

Derive security test cases for integrated system tests

Secure Coding Requirements drive testing during coding Identify security

bugs with secure code analysis

Identify vulnerabilities with pen tests during UAT cyclesTest secure

configuration before production release

OWASP

Security Risk Driven Strategy

3636

OWASP

The Security Risk Testing Strategy

Identify which attacks can be most likely used against your web application and your customers

Use attack methods where difficulty of attack is the least and the impact is the highest

Use misuse cases to test abuse of functionality/business logic

Use the same same attack vectors to try to evade countermeasures/defense mechanisms

Test defense in depth against threats identified in the application threat model

37

OWASP 38

Q&Q U E S T I O N SQ U E S T I O N SA N S W E R SA N S W E R S

OWASP

References

OWASP Testing Guide http://www.owasp.org/index.php/

OWASP_Testing_Guide_v3_Table_of_Contents Testing for Software Security Rethinking security bugs

http://www.ddj.com/184405196 Education Module Embed within SDLC

http://www.owasp.org/index.php?title=Education_Module_Embed_within_SDLC&setlang=es

OWASP CAL9000 Project http://www.owasp.org/index.php/

Category:OWASP_CAL9000_Project Security Flaws Identification and Technical Risk Analysis Through

Threat Modeling http://www.net-security.org/dl/insecure/INSECURE-Mag-17.pdf

OWASP Application Threat Modeling http://www.owasp.org/index.php/Application_Threat_Modeling

39