introduction to application security testing
TRANSCRIPT
Security Verified
© 2012 iCode information security All rights reserved
Introducing Security Testing
Mohamed Ridha Chebbi, CISSP [email protected] Mobile : +216 26 88 10 11
Security Verified
© 2012 iCode information security All rights reserved
Analyzing Applications with a Security Mindset
Testing the security of software requires a certain mindset, as a security test engineer needs to look at software from the perspective of someone who wants to attack it.
Adopting this mindset involves understanding the main information sources for security testing and how input can affect software.
Testing the security of a given piece of software often entails searching a very large set of inputs in order to find issues that can compromise that security.
Automation plays an important role in this search.
Security Verified
© 2012 iCode information security All rights reserved
The Value of Security Testing
In the past, Microsoft products used to be the primary targets for attackers. As a result of their prominent position in the industry, Microsoft operating systems received high scrutiny from the security community. This coupled with the fact that Microsoft products did contain a fair number of vulnerabilities, hackers and vulnerability finders had little need and interest in searching other platforms. The ubiquity of Microsoft products exacerbated this situation, as viruses and worms were able to cause epidemics by taking advantage of vulnerabilities in commonly used server and client applications. The Security Development Life Cycle (SDL) introduced at Microsoft in 2002 has raised the bar. Vulnerability finders have to work harder to penetrate Microsoft applications, and the focus of the security research community is now shifting to those vendors and platforms once considered to be secure. Well-engineered security testing plays a key role in the effectiveness of the SDL. Vulnerabilities found during the SDL have less chance of damaging customer confidence in Microsoft, and removing critical flaws before shipping leaves attackers with fewer reasons to focus on Microsoft.
Security Verified
© 2012 iCode information security All rights reserved
Security Testing and the SDL
Anecdotal evidence indicates that engineers spend less time testing for security bugs in products covered by the SDL. As the design and coding for these products is more secure in the first place, security testing in the SDL is more concerned with verifying that the product is secure, rather than with finding and eliminating security bugs. The purpose of security testing in the SDL is to allow the product team to perform a thorough assessment of the security and privacy of their application. Performing security testing during the verification phase of the SDL allows the product team to:
Perform adequate penetration testing activities on new and legacy code. Verify that the application is adequately reviewed for security vulnerabilities. Ensure that threats against the application are properly mitigated. Document the evidence indicating that users can trust the application. Perform a security push for those legacy components that were not covered by the SDL.
Security Verified
© 2012 iCode information security All rights reserved
Functional Testing vs Security Testing
Security testing and functional testing differ in four main areas:
Identifying helpful sources of information for test planning : Functional testing involves testing an application under realistic user scenarios, while in security testing the test engineer frequently deals with scenarios that may not be realistic for the common user.
Choosing an appropriate set of inputs : During functional testing the test engineer will most commonly use expected inputs derived from use cases, whereas during security testing the test engineer is required to use unexpected and malicious inputs that may expose a security problem.
Automating security tests. Interfaces that are not visible to the human eye are most interesting for security testing. In order to be viewed, such interfaces often require the use of specialized tools. Security test engineers must be familiar with these tools and their limitations. During functional testing it is simpler to automate a testing process, as results for a specific test are easily verified. Establishing whether a security defect has been triggered during security testing is much harder.
Deciding when testing is complete : When carrying out functional testing, it is straightforward to determine when testing is complete. When carrying out security testing, however, deciding when to stop is more complicated because the test engineer needs to help to quantify the risk and decide whether enough testing has been done to declare the product "safe".
Security Verified
© 2012 iCode information security All rights reserved
Gathering Information for Security Testing
The User Interface as a Source of Information :
The user interface is the first place to investigate when planning security testing. The information on the screen is primarily directed at legitimate users, but it is important to remember that hackers are users as well, and they can learn quite a bit from this information.
Identify the Field :
When testing a Web application, testers should be aware of how their interactions with the client will affect the behavior of the server application
Error Messages as a Source of Information :
Many applications output error messages to their users to inform them on what the expected format of input is. This can provide very useful information to the tester on the possible assumptions that developers made on the type of data that are expected through a given input vector.
Abuse Cases as a Source of Information :
Whereas functional test engineers consider use cases, security test engineers must consider abuse cases.
Abuse cases can often be gleaned from sources of historical information such as:
CERT and Bugtraq, which contain security advisories and discussions about existing and new vulnerabilities.
Black-hat literature.
Security Verified
© 2012 iCode information security All rights reserved
Considering Inputs with a Security Testing Mindset
Identifying Entry Points of the Application :
Not all inputs come from the keyboard or mouse. From a security perspective, the Graphical User Interface (GUI) is often the least of concerns, and testers need to worry more about input sources that are not quite so visible. In fact, a security tester has to learn not to think like a GUI tester. There are four sources of input that need to be considered when planning security testing :
OS/RTE : Input from the OS and the Run-time Environment (RTE) includes input from the OS APIs and any run-time environment that is supporting the application, such as the Common Language Run-time (CLR). User Interface (UI) : Examples of user interfaces are a local GUI and a Web front-end. External Resource : A Web server and a database are examples of resources that are external to the application. File System : Input from the file system includes temporary files, registry values, and configuration files.
Security Verified
© 2012 iCode information security All rights reserved
Categorizing Inputs
Categorizing Dangerous Inputs
Lists of dangerous inputs are a key asset for security test engineers.
Within an organization, such lists are often kept secret for fear that they may be used against a system by attackers.
A group should be diligent about doing the necessary research to ensure it has a complete list, as well as ensuring that everyone in the organization is aware of such a list and utilizes it.
Dangerous input can be categorized into the following groups:
Long strings
Format strings
Numeric boundaries
Scripts
Code
OS commands
Control characters
Error codes
Return values
Note that such lists are never static
Security Verified
© 2012 iCode information security All rights reserved
Automating Security Testing
The Value of Automated Security Testing Larger coverage : overage refers to the percentage of security-critical functions that are exercised by different inputs during a security test. Regression testing : During regression testing an application is tested for newly introduced bugs, usually by re-running previous tests and checking whether previously fixed faults have re-emerged Efficiency : Automated security tests are more efficient than manual testing. Automated tests are repeatable with no human intervention, as opposed to manual testing, which is far more time-consuming.
Automation allows for more targeted manual testing as they disclose areas of potential weakness in various execution, trust, and I/O boundaries, which means the test engineer need not try to find all weaknesses manually, but can focus on key areas and provide feedback to threat modeling and other phases of SDL.
Security Verified
© 2012 iCode information security All rights reserved
Generating the Input
In order to generate inputs that are dangerous to your application, a list detailing all possible inputs must be created. This list might be generated in real time as the software is being tested, manually or by using a fuzz test automation suite, or it may already be available to you from other security tests applied to similar products. If the latter, you must keep in mind that often a list is not cross-applicable, and you may need to modify it or create your own list.
Some interfaces, such as the network and the file system, lend themselves well to automation. For other interfaces it might be more difficult to automate testing because the instrumentation of them is more difficult and/or because their outputs are harder to observe.
Security Verified
© 2012 iCode information security All rights reserved
Software you Actually Build
The actual behavior of an application, as coded by developers, rarely matches perfectly with the intended software behavior, as specified in the design documents. Most security bugs are found in the area where behavior that was not part of the application’s intended behavior has been inadvertently implemented by developers.
Security Verified
© 2012 iCode information security All rights reserved
Quantifying the Risk
Quantifying the Risk : To help quantify the risk, you have to ask the following: Was the process which was established to prevent and detect security flaws fully implemented?
How many unmitigated vulnerabilities are currently extant?
Is additional testing necessary?
What can we do to make this product safer?
Assigning Severity: To help test engineers assign severity to defects, the SDL has established four different levels of severity:
Critical : Remote, anonymous user escalation of privilege
Arbitrary code execution
Important DoS (low bandwidth attack, blue-screen, or long duration)
Local elevation of privilege
Information disclosure with privacy implications
Tampering with user data
Spoofing a user or computer
Moderate Temporary DoS
Information disclosure, general
Low Non-persistent or hard-to-replicate issues
Security Verified
© 2012 iCode information security All rights reserved
Review the Process
Collecting the Proper Metrics: To make a well-informed decision on whether to stop testing requires relevant metrics. questions need to be asked :
How well did the team: - Create required documentation?
- Adhere to best practices? - Use all appropriate tools? - Respond to problems that arose?
How well did the testing team test: - Untrusted interfaces? resources? protocols? files? How well did the testing team cover the:
-Threat model? -Set of possible attacks that apply to the application?
Security Verified
© 2012 iCode information security All rights reserved
Security Assurance
Mohamed Ridha Chebbi, CISSP
Security Verified
© 2012 iCode information security All rights reserved
Security Assurance Maturity
Low
Low
High
High
People & Process
Tool
s &
Tec
hnol
ogy
Typical Progression Curve
Pit of Despair Security as Core Business Process Panic Scramble
Security Verified
© 2012 iCode information security All rights reserved
Stage 1 : Reactionary
Security Verified
© 2012 iCode information security All rights reserved
Stage 2 : Apply Band Aids
Security Verified
© 2012 iCode information security All rights reserved
Stage 3 : Beyond Badness-Ometer
Security Verified
© 2012 iCode information security All rights reserved
Stage 4 : Teach a Man to Fish
Security Verified
© 2012 iCode information security All rights reserved
Stage 5 : Homo Securous
Security Verified
© 2012 iCode information security All rights reserved
Typical Level 5 SAMM Assessments in SDLC
Pen Test
Software Development Life Cycle
Decision Support & Process Optimization
Code Review
Design Build Deploy Operate
Black Box Testing
White Box Testing
Build & Test Automation
Security Verified
© 2012 iCode information security All rights reserved
Security Testing Simplified
Mohamed Ridha Chebbi, CISSP
Security Verified
© 2012 iCode information security All rights reserved
Design Build Test Deploy
Static Testing
PEN Testing
Dynamic Testing
Internal Review
New Versions/Releases
Automated Testing During SDLC
Annually
Operate
…
…
Security Verified
© 2012 iCode information security All rights reserved
Design Build Test Deploy
Static Assessment
PEN Testing
Dynamic Assessment
Internal Review
New Versions/Releases
Application Security Assurance
Annually
Operate
…
…
Security Verified
© 2012 iCode information security All rights reserved
Static Testing Vectors
Security Verified
© 2012 iCode information security All rights reserved
CWE ID CWE Name Directory Traversal
73 External Control of File Name or Path Untrusted Search Path
114 Process Control 426 Untrusted Search Path 427 Uncontrolled Search Path Element
Buffer Overflow 121 Stack-based Buffer Overflow 122 Heap-based Buffer Overflow
Buffer Management Errors 129 Unchecked Array Indexing 135 Incorrect Calculation of Multi-Byte String Length 170 Improper Null Termination 249 Often Misused: Path Manipulation 125 Out-of-bounds Read
Format String 134 Uncontrolled Format String
Numeric Errors 190 Integer Overflow or Wraparound 191 Integer Underflow (Wrap or Wraparound) 192 Integer Coercion Error 195 Signed to Unsigned Conversion Error 196 Unsigned to Signed Conversion Error 197 Numeric Truncation Error
API Abuse 234 Failure to Handle Missing Parameter 243 Failure to Change Working Directory in chroot Jail 193 Off-by-one Error 245 J2EE Bad Practices: Direct Management of Connections 560 Use of umask() with chmod-style Argument
Static Assessment Vectors
Security Verified
© 2012 iCode information security All rights reserved
Information Leakage 201 Information Leak Through Sent Data ✔ 209 Error Message Information Leak ✔ 215 Information Leak Through Debug Information ✔ 497 Information Leak of System Data ✔ 498 Information Leak through Class Cloning ✔
Dangerous Functions 242 Use of Inherently Dangerous Function ✔
Credentials Management 256 Plaintext Storage of a Password ✔ 259 Hard-Coded Password ✔
Authentication Issues 247 Reliance on DNS Lookups in a Security Decision ✔
Cryptographic Issues 311 Failure to Encrypt Sensitive Data ✔ 316 Plaintext Storage in Memory ✔ 321 Use of Hard-coded Cryptographic Key ✔ 326 Weak Encryption ✔ 331 Insufficient Entropy ✔ 347 Improper Verification of Cryptographic Signature ✔ 354 Improper Validation of Integrity Check Value ✔
614 Sensitive Cookie in HTTPS Session Without 'Secure' ✔
310 Cryptographic Issues ✔
CWE ID CWE Name
Static Assessment Vectors
Security Verified
© 2012 iCode information security All rights reserved
Race Conditions 364 Signal Handler Race Condition ✔
367 Time-of-check Time-of-use (TOCTOU) Race Condition ✔ Time and State
377 Insecure Temporary File ✔
378 Creation of Temporary File With Insecure Permissions ✔ 382 J2EE Bad Practices: Use of System.exit() ✔ 557 Concurrency Issues ✔ 708 Incorrect Ownership Assignment ✔
Session Fixation 384 Session Fixation ✔
Error Handling 391 Unchecked Error Condition ✔
Code Quality 111 Direct Use of Unsafe JNI ✔
401 Failure to Release Memory Before Removing Last Reference ('Memory Leak') ✔
404 Improper Resource Shutdown or Release ✔ 415 Double Free ✔ 416 Use After Free ✔
470 Use of Externally-Controlled Input to Select Classes or Code ('Unsafe Reflection') ✔
479 Unsafe Function Call from a Signal Handler ✔ 489 Leftover Debug Code ✔ 597 Use of Wrong Operator in String Comparison ✔
CWE ID CWE Name
Static Assessment Vectors
Security Verified
© 2012 iCode information security All rights reserved
OS Command Injection
78 Failure to Preserve OS Command Structure ('OS Command Injection') ✔
SQL Injection 89 Failure to Preserve SQL Query Structure ('SQL Injection') ✔
564 SQL Injection: Hibernate ✔ Cross-Site Scripting (XSS)
80 Improper Sanitization of Script-Related HTML Tags in a Web Page (Basic XSS) ✔
CRLF Injection 93 Failure to Sanitize CRLF Sequences ('CRLF Injection') ✔
113 Failure to Sanitize CRLF Sequences in HTTP Headers ('HTTP Response Splitting') ✔
117 Improper Output Sanitization for Logs ✔ Encapsulation
494 Download of Code Without Integrity Check ✔ 501 Trust Boundary Violation Malicious ✔
Malicious Code 506 Embedded Malicious Code ✔ 511 Logic/time bomb ✔
656 Reliance on Security through Obscurity Untrusted Initialization ✔
Untrusted Initialization 15 External Control of System or Configuration Setting ✔
Insufficient Input Validation 103 Struts: Incomplete validate() Method Definition ✔ 104 Struts: Form Bean Does Not Extend Validation Class ✔ 112 Missing XML Validation ✔ 601 URL Redirection to Untrusted Site ('Open Redirect') ✔
CWE ID CWE Name
Static Assessment Vectors
Security Verified
© 2012 iCode information security All rights reserved
Automated Scanning •Cross Site Scripting (XSS) •Injection Flaws •Malicious File execution •Insecure Direct Object Reference •Information Leakage and Improper Error Handling •Insecure Cryptographic Storage
Dynamic/Manual Testing Required to identify •Cross Site Request Forgery (CSRF) •Broken Authentication and Session Management •Insecure Communications •Failure to restrict URL access
OWASP Top 10 Compliance
Security Verified
© 2012 iCode information security All rights reserved
Automated Scanning CWE-20: Improper Input Validation CWE-89: Failure to Preserve SQL Query Structure ('SQL Injection') CWE-78: Injection Flaws Failure to Preserve OS Command Structure (‘OS Command Injection’) CWE-209: Error Message Information Leak CWE-426: Untrusted Search Path CWE-494: Download of Code Without Integrity Check CWE-259: Hard-Coded Password CWE-330: Use of Insufficiently Random Values CWE-116: Improper Encoding or Escaping of Output CWE-79: Failure to Preserve Web Page Structure ('Cross-site Scripting') CWE-362: Race Condition CWE-119: Failure to Constrain Operations within the Bounds of a Memory Buffer CWE-404: Improper Resource Shutdown or Release CWE-682: Incorrect Calculation CWE-327: Use of a Broken or Risky Cryptographic Algorithm CWE-73: External Control of File Name or Path
SANS Top 25 Compliance
Security Verified
© 2012 iCode information security All rights reserved
Dynamic/Manual Testing Required to identify CWE-319: Cleartext Transmission of Sensitive Information CWE-665: Improper Initialization CWE-285: Improper Access Control (Authorization) CWE-642: External Control of Critical State Data CWE-352: Cross Site Request Forgery (CSRF) CWE-94: Failure to Control Generation of Code (aka 'Code Injection‘) CWE-732: Insecure Permission Assignment for Critical Resource CWE-602: Client-Side Enforcement of Server-Side Security CWE-250: Execution with Unnecessary Privileges
SANS Top 25 Compliance
Security Verified
© 2012 iCode information security All rights reserved
Dynamic Testing Vectors
Security Verified
© 2012 iCode information security All rights reserved
06-01 Application Path Disclosure 06-02 Browse HTTP from HTTPS 06-03 Brute Force Login 06-04 Auth Over HTTP 06-05 Credit Card Disclosure 06-06 Document Caching 06-07 Find Pages 06-08 Flush Permissions 06-09 Form Caching 06-10 HTTP Parameter Pollution 06-11 Login Redirect 06-12 Non-masked Password 06-13 Plateform Path Disclosure 06-14 Privacy Notification 06-15 Private IP Disclosure 06-16 Remote File Inclusion 06-17 Social Insurance Disclosure 06-18 Social Security Disclosure 06-19 Username & Password in HTTP request
Internal Security
Internal Security Vector
Security Verified
© 2012 iCode information security All rights reserved
07-01 Application Exception 07-02 Application Path Disclosure 07-03 Blind SQL Injection 07-04 Buffer Overflow 07-05 Cross-Site Scripting 07-06 Format String 07-07 Frame Injection 07-08 GET for POST 07-09 HTTP Parameter Pollution 07-10 HTTP Response Splitting 07-11 Integer Overflow 07-12 LDAP Exception 07-13 LDAP Injection 07-14 Open Redirect 07-15 Parameter Addition 07-16 PHP & Perl Code Injection 07-17 Remote File Inclusion 07-18 SQL Disclosure 07-19 SQL Error Message 07-20 SSI Injection 07-21 Unix Command Injection 07-22 Unix Relative Path 07-23 Windows Command Injection 07-24 Windows Relative Path
Entry Validation
Entry Validation Vector
Security Verified
© 2012 iCode information security All rights reserved
09-01 Acquire Session ID 09-02 Authentication Bypass 09-03 Authorization Boundary 09-04 Clickjacking 09-05 Cross Site Request Forgery 09-06 Ineffective Session Termination 09-07 Javascript Hijacking 09-08 Lockout 09-09 Password Change 09-10 Privilege Escalation 09-11 Register Password 09-12 Run Traversal Only 09-13 Session Fixation 09-14 Session Hijacking 09-15 Session ID Identification 09-16 Session ID in URL 09-17 Session ID Length 09-18 Session ID Randomness 09-19 Weak Password
Session Mgmt
Session Management Vector
Security Verified
© 2012 iCode information security All rights reserved
OWASP Top 10
10-A01-01 Blind SQL Injection 10-A01-02 LDAP Exception 10-A01-03 LDAP Injection 10-A01-04 SQL Error Message 10-A01-05 SQL Disclosure 10-A01-06 Unix Command Injection 10-A01-07 Windows Command Injection 10-A02-01 Cross-Site Scripting 10-A03-01 Authentication Bypass 10-A03-02 Cookie Vulnerabilities 10-A03-03 Login Redirect 10-A03-04 Non-masked Password 10-A03-05 Non-SSL Password 10-A03-06 Password Autocomplete 10-A03-07 Session ID Randomness 10-A04-01 Frame Injection 10-A04-02 Remote File Inclusion 10-A06-01 Cross Site Request Forgery 10-A06-01 Application Exception 10-A06-02 Application Path Disclosure 10-A06-03 HTTP Methods 10-A06-04 Directory Browsing 10-A06-05 Plateform Path Disclosure 10-A06-06 Web Server Vulnerabilities 10-A08-01 File & Directory Discovery 10-A08-02 Unix Relative Path 10-A08-03 Windows Relative Path 10-A09-01 Browse HTTP from HTTPS 10-A09-02 Non-SSL Form 10-A09-03 SSL Checks 10-A10-01 Cross-Frame Scripting 10-A10-02 HTTP Response Splitting 10-A10-03 Open Redirect
OWASP TOP 10 Compliance
Security Verified
© 2012 iCode information security All rights reserved
Thanks
Mohamed Ridha Chebbi, CISSP [email protected] Mobile : +216 26 88 10 11