App Assessments Reloaded

OWASP Austin Chapter

August 2010

Problems with Security in the SDLC

• Waterfall versus Agile (Technical debt)• Security backlogs, Hardening sprints• Ratcheting is pen-testing for the SDLC

Testing in Prod

• [Almost] Never test in production• Configure temporary DNS/IP for test box• Run only test cases that require on-Internet

AppSec Programs & App Assessments

• Don’t blindly hire external pen-testers• Don’t blindly follow the maturity models• Lead with a tool, but instrumentation• Not app|code scanners, manual pen-tests

Start with Instrumentation

• DBI (Pintool, DynamoRIO, IDA+PaiMei/PyDbg)• Compiler-based (does only gcc support this?)• Actually is perfect for web applications• Fortify PTA, Aspect Security, Morcilla PHP

TAOSSA Code-Audit Strategies

• Instrumentation (CC5) takes care of inputs (filters/validation) and outputs (escaping)• Candidate-points mostly taken care of

• CC1-4: Don’t worry about object-oriented• DGs: Use OOA&D with Patterns, EAI/Web2.0

Which Apps to Test?

• Don’t enumerate or discover web apps• Locate databases and understand data• Find where the data flows to• Threat-model and refactor to security

patterns. Then do posture assessments

How to Test Risky Apps

• Do the manual penetration-testing• Reverse testing• Tiered testing• Make somebody else do it for you

Dev-Test and SQE (Quality)

• Leverage any existing test-harness• Outsource to large usability tests• Company-wide bug hunt days

Leverage the Test Harness

• Webapp: HtmlUnit, Selenium RC, JsTestDriver• Fatapp: Test|Fake client, Corpus distillation• RESTful apps: SoapUI, Unit testing frameworks• Continuous-prevention development

Usability Outsourcing

• E.g. Nielsen Norman Group• Testing Intranets

• If you can’t do this, then do bug-hunts• Invite everybody

Bug-Hunts

<configuration>

<sessionstate timeout=”1” /> (or 1 million)</configuration>

• Red-Gate, Exceptioneer, Lambda Probe, NetLoony, App Logs, Ounce Open (O2)

Epic-Fail Guy (EFG) Revisited

• Required static analysis doesn’t stop EFG• OWASP ESAPI doesn’t stop EFG• Appsec training doesn’t stop EFG• They are legion

Static Analysis Tools Suck

• Too expensive in both money and time• 3k/2wk/app, 30k/yr, 60k/yr• Security coverage costs 25k/yr

• SATE 2009, ManVsAutoVulnAssessment

Fuzzers and Scanners Suck

• Software Security Testing & Quality Assurance• “… the fuzzers found, on average, over 50% more bugs

than just running the most effective fuzzer by itself “• “every 1% of code coverage = finding 1% more bugs”

• Wivet and SQLiBENCH results are still poor

Code Reviews Don’t Scale

• Walkthroughs rarely happen/useful• Specs and Requirements rarely happen/useful• They are awesome though

Pen-Tests Don’t Scale

• All pen-tests should include free, automated regressions that can be run in e.g. cron and provided to the business with free support

• The Appsec SaaS companies do this already

Types of Pen-Testing

• Peripheral (mostly point-and-shoot + reports)• Adversarial (threat-modeling required)• Still doesn’t scale, but pretty cool guy

State of the Art AppSec Risk Management

• Combine methods (SAST+DAST, VA+WAF, etc)• Threadfix, HoneyApps, O2, Aspect Security

• Pen-test specific: The Dradis Framework• Vendor specific: 360, AMP, Hybrid 2.0

The DevTest Security Analyst

• aka Security Bugfixer aka “Security Buddy”• Uses test harness, HP Test Data Management• Reads InfoQ, Hacker News, SpotTheVuln• Stamps out classes of security bugs

Tahnks

Info

@atdreandreg@gmail.com (Active GReader)

http://www.agilegamedevelopment.comhttp://www.fortify.com/products/fortify-360/ (PTA and RTA)http://pintool.orgThe Art of Software Security Assessment (taossa.com)Advanced Object-Oriented Analysis and Design Using UMLhttp://www.eaipatterns.comhttp://oreilly.com/catalog/9780596514433http://www.nngroup.comhttp://www.useit.com/alertbox/outsource_recruiting.htmlhttp://www.securityacts.com/securityacts02.pdf