srs program red teaming

DARPA SRS PI MTG 13 JULY 2005

SRS Program Red Teaming

Sandia National LabsJohn Clem

Program Manager IDART™505-844-9016

[email protected]://www.sandia.gov/idart

CigitalMark Wilson

Managing Security Consultant703-404-9293

[email protected]://www.cigital.com

Raba TechnologiesJohn Reel

Chief [email protected]

http://www.raba.com


Why red team?many reasons

• Confirms efficacy of security solutions• Identifies design weaknesses• Exposes system to malevolent adversary (emulated)

– Identify high consequence assets– Investigate system security

• Shows where defenses should be increased• Improves understanding of the system• Demonstrates the unimagined• Provides actionable information• It’s cost effective• Build better processes, systems• Reveals adversary/defensive MOP


The Red Team is your friend!

• A part of the greater team responsible for providing system security

• Cooperates with designers to build stronger, more reliable systems

• Help designers identify best options for system hardening

• Red teaming usually results in the identification of previously unknown vulns. or opportunities for improvement


What to expect?

• Red Team is to be provided all designer knowledge / system documents– Design– Implementation– Configuration– Security– Administration– User

• Red Team is to be provided no PWs or keys

• Lifecycle attacks are OOB

• Blue Team will need to freeze system development


What to expect? (2)

• Red Team will work hard (within its constraints) to beat your solution

• Some perspectives– Multiple vectors– For the duration– Control of key services– System control– Less interesting: underlying infrastructure

• We’d prefer to focus on your technology• But if an underlying service is crucial to your

system’s success…harden it


Schedule

• TBD, but…• Initial red team work must begin ASAP• Active red team engagements require preparation

– Plan to send all design documentation in early August

– Blue team must freeze technology/system development by a mutually negotiated date

– Weekly telecons• Red Teams should commence in

September/October


Sandia SRS projects (1)

1. CMU PASIS storage protocol (C.M.U.’s Increasing Intrusion Tolerance via Scalable Redundancy project)

2. Cortex (Honeywell’s Mission-Aware Closed-Loop Cyber Assessment and Response project

3. PMOP (M.I.T.’s Detecting & PreventingMisuse of Privilege project)


Sandia SRS projects (2)process

1. Perform high level, ultra quick look analysis on the three projects

– Review problem/solution proposal• Identify key system functionality• Understand technical strategy

– Review PI’s red team experiment proposal– Confirm state of readiness for red teaming– Reports to respective Pis, SRS Program Manager,

and White Team• Recommended red teaming activity• Suggested areas of red teaming focus


Sandia SRS Projects (3)process

2. One of the three projects will undergo a whiteboard attack red team activity

– Location TBD (at SNL, at developer site)

– Will perform red team attack (on whiteboard) to produce graph of possible attack routes• Done with the developers• Graph will be refined based on adversary model• Will focus on technology in development and

system dependencies

– Quick look report and results of White board activity to PI and SRS Program Manager


Sandia SRS Projects (4)process

3. Two projects will undergo lightweight* red teaming

– Identify vulnerabilities and security-related performance information

– Includes remote or on-site red team testing

– Focus on key system technology and certain dependencies

– Experiment design needs red team input

– High level report of red team activity

*Narrowly-defined/focused


Sandia Contact Info

Sandia National Labs

John Clem

Program Manager IDART™

505-844-9016

[email protected]

http://www.sandia.gov/idart


RABA Corporate Experience

• Founded in 1994 as Boutique Technology Company• Related Past Performance:

– RABA performed a Red Team review of Maryland's Direct Recording Electronic (DRE) voting system

• Only one of its kind in the nation• Recommendations were adopted, in part, by Maryland, Ohio, and California

– Trusted contractor to the Intelligence Community for over eight years designing and performing Red Team exercises for national systems

• Penetration testing in a classified environment both as government employees and private contractors

– Numerous computer forensics investigations supporting clients involved in both criminal and civil cases

• Recreated timelines, pieced together communications sessions, recovered documents, and developed evidentiary support from computer media

• Our Team:– All TS SCI cleared individuals– All have extensive experience in US Gov, DoD, and Intel Community– All have extensive experience in Information Warfare and

Information Operations


Teams RABA Will Evaluate

• MIT – Learning and Repair Techniques for Self-Healing

Systems

• MIT– AWDRAT

• JHU– Scalable Byzantine Replication

• Cornell– QuickSilver


RABA: Basic Methodology

• Anticipate about one week on-site with prep and a one-day test– Anticipate this makes setting up the test env easier for performers?

• MIT (Rinarn-Ernst)– Attack data structures in various ways: input, dynamic corrupt, etc.

• MIT (AWDRAT)– Emphasis on trust mechanism of the infrastructure– Emphasis on making the adaptive behavior asymptotically unstable

• JHU (Scalable Byzantine Replication)– Emphasis to compromise (f+1) nodes– Probably via injection & propagation of false data– Attack the propagation or update mechanisms– Attack Risk Assessment protocol

• Cornell (QuickSilver)– Emphasis to use Quicksilver API’s to create disruptive clients– Attack the gossip protocol– Create communications disruptions

6 - 7Independent Study

8 - 8SAT

9 - 9Travel

10 - 11Study w / Dev .

12 - 12Test

14 - 17Data Reduction

13 - 13TVL

18 - 18Report

Thu Fri Sat Sun Mon Tue Wed Thu Fri Sat Sun Mon Tue


Cigital Approach to Red Team Testing Procedures


University of Virginia Carnegie Mellon University

Genesis

• Evaluation will consist of applying the protection strategy to a software system with known vulnerabilities and verify that the vulnerabilities no longer exist

• Evaluation will also consist of how well Genesis protects against vulnerabilities whose presence is not already known (only to the Red Team) and verify that the vulnerabilities no longer exist


MIT (Williams, Robertson)Pervasive Self-Regeneration through Concurrent Model-Based

Execution

• Evaluate the robustness for the system to be “Fault Aware”, adapting to failure

• Evaluate methods for transitioning to intended states: monitoring progress, diagnosing failure, and repairing or reconfiguring components and effectively enact recovery/repair actions

• Evaluate robustness of service components for irreparably failure• Evaluate the robustness of continuous monitoring of system for

optimal performance metrics• Determine the robustness of the overall system to individual software

components to effectively regenerate large legacy systems


Telecordia, Rutgers Mitigating the Insider Threat using High-dimensional Search and Modeling

• Determine if the system can detect anomalies in high dimensional state spaces• Identify if the system can efficiently detect attacker goals and targets and pro-

actively respond to protect critical services• Evaluate the Sensor Network effectiveness• Determine if the system will annotate an attack on the Network History

Repository• Determine if the system will rank the “Top K” list of annotated states that are

similar to the attack through the High-Dimensional Search Engine• Evaluate the graph-based Insider Threat Modeling and Analysis tool to

determine potential insider attack points and threat scenarios • Evaluate the Response Engine ability to perform impact analysis of the attack

on critical services and minimize collateral damage within system parameters


Schedule

• Preparation– Work with White Team to determine Rules of

Engagement– Background investigation of architectural design and

system protocols: review documentation of system– Analysis, discussion and interaction with Blue Team

on Rules of Engagement to reduce overall burden of tests

– Determine classes of attack that are in scope with system attack vectors


Schedule

• Testing Schedule– Schedule and setup of testing environment at Blue team

site– Evaluation of system, architecture and topology– Seed software with vulnerabilities known only to the Red

Team and supply to the Blue team as appropriate– Determine exploitation of anomalous network traffic, as

appropriate– Identify attack classes that should be detected by the

system and conduct evaluation of systems ability to detect attack

– Evaluate effectiveness of the system’s ability to detect, conduct analysis and response capability


Deliverables

• Evaluation of system and review of findings with Blue team– Evaluate if vulnerabilities were mitigated by the

system transformation

– Draft document developed

– Comments from Blue team incorporated

– Delivery of analysis report to DARPA

srs program red teaming

Documents

red team attack

red team testingf

red teamingreports

butinitial red team

oobblue team

early augustblue team

greater team responsible

system dependenciesquick