iconference popovsky
DESCRIPTION
TRANSCRIPT
1
Collision of events…
2
Typical Network Incident Response
Technicians must choose: Expend effort collecting forensically sound data, or Simply restore network as quickly as possible
Evidentiary files altered in the process Forensic value limited
Expediency wins…and so do attackers!
3
New Zealand vs. Russian Cases
Characteristics NZ Hacker Case Russian Hacker Case
Type of attack Typical script kiddie
intrusion scenario Online criminal automated
auction scam
Damages $400,000 $25 million
Investigator time 417 hours 9 months
Consequences Community service 3 & 4 years in Federal prison
4
Lack of interest in prosecution
Inordinate effort/cost of investigations
Poor legal outcomes
Investigations not scalable Too expensive Too labor intensive Ties up brilliant technical minds Little comes of it
5
Growing Threat Spectrum
6
The Escalation Tendency of the Hacker Arms Race
7
Fueling the "arms race"
The volume of cyber attacks continues to increase.
It takes less technical knowledge to launch increasingly sophisticated attacks, using increasingly sophisticated hacker tools.
Organizations are becoming increasingly reliant on public networks, often without tempering enthusiasm with a concern for security
Surveys continue to report increased organizational investments in tools and techniques that protect information systems and prevent intrusions in response, yet criminal intrusions are escalating in number and severity.
8
Expect the appetite for prosecution to change
$$$$$$$$$$$$$$$$
9
The Problem
Why this problem must be solved
10
Frye / Daubert Standards
Frye Standards: Is the approach sufficiently
established?
Has the technique gained general acceptance in its field?
Does it require study/experience to gain special knowledge?
Does expertise lie in common experience/knowledge?
Daubert/Kumho Factors: Has the technique used to collect
evidence been tested? (or, can it be tested?)
Has the theory underlying the procedure, or the technique itself been subjected to peer review and publication?
Does the scientific technique have a known or potential rate of error?
Do standards exist, along with maintenance standards, for controlling the technique’s operation?
11
Expert Witness Testimony
The challenge:– Collect/store forensic data– Present forensic data credibly in court
Admissibility standards Frye v. United States. 293 F. 1013 (D.C. Cir. 1923) Daubert v. Merrell Dow Pharmaceuticals, Inc. Daubert, 509 U.S. 579 (1993) (further enunciated in Kumho Tire Co. v. Carmichael) Rule 702 (Federal Rules of Evidence)
12
Foundation
Expert believability based on jury trust
Experts either– Explain evidence so a jury can understand or– It’s so complex, only an expert can understand
Opposing counsel discredits witness by challenging testimony's foundation—
– 'how do you know this?’; – 'how can you say this?'; – 'how can we believe the validity of what you say?‘
Radar gun analogyThe Genuine Tipmra Speeding Ticket Defensehttp://www.tipmra.com/new_tipmra/washington_state_speeding_ticket.htm
13
Computer Forensic Tool Testing Project (CFTT-NIST)
"…to establish a methodology for testing computer forensic tools by the development of functional specifications, test procedures, test criteria, test sets and test hardware.“
Scope: 'software and hardware tools used by law enforcement agencies to acquire data from digital storage media'
Gap: Network devices that collect/gather data
14
Problem
…the courts may begin to expect the same high standards to which they've become accustomed for the preservation of evidence on computer hard drives, when evidence is gathered on complex networks or captured in transmission.
(Sommer, September 2002)
15
Rationale
Experts must speak competently about forensic data reliability– Skills of data gatherer – Process used– Devices employed
Establishing soundness of network data gathering devices can– Support prosecution/defense– Assist pursuit of legal remedies
BUT manufacturers rarely provide conclusive information– Proprietary design– Expense of calibration– As yet no demand
FURTHER manufacturers specifications are not reliable
We expect this to change…..
16
Consequences
A justice system subject to confusion—as innocent individuals are wrongly convicted and those deserving of punishment get away with criminal acts,
Escalating growth in online crime—as prosecution cases fail due to inadmissible evidence and digital crimes go unpunished,
Growing liability for companies—as sensitive customer information and digital assets are vulnerable to increasing online theft and as internal misusers challenge employee disciplinary action supported by
questionable digital evidence,
Decreasing trust in the e-economy—as companies and customers reassess doing business over public networks, and
A general halt to the progress of the Information Age—as online business and communications are no longer viable [FH07].
17
In the meantime…
No standards
No testing labs
Unreliable specifications
Network evidence admitted anyway
First responders still responsible
18
Proposed Solution
Develop device calibration standards
Comparison of instrument performance to a standard of known accuracy in order to determine deviation from nominal and/or make adjustments to minimize error
Start with user verification tests– Use current network testing protocols– Establish calibration approach
19
Calibration
"I often say that when you can measure what you are speaking about and express it in numbers you know something about it; but when you cannot express it in numbers your knowledge is a meager and unsatisfactory kind; it may be the beginning of knowledge but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be."
Lord Kelvin lecture to the Institution of Civil Engineers 3 May 1883[1] [4]
[1] Lord (William Thomson) Kelvin--scientist, engineer and pioneering metrologist--is associated with the development of the Kelvin temperature measurement scale
20
The Problem
"…the courts may begin to expect the same high standards to which they've become accustomed for the preservation of evidence on computer hard drives, when evidence is gathered on complex networks or captured in transmission." [Som02]
– Computer (disk) forensics – more developed science Disks seized by law enforcement Investigators trained in legal procedures Tools, procedures Data accepted in court
– Network forensics – can’t “bag and tag!” Crime scene a live network “Investigators” often untrained network administrators Tools developed for other purposes – troubleshooting, tuning, etc. Data admitted anyway
– Sophistication on both sides of the bar is growing – expect challenges!
21
Consequences
A justice system subject to confusion—as innocent individuals are wrongly convicted and those deserving of punishment get away with criminal acts,
Escalating growth in online crime—as prosecution cases fail due to inadmissible evidence and digital crimes go unpunished,
Growing liability for companies—as sensitive customer information and digital assets are vulnerable to increasing online theft and as internal misusers challenge employee disciplinary action supported by
questionable digital evidence,
Decreasing trust in the e-economy—as companies and customers reassess doing business over public networks, and
A general halt to the progress of the Information Age—as online business and communications are no longer viable [FH07].
22
Rationale for Calibration Focus
Without calibration of network devices used to collect forensic data, the data is:
Subject to serious legal challenge and At risk for inadmissibility in court proceedings [ECF07, Som02].
Calibration not currently performed: Proprietary architecture and forwarding algorithms Troubleshooting, network tuning functionality focus Collecting admissible evidence not primary No standards for device validation
23
Computer Forensic Tool Testing Project (CFTT-NIST) Established
Established in anticipation of legal challenge
Mission to develop testing methods to evaluate computer forensic tools
Scope limited to 'software and hardware tools used by law enforcement agencies to acquire data from digital storage media'
Gap: Enterprise network devices used collect forensic data out of scope
24
Rationale for Developing Network Device Calibration Methodology
Need to establish reliability of network data gathering devices
Need to provide conclusive information that manufacturers don’t provide FURTHER manufacturer specifications are not reliable
Courtroom challenges to network devices used to collect evidence is expected
Yet, no calibration standards/third party labs exist
Network evidence admitted anyway
First responders still responsible
25
Proposed Solution
Develop network device calibration standards
Start with user verification tests– Use current network testing protocols– Establish calibration approach
26
Summary of Progress
27