research in next-generation digital forensics
DESCRIPTION
Research in Next-Generation Digital Forensics. Golden G. Richard III, Ph.D. Associate Professor Dept. of Computer Science [email protected] http://www.cs.uno.edu/~golden. Digital Forensics Research Group. Fall 2006: Thursdays @ 1pm in NSSAL (Math 322) Primary Collaborators: - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/1.jpg)
Research in Next-Generation Digital Forensics
Golden G. Richard III, Ph.D.Associate Professor
Dept. of Computer Science [email protected]
http://www.cs.uno.edu/~golden
![Page 2: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/2.jpg)
Digital Forensics Research Group
• Fall 2006:– Thursdays @ 1pm in NSSAL (Math 322)
• Primary Collaborators:– Vassil Roussev [UNO CS]– Vico Marziale [UNO Ph.D. student]– Frank Adelstein [ATC-NY]
![Page 3: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/3.jpg)
Digital Forensics
Definition: “Tools and techniques to recover, preserve, and examine digital evidence on or transmitted by digital devices.”
Devices include computers, PDAs, cellular phones, videogame consoles, copy machines, printers, …
![Page 4: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/4.jpg)
Examples of Digital Evidence• Threatening emails• Documents (e.g., in places they shouldn’t be)• Suicide notes• Bomb-making diagrams• Malicious Software
– Viruses– Worms– …
• Child pornography (contraband)• Evidence that network connections were made
between machines• Cell phone SMS messages
![Page 5: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/5.jpg)
Facts (or: Why Digital Forensics?)
• Deleted files aren’t securely deleted– Recover deleted file + when it was deleted!
• Renaming files to avoid detection is pointless
• Formatting disks doesn’t delete much data• Web-based email can be (partially)
recovered directly from a computer• Files transferred over a network can be
reassembled and used as evidence
![Page 6: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/6.jpg)
Facts (2)• Uninstalling applications is much more difficult than it
might appear…• “Volatile” data hangs around for a long time (even across
reboots)• Remnants from previously executed applications• Using encryption properly is difficult, because data isn’t
useful unless decrypted• Anti-forensics (privacy-enhancing) software is mostly
broken• “Big” magnets (generally) don’t work• Media mutilation (except in the extreme) doesn’t work• Basic enabler: Data is very hard to kill
![Page 7: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/7.jpg)
Privacy Through Media Mutilation
degausser
or
orforensically-securefile deletion software(but make sure it works!)
or
![Page 8: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/8.jpg)
Digital Forensics Process• Identification of potential digital evidence
– Where might the evidence be?– Which devices did the suspect use?
• Preservation and copying of evidence– On the crime scene…– First, stabilize evidence…prevent loss and contamination– If possible, make identical copies of evidence for
examination• Careful examination of evidence• Presentation
– “The FAT was fubared, but using a hex editor I changed the first byte of directory entry 13 from 0xEF to 0x08 to restore ‘HITLIST.DOC’…”
– “The suspect attempted to hide the Microsoft Word document ‘HITLIST.DOC’ but I was able to recover it without tampering with the file contents.”
• Legal: Balance of need to investigate vs. privacy
![Page 9: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/9.jpg)
“Traditional” Digital Forensics
• Pull the plug• “Image” (make bit-perfect copies) of hard drives,
floppies, USB keys, etc.• Use forensics software to analyze copies of
drives• Investigator typically uses a single computer to
perform investigation in the lab• Present results to client, to officer-in-charge,
court
![Page 10: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/10.jpg)
Traditional: Where’s the evidence?
• Undeleted files, expect some names to be incorrect• Deleted files• Windows registry• Print spool files• Hibernation files• Temp files (all those .TMP files!)• Slack space• Swap files• Browser caches• Alternate or “hidden” partitions• On a variety of removable media (floppies, ZIP,
Jazz, tapes, …)
![Page 11: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/11.jpg)
But Evidence is Also…
• In RAM• “In” the network• On machine-critical machines
– Can’t turn off without severe disruption– Can’t turn them ALL off just to see!
• On huge storage devices– 1TB server: image entire machine and drag it
back to the lab to see if it’s interesting?– 10TB?
![Page 12: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/12.jpg)
Next Generation: Needs• Broad:
– Better design, better software• Yes, some of it is engineering (and hacking)• Someone has to do it
– Better vision, application of ‘real’ CS to problems• More specific:
– Need for speed– Machine correlation– Machine profiling– Better auditing of investigative process– On-the-spot forensics: Triage– Live forensics– Network forensics– Specific tools for detection and remediation of malware– Phishing investigation– …
![Page 13: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/13.jpg)
Next Generation: UNO
• Better file carving• Forensic-aware OS components• In-place file carving • Forensic accountability• On-the-spot forensics• Distributed digital forensics
![Page 14: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/14.jpg)
File Carving: Basic Ideaone cluster
one sector
header, e.g., 0x474946e8e761(GIF)
unrelated disk blocks interesting file
footer, e.g., 0x003B(GIF)
“milestones”or “anti-milestones”
![Page 15: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/15.jpg)
File Carving: Fragmentation
header, e.g., 0x474946e8e761(GIF)
footer, e.g., 0x003B(GIF)
“milestones”or “anti-milestones”
![Page 16: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/16.jpg)
File Carving: Fragmentation
header, e.g., 0x474946e8e761(GIF)
footer, e.g., 0x003B(GIF)
![Page 17: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/17.jpg)
File Carving: Damaged Files
header, e.g., 0x474946e8e761(GIF)
“milestones”or “anti-milestones”
No footer
![Page 18: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/18.jpg)
File Carving: Doing a Better Job• Better design • Faster• Distributed implementation• More flexible description of file types• Automatic generation of type descriptions
– Patterns– Rule sets
• Multiple-pass carving– Carve, “remove” validated files from block list, re-
carve, hope that some fragmented files coalesce• Block-sniffing
![Page 19: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/19.jpg)
File Carving: Block Sniffing
header, e.g., 0x474946e8e761(GIF)
Do these blocks “smell” right?
• N-gram analysis• entropy tests• parsing
![Page 20: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/20.jpg)
Better Software: File Carving: Scalpel
• Two-pass design• Minimizes:
– Reads– Seeks– Writes– Data copying– Memory usage
• Doesn’t yet incorporate all of the carving wizardry we have in mind
G. G. Richard III, V. Roussev, "Scalpel: A Frugal, High Performance File Carver," Proceedings of the 2005 Digital Forensics Research Workshop (DFRWS 2005), New Orleans, LA.
![Page 21: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/21.jpg)
Some Scalpel Results (1)
Big targets, large carve sizes, huge improvement (over 5 hours faster)
Tread + 238,270,750,000 bytes
![Page 22: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/22.jpg)
Some Scalpel Results (2)
Big targets, large carve sizes, huge improvement (over 7 hours faster)
Tread + 117,622,357,936 bytes
![Page 23: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/23.jpg)
OS Support for Digital Forensics
• Export raw disk devices across network for processing– Others: network block device (NBD)– Us: optimization
• “In-place” file carving– Us: Export results from file carving as a
filesystem, w/ minimal extra storage• Better auditing of investigative process
– Us: “digital evidence bag”-aware filesystems
![Page 24: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/24.jpg)
FUSE (Filesystem in User Space)
user space
kernel space
LinuxVirtual File System
Interface(VFS)
C library
dd if=/evidence/DEC/img.dd of=copy.dd
read()
FUSE
ext3
reiserFS
C library
FUSE library
Filesystem Implementation
![Page 25: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/25.jpg)
In-Place File Carving
preview databaseFUSE
scalpel_fs
client applications
nbd server
nbd client
networklocal drive
remote drive
G. G. Richard III, V. Roussev, V. Marziale, “In-Place File Carving,” submitted to the Third Annual IFIP WG 11.9 International Conference on Digital Forensics, 2007.
Scalpel
![Page 26: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/26.jpg)
Better Auditing
Want: Digital Evidence Bags
See: P. Turner, “Unification of Digital Evidence from Disparate Sources (Digital Evidence Bags),” DFRWS 2005See: Common Digital Evidence Storage Format (CDESF) working group, http://www.dfrws.org/CDESF/.
![Page 27: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/27.jpg)
Better Auditing (2)
DEC
(DEB, AFF,Gfzip …)
FDAM
dd scalpel FTK…
VFS Interface
TSK
EvidenceData
Audit Log
Import/Export
Applications(User space)
(Kernel)OperatingSystem
Block-levelData Access
FilesystemData Access
FDAM Block Device
G. G. Richard III, V. Roussev, "Toward Secure, Audited Processing of Digital Evidence: Filesystem Support for Digital Evidence Bags," Research Advances in Digital Forensics, Springer, 2006.
DigitalEvidenceContainer
![Page 28: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/28.jpg)
Bluepipe: On the Spot Digital Forensics
Cu Bootable Bluepipe CD Removable media
Target
Bluetooth or 802.11dongle 3G/VPN
Remote investigator(s)
Handheld Bluepipe client
Y. Gao, G. G. Richard III, V. Roussev, “Bluepipe: An Architecture for On-the-Spot Digital Forensics,” International Journal of Digital Evidence (IJDE), 3(1), 2004.
Bluepipe Patterns
![Page 29: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/29.jpg)
<BLUEPIPE NAME=”findcacti”> <!-- find illegal cacti pics using MD5 hash dictionary --> <DIR TARGET=”/pics/” /> <FINDFILE USEHASHES=TRUE LOCALDIR=”cactus” RECURSIVE=TRUE RETRIEVE=TRUE MSG="Found cactus %s with hash %h "> <FILE ID=3d1e79d11443498df78a1981652be454/> <FILE ID=6f5cd6182125fc4b9445aad18f412128/> <FILE ID=7de79a1ed753ac2980ee2f8e7afa5005/> <FILE ID=ab348734f7347a8a054aa2c774f7aae6/> <FILE ID=b57af575deef030baa709f5bf32ac1ed/> <FILE ID=7074c76fada0b4b419287ee28d705787/> <FILE ID=9de757840cc33d807307e1278f901d3a/> <FILE ID=b12fcf4144dc88cdb2927e91617842b0/> <FILE ID=e7183e5eec7d186f7b5d0ce38e7eaaad/> <FILE ID=808bac4a404911bf2facaa911651e051/> <FILE ID=fffbf594bbae2b3dd6af84e1af4be79c/> <FILE ID=b9776d04e384a10aef6d1c8258fdf054/> </FINDFILE> </BLUEPIPE>
![Page 30: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/30.jpg)
Distributed Digital Forensics
V. Roussev, G. G. Richard III, "Breaking the Performance Wall: The Case for Distributed Digital Forensics,“ Proceedings of the 2004 Digital Forensics Research Workshop (DFRWS 2004), Baltimore, MD
750GB750GB
300GB 300GB
![Page 31: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/31.jpg)
Distributed Digital Forensics• Scalable
– Want to support at least IMAGE SIZE / RAM_PER_NODE nodes• Platform independent
– Want to be able to incorporate any (reasonable) machine that’s available
• Lightweight– Horsepower is for forensics, not the framework—less fat
• Highly interactive• Extensible
– Allow incorporation of existing sequential tools– e.g., stegdetect, image thumbnailing, file classification, hashing,
…• Robust
– Must handle failed nodes smoothly
![Page 32: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/32.jpg)
Distributed Digital Forensics (2)
SHUTDOWN STARTUP
CACHE
FETCH
JOIN / LEAVE
CANCEL / EXITHASH / GREP / COMMAND/ …
DONE / REPORT / ERROR
CoordinatorWorker Worker
Common Store
Local Store
LOAD
STORE
SHUTDOWN STARTUP
CACHE
FETCH
JOIN / LEAVE
CANCEL / EXITHASH / GREP / COMMAND/ …
DONE / REPORT / ERROR
CoordinatorWorker Worker
Common Store
Local Store
LOAD
STORE
![Page 33: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/33.jpg)
Distributed Digital Forensics (3)
SCSIRAID: 504GBFile Server
CPU: 2x1.4GHz XeonRAM: 2GB
Switch96-port, 10/100/1000 Mb24 Gb Backplane
1Gb
NodeCPU: 2.4 GHz Pentium 4RAM: 1 GB
SCSIRAID: 504GBFile Server
CPU: 2x1.4GHz XeonRAM: 2GB
Switch96-port, 10/100/1000 Mb24 Gb Backplane
1Gb
NodeCPU: 2.4 GHz 4RAM: 1 GB
![Page 34: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/34.jpg)
Beowulf [RIP], Slayer of Computer Criminals…
![Page 35: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/35.jpg)
DDF: Results (1)
• Live string search:“Vassil Roussev”
• Regular expression search:
v[a-z]*i[a-z]*a[a-z]*g[a-z]*r[a-z]*a
Search time:String Expression
(mm:ss)
Search time:Regular Expression
(mm:ss)
FTK 08:27 41:508-node System 00:27 00:28
![Page 36: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/36.jpg)
DDF: Results (2)• Stego detection using Stegdetect 0.5 under RH9 Linux
on the cluster• Traditional:
– 6GB image mounted using loopback device– find /mnt/loop –exec ./stegdetect ‘{}’ \;– 790 seconds == 13:10 minutes
• Using the distributed framework– Stegdetect 0.5 code incorporated into framework– Detection against cached files– “STEGO” command (after IMAGE/CACHE)– 82 seconds == 1:22 minutes
• 9.6X faster with 8 machines• CPU bound operation
![Page 37: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/37.jpg)
DDF: To Do List
• User interface! (unless you love Putty)
![Page 38: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/38.jpg)
DDF: To Do (2)• Case persistence• Secure support for overlapping cases• Better fault tolerance• Intelligent caching schemes to support larger
images• Collaboration with colleagues (you?) working in:
– Image analysis/classification– Speech recognition– More stego– Other CPU horsepower-intensive, forensics-
applicable stuff– We provide cycles…you provide…
![Page 39: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/39.jpg)
Current: Live Forensics• Physical memory dumps
– Hard to do when adversarial OS is present– Via USB hacking?– Firewire proof of concept developed by Maximillian
Dornseif • Defeating process hiding techniques, e.g., FU
“rootkit”– Check OS components from many angles
• Remnants of applications (executed) past…– e.g., instant messenger fragments– e.g., recent invocations of process hiding– e.g., fingerprints of recently executed (or executing)
malware
![Page 40: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/40.jpg)
Conclusion: Lots of Work To Do
• Benevolent hacking (engineering) meets science
• Desperately need methods for pipelining investigative process
• Live forensics critically important– volatile computing– whole disk encryption– hardware-based whole disk encryption!– nasty malware
![Page 41: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/41.jpg)
Conclusion (2)• Arguably, almost any field in CS can collaborate
– All media handling needs work– Algorithms for dealing with huge, partially-organized
datasets– Attribution– Correlation– Profiling– Document similarity measures– Databases– High-performance computing– OS Internals
![Page 42: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/42.jpg)
Random Bedside Reading…• http://www.dfrws.org (Digital Forensics Research Workshop)• http://www.ijde.org/ (International Journal of Digital Evidence)• F. Adelstein, “Live Forensics: Diagnosing Your System Without Killing it First,” Communications
of the ACM, February 2006.• M. A. Caloyannides, Privacy Protection and Computer Forensics, Second Edition, 2004.• B. Carrier, File System Forensic Analsis, Addison-Wesley, 2005.• B. Carrier, “Risks of Live Digital Forensics Analysis,” Communications of the ACM, February
2006. • E. Casey, Digital Evidence and Computer Crime, Academic Press, 2004.• J. Chow, B. Pfaff, T. Garfinkel, M. Rosenblum, “Shredding Your Garbage: Reducing Data
Lifetime Through Secure Deallocation,” 14th USENIX Security Symposium, 2005.• M. Geiger, “Evaluating Commercial Counter-Forensic Tools,” 5th Annual Digital Forensic
Research Workshop (DFRWS 2005), New Orleans, 2005. • G. G. Richard III, V. Roussev, "Next Generation Digital Forensics," Communications of the ACM,
February 2006. • G. G. Richard III, V. Roussev, “Digital Forensics Tools: The Next Generation,” invited chapter in
Digital Crime and Forensic Science in Cyberspace, IDEA Group Publishing, 2005.• A. Schuster, “Searching for Processes and Threads in Microsoft Windows Memory Dumps,” 6th
Annual Digital Forensic Research Workshop (DFRWS 2006), West Lafayette, IN, 2006. • S. Sparks, J. Butler, “Raising the Bar for Windows Rootkit Detection,” Phrack Issue # 63. • G. Hoglund, J. Butler, “Rootkits: Subverting the Windows Kernel,” Addison-Wesley, 2005.
![Page 43: Research in Next-Generation Digital Forensics](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815b69550346895dc96075/html5/thumbnails/43.jpg)
Presentation available:http://www.cs.uno.edu/~golden/teach.html
Security Lab (NSSAL): Math 322
?