hiding data, forensics, and anti-forensics

COMMUNICATIONS OF THE ACM April 2007/Vol. 50, No. 4 15

Data hiding hasbeen with us aslong as there have

been digital computersand networks. Somereaders of this columnmight be old enough toremember data hidingon tracks above 80 ofthe ubiquitous 5-1/4-inch double-sided, dou-ble-density floppy disksin the late 1970s. It wasnot uncommon to storea program key on theupper regions of the diskfor copy protection ofPC software. The sim-plicity of this schemewas elegant: the DOSoperating system wouldonly recognize the first80 tracks, so the pro-gram key would be lostduring any DOS copyprocedure. This became one ofthe more common techniques ofdata hiding in the early micro-computer era, although its effec-tiveness was short-lived becauseapplications programs couldaccess the out-of-standard tracksdirectly by bypassing the OSfunction calls and accessing disk

controller directly. This gave riseto a cottage industry of copy-protection-defeating (aka “pirat-ing”) software as bitsmithsquickly developed controller-based copy software that ren-dered this form of out-of-standard copy protection obso-lete. Now of only historical inter-

est, data hiding tech-niques such as this led tomore sophisticatedapproaches that remainwith us today.

Similar strategies existfor data hiding over net-works. Covert channelingis a case in point. Twopopular covert channelingtechniques, protocol bend-ing and packet crafting,share the same out-of-standard approach to con-cealing data as the PC datahiding example given pre-viously.

Protocol bendinginvolves the use of a net-work protocol for someunintended purpose. Typi-cally, this involves embed-ding data in TCP/IPpackets in unexpectedplaces (akin to the higher-

level tracks in the floppy diskexample). A time-worn tactic iscovert channeling over InternetControl Message Protocol(ICMP) packets, for example, byusing the ICMP options field ineach packet to convey applica-tions-layer covert data. SinceICMP was created to transmitPE

PM

ON

TSER

RA

T

Digital Village Hal Berghel

Hiding Data, Forensics, and Anti-ForensicsDelving into the digital warrens for concealing data.

16 April 2007/Vol. 50, No. 4 COMMUNICATIONS OF THE ACM

command and control informa-tion between network appliancessuch as network destinationunreachable, source quenching,echoes (pings) and their replies,there is no expectation that appli-cations-layer data will be includedin ICMP packets.

As a result, most firewalls andintrusion detection/preventionsystems don’t inspect ICMP pack-ets. This is where the protocol is‘bent,’ which results in the estab-lishment of a covert channelbetween network endpoints thatlies under the radar of any net-work administration tool thatassumes that ICMP packets willall conform to IETF specifica-tions. Perhaps the most widelyknown ICMP covert channel toolis Loki, a program named afterthe contriver of mischief in Norsemythology. In the absence ofexhaustive packet analysis, Lokitraffic looks like any other routineICMP request-reply pattern forpings, source quenching, and soforth, while in fact these ICMPpackets are transmitting covertdata. Another popular protocolbender is Reverse WWW Shell,which uses a form of protocolbending called shell shoveling overHTTP.

Covert channeling via protocolbenders deploys protocols in non-standard and perhaps nefarious

ways. Contrasted with protocolbenders are covert channelingtools that use packet crafting toembed data in the actual packetheaders themselves. Covert_TCPand NUSHU are two such exam-ples. Covert_TCP uses activechanneling where it generates itsown packet train to create thechannel. On the other hand,NUSHU is a passive channelerthat piggybacks on packets trans-mitted to the TCP/IP stack byother applications. The coverteffect is the same.

So there you have it: a summaryof data hiding from hiding applica-tion data on either storage mediaor TCP/IP packets in places wherethe standards suggest it doesn’tbelong. Part one of this two-partcolumn deals with physical datahiding on disk file systems.

PHYSICAL DATA HIDING

Physically hidden data is a specialcase of dark data (aka data darkmatter). Information technolo-gists speak of cyberspace, theInternet, and corporate intranetsas mostly “dark,” in that theycontain large amounts of undis-covered, concealed, misplaced,missing, or hidden data. By someaccounts, dark data is an order ofmagnitude larger than light data(data that is known, linked,observed, recovered, retrievable).

Covert data may be thought ofas a small subset of dark data.There are many categories ofcovert data. Encryption producesdark data in the sense that whilethe existence of the data isn’t hid-den, its content is only readableand usable to those who have thecorrect decryption key. Steganog-raphy produces dark data that istypically buried within light data(for example, a non-perceptibledigital watermark buried within adigital photograph). Both areillustrations of intentional con-cealment. They share this charac-teristic with physical data hiding.

Currently, the forensicallyinteresting dimension of physicaldata hiding involves the tech-niques that take advantage of thephysical characteristics of format-ted storage media to hide data.One early attempt to do this wasillustrated by Camouflage (cam-ouflage.unfiction.com) that hiddata in the area between the logi-cal end-of-file and the end of theassociated cluster in which the filewas placed (called file slack orslack space). Though primitive,hiding data in file slack has thedual advantage that the host orcarrier file is unaffected while thehidden data is transparent to thehost OS and file managers. Thedisadvantage is that the hiddenmessage is easily recovered with a

Digital Village

Currently, the forensically interesting dimension of physical data hiding involves the techniques that take advantage of the

physical characteristics of formatted storage media to hide data.

basic disk editor. The ability to hide data on

computer storage media is abyproduct of the system andperipheral architectures. If all stor-age were bit-addressable at the OSlevel, there would be no place tohide data, hence no physical con-cealment. But for efficiency con-siderations, system addressabilitymust be at more abstract levels(typically words in primary, andblocks in secondary). Suchabstractions create digital warrenswhere data may go unnoticed or,in some cases, be inaccessible.

DISK DRIVES AND DIGITAL

WARRENS

Where on a disk can data possi-bly be hidden? A formatted harddrive may be thought of as a logi-cal structure mapped onto a phys-ical medium. The logical structureconsists of partitions, file systems,files, records, fields, and so forth.The physical structure consists ofdisks, cylinders, tracks, clusters,and sectors. The absence of 1:1mappings between the logical andthe physical realms creates thedigital warrens for concealed data.

This has several implications.For example, applications softwareand OSs typically interface withthe logical structure. If data wasconcealed on a disk, the typicaluser would never know it.

More frightening, however, isthat modern computer forensicstools are not designed to uncoverall digital warrens. They typicallyfocus on those disk areas that havealready been observed to holdconcealed data. This presents a

major problem for law enforce-ment, because the more sophisti-cated hacker, criminal, or terroristcould take advantage of the diskwarrens that are not easily foundby current forensics tools. So,from a security and forensicspoint of view it is wise toapproach the problem of data hid-ing from the point of view ofwhat’s possible rather than what’salready known.

For example, computer vendorscommonly create two reservedareas when they format new com-puter hard disks: a Host ProtectedArea (HPA) for their proprietarysoftware and data, and a DeviceConfiguration Overlay (DCO)area for disk metadata. You’veprobably noticed the abundanceof software that bears the name ofthe manufacturer that came withyour computer for management,updating, and diagnostic func-tions. The manufacturer wantsthis software available to the user,and wants to make it difficult forthe user to delete it. Such softwarewould typically fit in the HPA.

Access to of these areas by anOS is prohibited by the disk con-troller. This is the modern-dayanalogue to the track 81–82 copyprotection scheme that wasdescribed in the first paragraph ofthis column. The hack in this casewould be to write a program at alow enough level to access thedisk controller, and then hide thedata in the HPA or DCO—notdifficult at all if one knows thephysical boundaries and boots toa non-host OS. Even within theOS, it’s possible to reassign these

areas to OS control, change thecontents, and then reassign themto the HPA or DCO. This is anexample of a hiding method thattakes advantage of what is moreor less a “physical” feature of thedrive architecture. So, with a littlesophistication one could burycovert data in either the HPA orDCO where it would be con-cealed from even the OS.

Further down in the disk hier-archy, we have the disk partition.Modern OSs allow the adminis-trator to redefine the number andsizes of disk partitions with anynumber of commercial and share-ware utilities. It is fairly commonto place the OS (that is seldommodified) on a partition by itselfand place all applications onanother partition. In this manner,rigorous configuration changes tothe applications software wouldbe unlikely to affect the OS.

Therein lies another opportu-nity to conceal data. Because thelogical partition may not fit per-fectly within the physical subdivi-sions of the disk, partition slackresults. Partition slack is the areabetween the end of a logical parti-tion and the end of the physicalblock the partition falls within. Aswith the HPA/DCO example,this partition slack space is unus-able to applications and the OS.Extended partitions exacerbate theproblem by enabling a multitudeof embedded logical partitions,each one of which contains a digi-tal warren of 62 sectors.

If the partitions in aggregate donot use up all of the available diskspace, volume slack results. One


could easily create a multitude ofpartitions, load one with covertdata and then delete it. Sincedeleting the partition does notdelete the data but only the refer-ence to it by the OS, the dataremains, located beyond the reachof applications and OS.

Down further still, is routinedisk slack. It would be veryunusual to find files that areexactly as long as the sector/clus-

ter sequence in which they arestored. At the sector level, anyunused part of a partially filledsector is padded with either datafrom memory (RAM slack) ornull characters (sector slack). Afterthe padded sector, any remaining,unused sectors are simply ignored(file slack). Once again, the OSand applications have no access tothis space, which follows the endof an active file but is within theallocated sectors and clusters. Fig-ure 1 lists 11 data warrens on filesystems that are typically unob-servable. Variations on this themeare endless.

FORENSIC IMPLICATIONS

Without question, the mostfrightening side effect of thesedigital warrens is the inability ofmodern forensic tools to easilyrecover the data. With worksta-tions now shipping with RAIDfive stacks and terabytes of diskspace, manual investigation ofhard drives at the byte level issimply not viable.

In a sense, we’ve been living ina fool’s paradise because today’scrooks and criminals seldom takeextraordinary measures to concealdata. Most of the forensics workin law enforcement that I’m awareof involves very basic data recov-ery techniques with a few popularforensics tools. However, it wouldbe unwise to expect this to con-tinue, as miscreants and their mis-deeds become more sophisticated.

For simplicity, I will illustratethe principle of covert data hidingon a hard disk with a simpleexample based on the old FAT 16format. The relevant design con-sideration is a 1:1 mappingbetween the entries in the fileallocation table and the physicalclusters on the disk. For example,a FAT entry of hex 0000 indicatesthe corresponding cluster on thedisk is free for use. A hex value of0002-FFEF is a pointer to thenext cluster on the disk that ispart of a file. Hex FFF7 indicatesa bad cluster that has been culledso that it can’t be reallocated.

You guessed it, our simpleexample will involve changingsome entries in the FAT from“free” to “bad,” and then storingdata on the bad clusters. In our


Digital Village

Figure 1. Covert data warrens on disk drives(source: H. Berghel, D. Hoelzer, and M.Sthultz, Data hiding tactics for Windows andUnix file systems; www.berghel.net/publications/data_hiding/data_hiding.php).


Out-of-standard disk copying software haspassed into the digital dustbin. Trade maga-zines of the late 1970s and early 1980s

would reveal widespread use of such software amongcomputer enthusiasts of that era.

Loki (www.phrack.org) has been a premier covertchanneling tool for Unix systems for many years.Although it is widely associated with ICMP, in principleit could use any protocol that is unlikely to be sub-jected to close inspection by network security appli-ances. Unless the packets are analyzed, the Lokitransmission looks innocuous (for example, an ICMPping request or a UDP DNS query). Loki can encrypt alldata for additional stealth, and swap between ICMPand UDP on the fly. For further details on the ICMP andUDP packet formats, see our Packet Pal Primer atwww.berghel.net/resources/packetpal/index.php.

Another approach to covert channeling is the reverseWWW shell (aka, shoveling shell) developed by vanHauser in the late 1990s (www.megasecurity.org/Sources/rwwwshell-1_6_perl.txt). Like Loki, thereverse WWW shell requires a server daemon to be run-ning on the server. The daemon submits outboundHTTP requests for commands from an external com-puter. The intruder’s command is contained within theHTTP response. The command is executed on the com-promised computer, and the results are subsequentlyshoveled to the intruder via a stream of outboundHTTP packets. The HTTP traffic that contains thecovert data appears to the network to be routine Websurfing.

Where Loki and the reverse WWW shell establish thecovert channel over an embedded protocol by means ofprotocol bending, other techniques exist for establish-ing a covert channel by means of packet crafting. CraigRowland’s Covert_TCP (www.securityfocus.com/tools/1475) and Joanna Rutkowska’s NUSHU(http://invisiblethings.org/papers/passive-covert-channels-linux.pdf) are two such examples.

Covert_TCP creates “active channels,” that is, thedaemon actually generates packets with data buried ineither the ID field of the IP packet or the Sequence orAcknowledgment Number fields of the TCP packet (seewww.berghel.net/resources/packetpal/index.php). Bycontrast, NUSHU creates “passive channels” byembedding the data in the SEQ and ACK fields ofexisting packets by adding an offset (data value) tothe existing sequence number. The sequence of offsetvalues is the covert data. The daemon just has toremember to subtract that offset from the returnedsequence number to fool the application. Nushu, inci-dentally, means “woman’s writing.” It is a apparentlya secret language developed by Chinese women. “Tra-ditional Chinese culture is male-centered and forbidsgirls from any kind of formal education, so Nushu wasdeveloped in secrecy over hundreds of years in theJiangyong county of Hunan province”; see www.crys-talinks.com/nushu.html.

Dark data/digital dark matter is usually used insome search or indexing context. See Paul Chin’s 2005summary of dark data within intranets(www.intranetjournal.com/articles/200507/pij_07_07_05a.html) or the recent discussion on the PC Forum blogwhere Yahoo’s Jeff Weiner estimates that 99% of theworld’s collective knowledge is dark data: blogs.zdnet.com/BTL/?p=2715.

Cryptography, steganography, and digital water-marking have been extensively reported in the profes-sional literature, so a Web search will provide millionsof links.

For readers interested in file carving and disk wiping,consult my August 2006 column. For a more thoroughtreatment of the topic, see “Data hiding tactics forWindows and Unix file systems,” by H. Berghel, D.Hoelzer, and M. Sthultz at www.berghel.net/publica-tions/data_hiding/data_hiding.php, and the February2006 section of Communications on next-generationcyber forensics. c

URL Pearls

case, we modified theFAT to show clusters24–29 as bad, andthen stored a GIF fileon those clusters.The OS sees the clus-ters as bad and won’t access them,so the data is covert from the OSpoint of view. But suppose welook at this forensically.

A mainstay of modern forensicstools is a file carver. File carversattempt to reconstruct the diskcontents without using the OS’smeta-level information. Figure 2shows the result of looking theexample disk with a modern filecarver. It can be observed that thefile carver ignored the file’s “magicnumber” identifier that revealed itas a GIF graphics file, and simplyreported the clusters as a lost filefragment, that is, it saw somethingthere, but didn’t look to see whatit was so it, erroneously, assumedthat it must have been dataresidue from a broken file alloca-tion chain. This is analogous tonetwork administration ignoringthe contents of the options field ofan ICMP packet.

CONCLUSION

It isn’t a question of whethercovert data is being hidden onhard drives of unsuspecting

users, but what is being hiddenand for what purposes. Well-funded hackers, criminals, andterrorists are already hiding thedata, while law enforcement triesto catch up with the latest tacticof the day. Challenged byresource limitations, law enforce-ment personnel must rely on thetechnical community to helpprovide solutions and motivatevendors to pay closer attention tosuch potential security breaches.

To make matters worse, anti-forensic tools have been devel-oped that are becoming assophisticated as the forensics toolsthey seek to defeat. To illustrate,the Metasploit project(www.metasploit.com/projects/antiforensics) has developedthree tools that are devastatingfor automated forensic analysistools:

Timestamp provides completeediting capabilities of the NTFStimestamp rendering the time-stamps recovered by forensicstools unreliable in court),

Slacker is an automated tool

for storing files in slack space, andto appear in the near future,

Transmogrify is a tool to defeatfile signature analysis.

The importance of this bur-geoning art of anti-forensics cannot be overstated. Imagine theimpact on law enforcement if fin-gerprint evidence was unreliableand iris scans could be easilyspoofed. In many ways, anti-forensics is scarier than networkhacking. It offers the triple threatof hiding covert data, manipulat-ing system data to exonerate acriminal, and planting systemdata to implicate an innocentparty—without leaving behindtelltale evidence.

Hal Berghel is associate dean of theHoward R. Hughes College of Engineering atthe University of Nevada-Las Vegas, thedirector of the Center for CybersecurityResearch (ccr.i2.nscee.edu), and co-director ofthe Identity Theft and Financial FraudResearch and Operations Center(www.it.ffroc.org).

© 2007 ACM 0001-0782/07/0400 $5.00

c


Digital Village

Figure 2. File carver analysis of covert data on“bad” clusters (source: H.

Berghel, D. Hoelzer, andM. Sthultz, Data hiding

tactics for Windows andUnix file systems;

www.berghel.net/publica-tions/data_hiding/data_

hiding.php).

hiding data, forensics, and anti-forensics

Documents