1

Persistence of MemoryPersistence of MemoryHow Hard Is It To Erase Data?How Hard Is It To Erase Data?

Dr. Victor RalevichSheridan College

2

Did I Delete Sensitive Data?Did I Delete Sensitive Data?

• Last year, MIT graduate students Simson Garfinkel and Abhi Shelat revealed findings of a two-year project in which they collected and analyzed 158 hard drives bought from computer stores, businesses, and eBay.

• The researchers discovered that most computer users did not properly wipe their hard drives before selling them. On the 129 drives they found thousands of credit card numbers, emails, medical information, love letters, and other information.

3

DeleteDelete Command (1) Command (1)

• All operating systems have some form of delete, erase, or remove command.

• Most of these commands never even touch the actual data that is recorded on the disk drive.

• They typically remove the index entry and pointers to the data file so that it appears the file is no longer there, and the space allocated to that file is made available for future write commands.

4

DeleteDelete Command (2) Command (2)

• Commonly available utilities allow any knowledgeable technician to move beyond the operating system's file indexing scheme and examine or rebuild previously deleted information.

• Some advanced DELETE programs are available that go out of their way to actually overwrite the sectors used by a file to store data. These are an improvement, but still pose a security threat.

5

DeleteDelete Command (3) Command (3)

• There are usually bits and pieces of data not associated or indexed with the actual file that can be missed.

• For example, most application programs (and many operating systems) will open temporary or swap/cache files while working on the data from a file.

• When the program is closed or exited, the application "deletes" these temp files. So even if the original file has been overwritten, multiple copies of the raw data may still exist in various unused parts of the disk drive.

6

Disk FormattingDisk Formatting

• The word format has come to describe several different processes in the set-up and initialization of a hard disk drive. There are physical or low level formats, operating system formats, quick formats, partitioning formats, etc...

• Depending on the technology of the disk drive and the format utility that is used, each of these may perform a different function. In many cases, previously written data is unaffected.

• The format creates a new blank indexing scheme for the operating system, making all the sectors available for the writing of new files, making it appear that there are no files on the drive.

7

Data Deletion by OverwritingData Deletion by Overwriting

• Overwriting of the data means replacing previously stored data on a drive or disk with a predetermined pattern of meaningless information.

• This is an accepted and effective means of rendering data unrecoverable but the process must be correctly understood and carefully implemented.

8

Data ClearingData Clearing

• Clearing is the removal of sensitive data from storage devices in such a way that there is assurance that the data may not be reconstructed using normal system capabilities, i.e., through the keyboard.

• Data reconstruction may include use of data recovery utilities and advanced diagnostic routines.

9

Disk Cleaning Software (1)Disk Cleaning Software (1)Clean Disk SecurityClean Disk Security

Clean Disk Security• Completely eliminates the

contents of deleted files. • Gutmann disk cleaning method

is now available as an option. • Can clean the Window's swap

file, and unneeded temporary files from the hard disk, such as your Internet browser cache, files in system's Recycle Bin, and "recent files" list.

• Comes with a direct disk viewer for discovering exactly what is on your hard disk.

10

Disk Cleaning Software (2)Disk Cleaning Software (2)WhiteCanyon SecureCleanWhiteCanyon SecureClean

11

Data Recovery and Forensics Data Recovery and Forensics ToolsTools

• Guidance Software – EnCase ForensicEnCase Forensic

• AccessData- – Forensic ToolkitForensic Toolkit– Password Recovery ToolkitPassword Recovery Toolkit– Registry ViewerRegistry Viewer– Distributed Network AttackDistributed Network Attack

12

Data Purging (Sanitization)Data Purging (Sanitization)

• Purging is the removal of sensitive data from a system or storage device in such a way that there is assurance that the data may not be reconstructed through open-ended laboratory techniques.

• The United States Department of Defense (DoD) has approved both overwriting and degaussing for purging data, although the effectiveness of overwriting cannot be guaranteed without examining each specific situation.

13

DegaussersDegaussers

• Mag EraSURE ME-P3E NSA Listed Degausser

14

DestructionDestruction

• It is good practice to purge media before submitting it for destruction. Media may generally be destroyed by one of the following methods:– Destruction at an approved metal destruction facility,

i.e., smelting, disintegration, or pulverization: – Incineration. – Application of corrosive chemicals, such as acids, to

recording surfaces. – Application of an abrasive substance (emery wheel or

disk sander) to a magnetic disk or drum recording surface. Make certain that the entire recording surface is completely removed before disposal.

15

Can Overwritten Data be Can Overwritten Data be Recovered? (1)Recovered? (1)

• It is commonly quoted that data can be recovered if it has been only overwritten once or twice and that it actually takes up to ten overwrites to securely protect previous data.

• If a head positioning system is not exact enough, new data written to a drive may not be written back to the precise location of the original data.

• Due to this track misalignment, it is possible to identify traces of data from earlier magnetic patterns alongside the current track. (At least that was the case with high capacity floppy diskette drives, which have a rudimentary position mechanism.)

16

Can Overwritten Data be Can Overwritten Data be Recovered? (2)Recovered? (2)

• When 1 is written to disk the actual effect is closer to obtaining a 0.95 when a zero is overwritten with 1, and a 1.05 when 1 is overwritten with 1.

• Normal disk circuitry is set up so that both these values are read as 1, but using specialised circuitry it is possible to work out what previous "layers" contained.

• It turns out that each track contains an image of everything ever written to it, but that the contribution from each "layer" gets progressively smaller the further back it was made.

• Intelligence organizations have a lot of expertise in recovering these palimpsestuous images.

17

Scanning Probe Microscopy (SPM)Scanning Probe Microscopy (SPM)

• Scanning Tunneling Microscopy

• Atomic Force Microscopy – Contact AFM – Non-contact AFM – Intermittent-contact AFM

• Magnetic Force Microscopy

• Lateral Force Microscopy

18

Other SPM TechniquesOther SPM Techniques

• Force Modulation Microscopy

• Phase Detection Microscopy

• Electrostatic Force Microscopy

• Scanning Capacitance Microscopy

• Thermal Scanning Microscopy

• Near-field Scanning Optical Microscopy

• Nanolithography

19

20

Atomic Force Microscopy (1)Atomic Force Microscopy (1)• The atomic force microscope

(AFM), or scanning force microscope (SFM) was invented in 1986 by Binnig, Quate and Gerber. The AFM utilises a sharp probe moving over the surface of a sample in a raster scan.

• In the case of the AFM, the probe is a tip on the end of a cantilever which bends in response to the force between the tip and the sample.

• As the cantilever flexes, the light from the laser is reflected onto the split photo-diode. By measuring the difference signal (A – B), changes in the bending of the cantilever can be measured.

21

Atomic Force Microscopy (2)Atomic Force Microscopy (2)

• Since the cantilever obeys Hooke's Law for small displacements, it is possible to estimate the interaction force between the tip and the sample.

• The movement of the tip or sample is performed by an extremely precise positioning device made from piezo-electric ceramics, most often in the form of a tube scanner. The scanner is capable of sub-angström resolution in x-, y- and z-directions. The z-axis is conventionally perpendicular to the sample.

• The AFM can be operated in two principal modes – with feedback control – without feedback control

22

Atomic Force Microscopy (3)Atomic Force Microscopy (3)

• The electronic feedback mode of operation is known as constant force, and usually enables a fairly faithful topographical image to be obtained (hence the alternative name, height mode).

• If the feedback electronics are switched off, then the microscope is said to be operating in constant height or deflection mode. This is particularly useful for imaging very flat samples at high resolution.

23

Atomic Force Microscopy (4) Atomic Force Microscopy (4) Tip-sample interactionTip-sample interaction

• The image contrast can be achieved in many ways.

• The three main classes of interaction are – contact mode, – tapping mode, and – non-contact mode. 

24

Sample of Atomic Force Sample of Atomic Force Microscopy ImageMicroscopy Image

• Height (contact) image of a 100 µm piece of floppy disc (T.J. McMaster et al.)

25

Magnetic Force Microscopy (1)Magnetic Force Microscopy (1)

• Magnetic force microscopy (MFM) images the spatial variation of magnetic forces on a sample surface.

• For MFM, the tip is coated with a ferromagnetic thin film. The system operates in non-contact mode, detecting changes in the resonant frequency of the cantilever induced by the magnetic field's dependence on tip-to-sample separation.

• MFM can be used to image naturally occurring and deliberately written domain structures in magnetic materials.

26

Magnetic Force Microscopy (2)Magnetic Force Microscopy (2)

27

Magnetic Force Microscopy (3)Magnetic Force Microscopy (3)

• MFM images of overwritten tracks on a textured hard disk. • The topography (left) was imaged using Tapping Mode; the magnetic force

image of the same area (right) was captured with Lift Mode (lift height 35 nm) by mapping shifts in cantilever resonant frequency.

• Acquisition time was about five minutes. Track width and skew, transition irregularities, and the difference between erased and virgin areas are visible. 25 µm scan.

28

Magnetic Force Microscopy (4)Magnetic Force Microscopy (4)

• The bright and dark lines indicate transition between the longitudinal bits

Field of view 100 µm x 100 µm

Magnetic force microscopy image of magnetic domains in the servo tracks of a hard disk.

29

Magnetic Force Microscopy (5)Magnetic Force Microscopy (5)

• The Magnetic Force Microscope senses the magnetic field just above the disk surface.  20 micron scan.

• Magnetic force images of a 100 µm piece of floppy disc (T.J. McMaster et al.)

30

Magnetic Media – Data Erasure (1)Magnetic Media – Data Erasure (1)

• Concept behind an overwriting scheme is to flip each magnetic domain on the disk back and forth as much as possible without writing the same pattern twice in a row.

• If the data was encoded directly, we could simply choose the desired overwrite pattern of ones and zeroes and write it repeatedly.

• However, disks generally use some form of run-length limited (RLL) encoding, so that the adjacent 1’s won't be written.

31

Magnetic Media – Data Erasure (2)Magnetic Media – Data Erasure (2)

• To erase magnetic media, we need to overwrite it many times with alternating patterns in order to expose it to a fast oscillating magnetic field.

• We need to saturate the disk surface to the greatest depth possible, but very high frequency signals only "scratch the surface" of the magnetic medium.

• Disk drive manufacturers, in trying to achieve ever-higher densities, use the highest possible frequencies.

• The best we can do is to use the lowest frequency possible for overwrites, to penetrate as deeply as possible into the recording medium.

32

Magnetic Media – Data Erasure (3)Magnetic Media – Data Erasure (3)

• Disk data encoding schemes:– FM (Frequency Modulation) – oldest– MFM (Modified FM)– RLL (Run Length Limited)– PRML (Partial Response, Maximum

Likelihood)– EPRMS (Extended PRML)

33

Magnetic Media – Data Erasure (4)Magnetic Media – Data Erasure (4)

• FM, MFM and 2,7 RLL encoding write waveform for the byte "10001111".

• RLL improves further on MFM by reducing the amount of space required for the same data bits to one third that required for regular FM encoding.

34

Magnetic Media – Data Erasure (5)Magnetic Media – Data Erasure (5)

• “We now have a set of 22 overwrite patterns which should erase everything, regardless of the raw encoding. The basic disk eraser can be improved slightly by adding random passes before and after the erase process, and by performing the deterministic passes in random order to make it more difficult to guess which of the known data passes were made at which point.”

“Secure Deletion of Data from Magnetic and Solid-State Memory” Peter Gutmann, Department of Computer Science, University of Auckland, 1996

35

Gutmann’s AlgorithmGutmann’s Algorithm

• Peter Gutmann suggested that we use the sequence of 35 consecutive writes with predefined patterns.

• The MFM-specific patterns are repeated twice because MFM drives have the lowest density and are thus particularly easy to examine.

• The deterministic patterns between the random writes are permuted before the write is performed, to make it more difficult for an opponent to use knowledge of the erasure data written to attempt to recover overwritten data.

36

Hard Disc OrganizationHard Disc Organization• Track

A concentric set of magnetic bits on the disk is called a track. Each track is divided into 512 bytes (usually) sectors.

• SectorA part of each track defined with magnetic marking and an ID number. Sectors have a sector header and an error correction code (ECC).

• CylinderA group of tracks with the same radius is called a cylinder (red tracks on the picture belong to one cylinder).

• Data addressingThere are two methods for data addressing: CHS (cylinder-head-sector) and LBA (logical block address).

37

Other Problems with Magnetic Media (1)Other Problems with Magnetic Media (1)Defective Sector HandlingDefective Sector Handling

• There are several techniques which are used to mask the defects in the defect list.

– Alternate tracks, moves data from tracks with defects to known good tracks.

– Alternate sectors, allocates alternate sectors at the end of the track to minimise seeks caused by defective sectors.

– Inline sector sparing, allocates a spare sector at the end of each track, but resequences the sector ID's to skip the defective sector and include the spare sector at the end of the track.

38

Other Problems with Magnetic Media (2)Other Problems with Magnetic Media (2)AgeingAgeing

• Long-term ageing can also have an effect on the erasability of magnetic media.

• Some types of magnetic tape become increasingly difficult to erase after being stored at an elevated temperature.

• The erasability of the data depends on the amount of time it has been stored on the media, not on the age of the media itself.

39

Other Problems with Magnetic Media (3)Other Problems with Magnetic Media (3)TemperatureTemperature

• The dependence of media coercivity on temperature can affect overwrite capability.

• This is important in hard disk drives, where the temperature varies depending on how long the unit has been used and, in the case of drives with power-saving features enabled, how recently and frequently it has been used.

• The overwrite performance depends also on temperature-dependent changes in the read/write head.

40

Other Problems with Magnetic Media (4)Other Problems with Magnetic Media (4)Error-correction SchemesError-correction Schemes

• Newer storage devices are, through the use of various error-correction schemes, able to recover from having a remarkable amount of damage inflicted on them.

• Error-correction codes (ECC's) are capable of correcting multiple error bursts.

41

Recovering Data stored in ROMRecovering Data stored in ROM

• “Volatile" semiconductor memory does not entirely lose its contents when power is removed.

• Both static (SRAM) and dynamic (DRAM) memory retains some information on the data stored in it while power was still applied.

• Older SRAM chips could often "remember" the previously held state for several days.

42

Erasing Data stored in ROMErasing Data stored in ROM

• Heat – Both DRAM and SRAM will lose their content much faster on 1400C than on room temperature.

• Constantly flip the bits in memory – ensure that a memory cell never holds a charge long enough for it to be "remembered".

– It is possible to do this for small amounts of very sensitive data such as encryption keys.

43

Conclusion (1)Conclusion (1)

• Data overwritten once or twice may be recovered by subtracting what is expected to be read from a storage location from what is actually read.

• Data which is overwritten an arbitrarily large number of times can still be recovered provided that the new data isn't written to the same location as the original data (for magnetic media), or that the recovery attempt is carried out fairly soon after the new data was written (for RAM).

• For this reason it is effectively impossible to sanitise storage locations by simple overwriting them, no matter how many overwrite passes are made or what data patterns are written.

44

Conclusion (2)Conclusion (2)

• Data recovery can be made significantly more difficult, if not prohibitively expensive.

• The best way to make sure that you got rid of data is to destroy the disk.

• Encrypt data whenever possible. • For sensitive information prevent paging of

memory to the hard drive.

45

LinksLinks

• Peter Gutmann “Secure Deletion of Data from Magnetic and Solid-State Memory”

www.cs.auckland.ac.nz/~pgut001/pubs/secure_del.html• Clean Disk Security -

www.theabsolute.net/sware/clndisk.html• WipeDrive, Secure Clean www.whitecanyon.com/• Data Forensics Software (EnCase)

www.guidancesoftware.com/• AccessData Forensic Toolkit www.accessdata.com/• A Practical Guide to Scanning Probe Microscopy

mechmat.caltech.edu/~kaushik/park/contents.htm