Presentation is loading. Please wait.

Presentation is loading. Please wait.

Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Data deletion Out damn spot, out! Kurt Seifried,

Similar presentations


Presentation on theme: "Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Data deletion Out damn spot, out! Kurt Seifried,"— Presentation transcript:

1 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Data deletion Out damn spot, out! Kurt Seifried,

2 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ TOC The basic problem The attacker Some examples of failure Wiping hardware Wiping files Wiping information Wiping memory Encrypting information Common failure modes The failure of containment The future

3 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ The basic problem Data is valuable, some data increases in value with age, some decreases (Tobacco company studies for example) Increasingly powerful data recovery tools Deleting data rarely gets rid of it, instead freeing up storage space, formatting does not destroy data either in most cases

4 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ The basic problem (cont.) Existing tools such as EnCase make data recovery very easy As data becomes more abstracted it becomes more difficult to locate where it has been stored More data is being stored on network file systems Copies are perfect, file fragments are perfect partial copies

5 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ The attacker Different types of attackers, with various resource levels and attack methods, this must be taken into account when creating a security policy and protection mechanisms –Unintended recipient –Malicious insider –Outside hacker –Civil litigants –Law enforcement

6 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Some examples of failure Hardware devices not sanitized due to weak magnetic fields Failure to wipe alternate data streams attached to files Data being replicated in unexpected places due to defragmentation, backups, etc

7 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Wiping hardware Wiping hard drive, floppy disks and tapes Wiping cd-roms and other optical media Wiping memory

8 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Wiping hard drive, floppy disks and tapes Raid issues RAID examples Bad block / clusters Destroying hardware Hard and soft 0's and 1's Degaussing issues Verification of wiping

9 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ RAID issues If a drive in a volume set fails most of the data available on it will still be available If a stripped drive without or without parity (RAID 0, 3, 5) fails chances are large pieces of data can be retrieved, depending on cluster size used (up to 64k in some cases) Mirrored drives (RAID 1) have a complete copy of the data

10 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ RAID examples If RAID level 3 or 5 operation is interrupted, e.g. the data blocks have been scrubbed, but parity has not been regenerated it may be possible to regenerate the data from parity and data on the other drives RAID level 1 can be done in software and hardware, scrubbing clusters may not get the correct clusters on both drives

11 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Bad blocks / clusters Blocks or clusters that show damage are eventually marked as bad, this can be done by the hardware itself (i.e. SCSI hard drives) or by software (the OS) Impossible to scrub bad blocks in many cases (the hard drive itself makes them inaccessible), the drive must be physically destroyed

12 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Destroying hardware Destroys resale value (bad pun) Grinding requires reasonably small particles, especially as data density rises Punching a hole disks with a power drill will deter most attackers and is easily verified visually Use of hazardous materials can make proper disposal difficult and expensive

13 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Hard and soft 0's and 1's Data is either 0 or a 1 on the physical medium, expressed as orientation by magnetic particles Hard drive heads wander, data is written on a track, this track can move slightly, thus data on the outside or inside of the track may not be overwritten

14 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Hard and soft 0's and 1's (cont.) Data that is a 0 and then overwritten as a 1 will be a soft 0, some 1's remain Data that is a 0 and then overwritten with a 0 will be a hard 0, very few 1's remain Multiple passes help, however data on the outside or inside of the track may remain intact, this requires physical inspection to retrieve however

15 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Degaussing issues Degaussing requires strong magnetic fields Hard drives use increasingly dense data storage and much stronger and tightly focused magnetic fields, old degaussing equipment may not generate enough field strength to wipe data May not be possible to reformat and verify that data is wiped

16 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Verification of wiping Hard drives have serial numbers, individual platters do not, harder to track Visual verification is possible with grinding, folding and so on, however without serial numbers it could be any drive Verification is never 100%, some unknown technique may restore data

17 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Wiping cd-roms and other optical media Media must usually be destroyed by grinding or shredding Huge volumes of media, easily lost or mixed up with other disks Machines to declassify cd-roms are expensive

18 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Wiping memory Numerous hardware related issues Potential business issues when decommissioning older systems Please see wiping memory section

19 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Wiping files Wiping memory ATA protected storage Verification of wiping Wiping free space Microsoft issues UNIX issues

20 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Wiping memory Files are loaded into memory, consequently they can end up in a number of interesting locations Please see wiping memory section

21 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ ATA protected storage Protected area of hard drive, not accessible to BIOS or OS, used to store recovery data (i.e. OS installation files) MBR must be modified or special boot media used to access the protected areas Not wiped by most software packages including hardware wiping software Tools such as dd will not copy the data reliably

22 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Verification of wiping Checking the media Disk defragmentation Looking for data

23 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Checking the media You must check individual clusters/etc for data, this means using a known pattern (such as all 0's) and then checking for any 1's for example This of course assumes there is only one copy of the data file, data can be copied as a result of being in swap space or swap files.

24 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Disk defragmentation Disk defragmentation results in data being copied and the original space being marked as free. Operating system does not store disk defragmentation data, the wiping software consequently has no idea of where the data has been

25 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Looking for data Verifying data has been wiped requires a search to ensure no file fragments or copies exist, pattern matching partial strings and so on is expensive computationally and may not be possible on large storage arrays, this of course requires a copy of the data (which requires wiping...), use of signatures (i.e. MD5 sums) or watermarks is possible but this will not catch partial data fragments

26 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Looking for data (cont.) Data may have been copied to temporary files on other file systems (local or remote) Files can be very large and contain multiple copies of data (i.e. MS Word with auto save)

27 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Wiping free space As a consequence of not being able to verify data has been copied before it was wiped all unused space must be wiped, this include slack space (partially unused inodes), free space, swap file space, and so on. Modern hard drives are huge, 160 gigabytes and growing, wiping free space cane take hours or even days, may not be possible at all on busy systems

28 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Wiping free space (cont.) Free space cannot be locked, free space may be reserved by another process for a file and thus be inaccessible, but not overwritten yet, you would effectively need to stop the system, boot from different media, wipe all free space and slack space to guarantee destruction

29 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Microsoft issues NTFS and NTFS5 File locking File replication services

30 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ NTFS and NTFS5 NTFS and NTFS5 Overview Slack space Defragmentation Alternate Data Streams Master File Table Encrypted File System Journaling Sparse files Compressed files and directories

31 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ NTFS and NTFS5 Overview NTFS5 needed to support new features such as disk quotas, file encryption, reparse points, directory junctions, volume mount points, sparse files, and the change journal NTFS can be converted to NTFS5, NTFS5 cannot be converted to NTFS NTFS is a journaling file system with database style components

32 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Slack space Most files do not fully use the clusters they are allocated, thus even when a file is overwritten parts of it may survive Difficult to wipe slack space since it has been allocated, not all software wipes slack space properly

33 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Defragmentation Files are copied around the disk, in essence you end up with multiple copies of any defragmented file Often runs as an automated task on servers Must wipe all free space to deal with this issue

34 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Alternate Data Streams Few wiping programs properly wipe alternate data streams (e.g. PGP wipe has not been fixed) Used by default in Explorer to store thumbnails of images, and by Excel 2000 and others to store temporary files Must wipe all free space to deal with this issue

35 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Master File Table Small files (under 1k) stored directly in MFT sometimes MFT cannot be safely modified directly, damage to MFT can destroy the file system (many products make no attempt to touch the MFT) MFT never grows smaller, small files stored in MFT only overwritten by other MFT events

36 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Encrypted File System Encrypts files and directories, existing files and directories marked for encryption leave plain text copies If only files marked as encrypted they may be written in decrypted form to hard drive when you access them Microsoft advises creating an encrypted folder, and then creating files inside of it

37 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Journaling File data is stored in a journal before being committed, this increases the number of locations data is stored Journal areas may be cleaned with wipe free space, however this is problematic

38 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Sparse files Large files containing long strings of zeros can be created, but only the actual data (i.e. not the 0s) is stored, resulting in significant space savings Should not interfere with wipe free space (but untested as of yet) Sparse files cannot be changed to normal files

39 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Compressed files and directories Files stored in compressed format, files are automatically decompressed when opened and compressed when saved Large number of file copies executed (to decompress and compress file), essentially each time you open or save a file

40 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ File locking Locked files cannot be deleted or modified (can be scheduled for after a reboot takes place however) Difficult to remove a lock, easy to create a lock Dlock from 32bits can be used to lock files

41 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ File replication services Data files are automatically replicated when written to When deleted the remote copy is simply deleted, files cannot be wiped on remote systems Files are staged in a temporary directory as well on remote servers

42 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ UNIX issues Wiping free space is not possible on most systems due to lack of utilities, utilities that do exist generally do not wipe slack space, leaving file fragments Extensive use of network file storage via NFS, AFS and others

43 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Wiping information Overview Application issues Protocol issues

44 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Wiping information overview All the problems of wiping files and media come into play More difficult then wiping files as information typically gets copied, moved, merged and shared in many forms Existence of information can be as useful to an attacker as the actual information

45 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Wiping information overview (cont.) Non-existence of information can also be useful to attackers Tracking information is nearly impossible, file moves, copies, defragmentation, s containing data, cutting and pasting data (data is stored in clipboard) and so on

46 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Application issues Databases Printers / Print servers Search engines Exchange server

47 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Database issues Data storage is heavily abstracted, even if an item is deleted wiping free space may not work as the database is still using the file space on the disk Database optimization tools, data integrity and so forth can also cause data to be moved around, resulting in multiple copies on the disk

48 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Printers / Print servers Modern prints servers typically have solid state storage for print spools, wiping is rarely supported (do any?) Many are easily broken into, someone contain full operating systems such as Linux with webservers and so on

49 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Search engines Often contain large part of the data, certainly enough to look for keywords Some cache documents (such as google.com) Removing data can be difficult depending upon implementation

50 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Exchange server Stores messages in a database, impossible to ensure they are wiped Incoming and outgoing messages are stored in temporary areas resulting in multiple copies

51 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Protocol issues Most network file sharing protocols used to transfer data are not encrypted by default, SMB, CIFS, NFS, etc. Network printing protocols do not support encryption, very few end devices (printers) support IPSec/etc. Proxy servers commonly cache data in memory and on disk

52 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Wiping memory Wiping RAM Hibernation / suspend mode Swap space / file

53 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Wiping RAM Memory can be volatile or non-volatile (i.e. Requires a charge to hold data) Volatile memory (conventional computer memory typically) can retain data even without a charge, when the power is cycled (i.e. the system is turned on) the data is actually wiped at this point, as opposed to when the system is turned off

54 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Wiping RAM (cont.) Flash memory can hold data indefinitely (embedded devices, flash cards in routers, digital cameras, etc.) Replacing old memory is difficult at best, voltages and other issues, physical destruction may render the system unsaleable

55 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Hibernation / suspend mode Many modern systems support suspend or hibernation modes The system is put into a minimal power consumption mode Memory (both system RAM and video) is fed a trickle charge or copied to a physical file which is read back into memory when the system is brought back up

56 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Swap space / file Data is moved from memory back onto a disk Swap files can migrate and become fragmented, leaving traces all over the disk Swap partitions when used heavily will leave data at the end, unless heavy usage occurs again data can remain resident for several years (surviving formats and OS reinstallation)

57 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Encrypting information Many file encryption packages encrypt the file but do not wipe the original When file is decrypted into memory it may be written to swap space / file, few applications use memory only flag Key management and storage issues, weak passphrases, easily attacked applications Lack of complete disk encryption programs

58 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Encrypting information (cont.) Legal aspects, data deletion vs. Destruction of evidence, laws like the U.K. RIP bill Requirements for key and data recovery in most organizations (otherwise data dies with the user)

59 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ The failure of containment Few commercial operating systems support data classification (i.e. SECRET, TOP SECRET) Software to encrypt / control distribution expensive, requires deployment onto semi secure systems

60 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Common failure modes Most software fails when dealing with bad blocks Most software does not scrub slack space by default Most software fails when dealing with NTFS ADS or the MFT Most software fails with network storage devices such as NFS/SAMBA/SANS

61 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Common failure modes (cont.) Disk wiping utilities such as East-Tec eraser fail to overwrite all sectors on hard drives (Redemtech report).

62 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ The failure of containment Few commercial operating systems support data classification (i.e. SECRET, TOP SECRET) Software to encrypt / control distribution expensive, requires deployment onto semi secure systems

63 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ The future Extremely large drives wiping free space, slack space will take huge amounts of time, data will survive extended periods Microsoft DFS - Distributed File System - do you know where your data is? Database style file systems such as Microsofts OFS, due out in Longhorn, data is heavily abstracted and difficult to trace down

64 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ The future (cont.) Increased storage of data on network servers through protocols such as SMB, CIFS, HTTP, HTTPS and so on Cross platform interaction with large back end storage such as SANS that do not allow wiping software to be used Reliance on encryption and DRM systems to secure data, wiping may not be supported

65 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ The future (cont.) Network storage arrays, SANS, SWAN, acronym soup iSCSI protocol becoming mainstream IBM storage bricks and other huge data repositories that are disposable Mobile devices with distributed storage, PerosnalRAID

66 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ URLs NTFS resources =8294 UNIX filesystem information

67 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ URLs (cont.) Microsoft file replication service mplechapters/dsdh/dsdh_frs_bnyr.asp Dlock (windows file and folder locking) ATA protected space paper %20Analysis.pdf Redemtech report on disk wiping

68 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ URL's (cont.) slack space wiping in UNIX – ftp://ftp.scyld.com/pub/bmap/ information-from-exposure.html Basics of magnetic recording - IBM Storage Bricks t02/morris.pdf PersonalRAID t02/sobti.html

69 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ MS knowledge base Q221111, Q103657, Q310749, Q231388

70 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Remediation tips Wiping slack space on UNIX find / -type f -exec bcwipe -S {} \; Wiping free space in UNIX – create a large file and then wipe it, this significantly impacts server availability however and is not reliable at all.

71 Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ The End Question and answers if time permits Run for emergency exit if crowd is hostile


Download ppt "Kurt Seifried – kurt.seifried-org – data deletion – seifried.org/security/ Data deletion Out damn spot, out! Kurt Seifried,"

Similar presentations


Ads by Google