COMP 1321 Digital Infrastructure Richard Henson University of Worcester October 2012

Week 5: Data representation Learning Objectives: Explain how storage media store data and how it can be retrieved Explain how to find data held on a storage medium by using “search string” or location Explain how data on a storage medium is displayed by software tools, and some practical uses of this ability

Memory and Boot-up Programs to make the computer work have to be stored somewhere Booting up is a matter of loading all these programs Makes sense to store everything on ROM can’t be tampered with Rapid boot-up

Primary & Secondary Storage Primary storage: interacts directly with CPU held on motherboard & controlled by system clock very fast Secondary storage: accessible to CPU via i/o calls e.g. INT 21 (Intel 8086…) contained on media attached via ports can be slow…

Virtual Memory Use of secondary storage device as if it were primary and can be addressed by the CPU usually hard disk Requires programmed mapping between extra primary storage locations & secondary storage locations makes it unlikely that the CPU will run out of “memory” But performance “falls of a cliff” when virtual memory is invoked

Firmware Usually software held on ROM May need updating low-level operating system embedded applications May need updating some flexibility to overwrite medium has to allow this EPROM (erasable, programmable)

Questions Is virtual memory primary or secondary storage? What about firmware?

What could Stored Data Represent… Convention: 1 byte per memory location Many possibilities!!! Can be difficult to decide what the data represents… e.g: data has been recovered from a location; presented as 4E (hex)

What could “4E” represent? 0 1 0 0 1 1 1 0 Could be part of a program instruction in assembly language or source code (as ASCII code) Program data as a number as an ASCII character…

More possibilities for “4E” Over to you… In groups… Five minutes…

Putting meaning onto raw data… (1) Single item of data… at a single location… such as 4E (hex) could be anything! Can only find out context by studying other bytes around it… if next byte is… 4F (hex) and byte after that is… 57 (hex) the ASCII codes together spell “NOW” so the bytes are probably all ASCII codes

Putting meaning onto raw data… (2) What if the next bytes were 6B and 7D? ASCII codes would deliver… Nk} not a proper word data probably not ASCII codes What else? could be integers between 0 and 255 116 153 1275 these could be stored variables, or constants… Wrong to assume… keep an open mind!

Use of “Control Bits” The byte could also be broken up into two nibbles of data 0100… could be an integer of value 4 1110… could be an integer of value 14 It could also be made up of 8 “Boolean” values, which could control outputs to machinery i.e. 0 = off; 1 = on

Looking at Memory… (Peeking) For Intel 8086 family… debug Tool available since early days of DOS Debug needs –d parameter to peek… shows 128 bytes at a time (& corresp ASCII codes) default memory location is the start of “free” memory locations may still contain data from previous usage Different locations can be added e.g. –d 0200 would give the next 128 bytes starting from address *0200”

“Peeking” and “Poking”… Debug –d can be used to present a whole range of memory e.g. –d 0200 0300 would display every byte between 0200 (hex) and 0300 (hex) Debug –e can overwrite a specified location (or sequence of locations) with new data direct memory overwrite called “poking” by some potentially can crash the computer… (!)

Protection against memory overwrite Operating system protects memory used by “active” processes Use of debug e bypasses protection only protection for computer’s primary memory is to disable the debug program but could in theory still be executed remotely, if administrative access to local computer has been granted… (!)

Finding data on Secondary Storage (1) Operating systems normally use “files” to send data to secondary storage Secondary storage devices logically structured into “partitions” Each Windows partition allocated a letter (e.g. C:, D:, E:, etc.

Finding data on Secondary Storage (2) Essential for each partition to create a table or catalogue for starting address of files that are written to it otherwise the file becomes very difficult to retrieve… Method depends on filing system chosen when partition formatted… each filing system formats (or structures) the media to receive data in its own unique way…

Bootable partitions Some operating system files kept on secondary storage need to be loaded during final stages of boot up one partition needs to have an active “boot sector” provides configuration information for effective communication with CPU if boot sector damaged, boot up halted! should have a backup… needs to be copied to boot sector to overwrite corrupted data

Fragmentation of Data on Secondary Storage General problem with hierarchical data storage… deleted data items leave holes in the structure New data items saved try to fill the gaps large files can be broken into fragments fragments linked by address pointers slows down retrieval

Removing Fragmentation If disk only partly fragmented… defragmented files copied into memory remaining files moved around to close up holes Previously defragmented files copied back to disk as complete files If disk >75% fragmented most effective solution is to copy all files to another partition can copy back later once original partition has all data deleted

Is it true that deleted files aren’t really deleted? Absolutely! Two things happen when a file is deleted: the first data item stored in the file (first character of filename) is changed to “?” the catalogue entry ceases to recognise & display the filename starting address shows “?” Character file system is programmed to ignore ? Means that the rest of the data is untouched… easily demonstrated through use of a Hex editor program: can show file contents “before” and “after”

“Normal” Loading of a File from Secondary Media File catalogue essential for data retrieval application reads file catalogue, usually structured into folders and files user chooses file, and application loads file into memory File catalogue has a backup copy… but what if both become corrupted?

Direct access by address on Secondary Media If both file catalogues are damaged… file (and its data) cannot be located “Hex editors” available to do the equivalent of debug –d (peek) and –e (poke) enables full search of all addresses for particular ASCII string(s) essential for recovery of data… and for restoring recently deleted files “?” character restored to a real character can then be picked up & shown on catalogue display

WinHex Probably the most popular tool to examine hard disks readout quite similar to debug –e data presented byte-by-byte according to catalogue address range of options for extracting, overwriting data, and (like debug) writing consecutive raw data items to a file