1. COMP 1321 Digital Infrastructure
Richard Henson
University of Worcester
October 2012

2. Week 5: Data representation
Learning Objectives:
Explain how storage media store data and how it can be retrieved
Explain how to find data held on a storage medium by using “search string” or location
Explain how data on a storage medium is displayed by software tools, and some practical uses of this ability

3. Memory and Boot-up
Programs to make the computer work have to be stored somewhere
Booting up is a matter of loading all these programs
Makes sense to store everything on ROM
can’t be tampered with
Rapid boot-up

4. Primary & Secondary Storage
Primary storage:
interacts directly with CPU
held on motherboard & controlled by system clock
very fast
Secondary storage:
accessible to CPU via i/o calls
e.g. INT 21 (Intel 8086…)
contained on media attached via ports
can be slow…

5. Virtual Memory
Use of secondary storage device as if it were primary and can be addressed by the CPU
usually hard disk
Requires programmed mapping between extra primary storage locations & secondary storage locations
makes it unlikely that the CPU will run out of “memory”
But performance “falls of a cliff” when virtual memory is invoked

6. Firmware Usually software held on ROM May need updating
low-level operating system
embedded applications
May need updating
some flexibility to overwrite
medium has to allow this
EPROM (erasable, programmable)

7. Questions Is virtual memory primary or secondary storage?
What about firmware?

8. What could Stored Data Represent…
Convention:
1 byte per memory location
Many possibilities!!!
Can be difficult to decide what the data represents…
e.g: data has been recovered from a location; presented as 4E (hex)

9. What could “4E” represent?
Could be part of a program instruction
in assembly language
or source code (as ASCII code)
Program data
as a number
as an ASCII character…

10. More possibilities for “4E”
Over to you…
In groups…
Five minutes…

11. Putting meaning onto raw data… (1)
Single item of data… at a single location… such as 4E (hex) could be anything!
Can only find out context by studying other bytes around it…
if next byte is… 4F (hex)
and byte after that is… 57 (hex)
the ASCII codes together spell “NOW”
so the bytes are probably all ASCII codes

12. Putting meaning onto raw data… (2)
What if the next bytes were 6B and 7D?
ASCII codes would deliver… Nk}
not a proper word
data probably not ASCII codes
What else?
could be integers between 0 and 255
these could be stored variables, or constants…
Wrong to assume… keep an open mind!

13. Use of “Control Bits”
The byte could also be broken up into two nibbles of data
0100… could be an integer of value 4
1110… could be an integer of value 14
It could also be made up of 8 “Boolean” values, which could control outputs to machinery
i.e. 0 = off; 1 = on

14. Looking at Memory… (Peeking)
For Intel 8086 family… debug
Tool available since early days of DOS
Debug needs –d parameter to peek…
shows 128 bytes at a time (& corresp ASCII codes)
default memory location is the start of “free” memory
locations may still contain data from previous usage
Different locations can be added
e.g. –d 0200 would give the next 128 bytes starting from address *0200”

15. “Peeking” and “Poking”…
Debug –d can be used to present a whole range of memory
e.g. –d would display every byte between 0200 (hex) and 0300 (hex)
Debug –e can overwrite a specified location (or sequence of locations) with new data
direct memory overwrite called “poking” by some
potentially can crash the computer… (!)

16. Protection against memory overwrite
Operating system protects memory used by “active” processes
Use of debug e bypasses protection
only protection for computer’s primary memory is to disable the debug program
but could in theory still be executed remotely, if administrative access to local computer has been granted… (!)

17. Finding data on Secondary Storage (1)
Operating systems normally use “files” to send data to secondary storage
Secondary storage devices logically structured into “partitions”
Each Windows partition allocated a letter (e.g. C:, D:, E:, etc.

18. Finding data on Secondary Storage (2)
Essential for each partition to create a table or catalogue for starting address of files that are written to it
otherwise the file becomes very difficult to retrieve…
Method depends on filing system chosen when partition formatted…
each filing system formats (or structures) the media to receive data in its own unique way…

19. Bootable partitions
Some operating system files kept on secondary storage
need to be loaded during final stages of boot up
one partition needs to have an active “boot sector”
provides configuration information for effective communication with CPU
if boot sector damaged, boot up halted!
should have a backup… needs to be copied to boot sector to overwrite corrupted data

20. Fragmentation of Data on Secondary Storage
General problem with hierarchical data storage…
deleted data items leave holes in the structure
New data items saved try to fill the gaps
large files can be broken into fragments
fragments linked by address pointers
slows down retrieval

21. Removing Fragmentation
If disk only partly fragmented…
defragmented files copied into memory
remaining files moved around to close up holes
Previously defragmented files copied back to disk as complete files
If disk >75% fragmented
most effective solution is to copy all files to another partition
can copy back later once original partition has all data deleted

22. Is it true that deleted files aren’t really deleted?
Absolutely!
Two things happen when a file is deleted:
the first data item stored in the file (first character of filename) is changed to “?”
the catalogue entry ceases to recognise & display the filename
starting address shows “?” Character
file system is programmed to ignore ?
Means that the rest of the data is untouched…
easily demonstrated through use of a Hex editor program: can show file contents “before” and “after”

23. “Normal” Loading of a File from Secondary Media
File catalogue essential for data retrieval
application reads file catalogue, usually structured into folders and files
user chooses file, and application loads file into memory
File catalogue has a backup copy…
but what if both become corrupted?

24. Direct access by address on Secondary Media
If both file catalogues are damaged…
file (and its data) cannot be located
“Hex editors” available to do the equivalent of debug –d (peek) and –e (poke)
enables full search of all addresses for particular ASCII string(s)
essential for recovery of data…
and for restoring recently deleted files
“?” character restored to a real character
can then be picked up & shown on catalogue display

25. WinHex Probably the most popular tool to examine hard disks
readout quite similar to debug –e
data presented byte-by-byte according to catalogue address
range of options for extracting, overwriting data, and (like debug) writing consecutive raw data items to a file