1. CIT 383: Administrative Scripting
Data Persistence
CIT 383: Administrative Scripting

2. CIT 383: Administrative Scripting
Topics
Data Models
Delimited Fields
Fixed Length Records
Serialization
Concurrency
YAML
CIT 383: Administrative Scripting

3. CIT 383: Administrative Scripting
Data Models
Flat Model
Single two-dimensional array of data elements.
Example: CSV
Hierarchical Model
Data organized into a tree-like structure.
Example: XML
Relational Model
Multiple tables with precisely defined relations.
Example: MySQL
CIT 383: Administrative Scripting

4. CIT 383: Administrative Scripting
Storage Types
Flat Text Files
Delimited fields
Fixed length records
Flat Binary Files
Marshal
DBM
Hierarchical Structured Text
XML
YAML
CIT 383: Administrative Scripting

5. CIT 383: Administrative Scripting
Delimited Fields
Text fields separated by a delimiter.
CSV
/etc/passwd
Advantages
Simple: use standard file I/O and split.
Humanly readable storage.
Disadvantages
Must have technique to escape delimiter.
Slow to access a specific record.
Hard to update or delete data (must rewrite file.)
CIT 383: Administrative Scripting

6. CIT 383: Administrative Scripting
Fixed Length Records
Use record sizes of a fixed number of bytes.
aaaabbccccccccdddd
Advantages
Do not need to use any escaping.
Fast direct access to records (n * record_size).
Easy to update/delete: just write over record.
Disadvantages
Wastes space: record size >= size of largest data.
Need special character to fill fields to record size.
CIT 383: Administrative Scripting

7. CIT 383: Administrative Scripting
Serialization
Converting an object into a sequence of bits.
A/K/A: marshaling, deflating.
Necessary to save data beyond simple strings.
Converting an array to a sequence:
favs = %w{Haskell OCaml Python Ruby}
Marshal.dump(favs)
"\004\010[\n\"\fHaskell\"\nOCaml\"\
vPython\"\tRuby"
CIT 383: Administrative Scripting

8. CIT 383: Administrative Scripting
Concurrency
What if two programs access data at once?
Two reads: no problem.
Two writes: race condition.
Race Condition Example
Two payments: A for $50, B for $100, funds = $1000 A
1. data = read(file)
data[‘funds’] = 1000
3. data[‘funds’] += 50
data[‘funds’] = 1050
5. write(file, data)
2. data = read(file)
data[‘funds’] = 1000
4. data[‘funds’] += 100
data[‘funds’] = 1100
6. write(file, data)
CIT 383: Administrative Scripting

9. CIT 383: Administrative Scripting
Locking Files
Prevent concurrent access by locking files.
A: lock(file)
B: lock(file) fails since A has lock
A: does stuff with file
A: unlock(file)
B: lock(file) succeeds
B: does stuff with file
Ruby code for file locking:
file = File.new(‘data’)
file.flock(File::LOCK_EX)
... do stuff with file ...
file.flock(File::LOCK_UN)
CIT 383: Administrative Scripting

10. CIT 383: Administrative Scripting
YAML
YAML Ain’t a Markup Language
Originally Yet Another Markup Language.
Changed since primarily used to store data.
Simple language for storing data.
Humanly readable and writable.
Less verbose than XML.
Superset of JSON (JavaScript Object Notation.)
CIT 383: Administrative Scripting

11. CIT 383: Administrative Scripting
YAML Syntax
Lists
--- # List of languages
- Ruby
- Python
- Perl
--- # Short form of list
[Ruby, Python, Perl]
CIT 383: Administrative Scripting

12. CIT 383: Administrative Scripting
YAML Syntax
Hashes
--- # User record
username: smithj
uid: 11387
gcos: John Smith
--- # Short form of hash
{username: smithj, uid: 11387, gcos: John Smith}
CIT 383: Administrative Scripting

13. CIT 383: Administrative Scripting
YAML Syntax
Strings
Regular strings need no punctuation.
Long strings use --- beginning.
--- |
This section preserves newlines
just as they appear.
--- >
This section ignores newlines, and their only
purpose is to make text readable.
CIT 383: Administrative Scripting

14. CIT 383: Administrative Scripting
YAML vs XML
XML
<user id="smith">
<firstname>Al
</firstname>
<lastname>Smith
</lastname>
<phone>
</phone>
</ >
</user>
YAML
smith:
firstname: Al
lastname: Smith
phone: s: - -
CIT 383: Administrative Scripting

15. CIT 383: Administrative Scripting
YAML in Ruby
Serializing objects to YAML
require ‘yaml’
favs = %w{Haskell OCaml Python Ruby}
puts favs.to_yaml
Retrieving objects from YAML
file = File.new(‘data.yaml’)
favs = YAML.load(file)
CIT 383: Administrative Scripting

16. CIT 383: Administrative Scripting
References
Michael Fitzgerald, Learning Ruby, O’Reilly,
David Flanagan and Yukihiro Matsumoto, The Ruby Programming Language, O’Reilly, 2008.
Hal Fulton, The Ruby Way, 2nd edition, Addison- Wesley, 2007.
Robert C. Martin, Clean Code, Prentice Hall,
Dave Thomas with Chad Fowler and Andy Hunt, Programming Ruby, 2nd edition, Pragmatic Programmers, 2005.
CIT 383: Administrative Scripting