CIT 383: Administrative Scripting Data Persistence CIT 383: Administrative Scripting

CIT 383: Administrative Scripting Topics Data Models Delimited Fields Fixed Length Records Serialization Concurrency YAML CIT 383: Administrative Scripting

CIT 383: Administrative Scripting Data Models Flat Model Single two-dimensional array of data elements. Example: CSV Hierarchical Model Data organized into a tree-like structure. Example: XML Relational Model Multiple tables with precisely defined relations. Example: MySQL CIT 383: Administrative Scripting

CIT 383: Administrative Scripting Storage Types Flat Text Files Delimited fields Fixed length records Flat Binary Files Marshal DBM Hierarchical Structured Text XML YAML CIT 383: Administrative Scripting

CIT 383: Administrative Scripting Delimited Fields Text fields separated by a delimiter. CSV /etc/passwd Advantages Simple: use standard file I/O and split. Humanly readable storage. Disadvantages Must have technique to escape delimiter. Slow to access a specific record. Hard to update or delete data (must rewrite file.) CIT 383: Administrative Scripting

CIT 383: Administrative Scripting Fixed Length Records Use record sizes of a fixed number of bytes. aaaabbccccccccdddd Advantages Do not need to use any escaping. Fast direct access to records (n * record_size). Easy to update/delete: just write over record. Disadvantages Wastes space: record size >= size of largest data. Need special character to fill fields to record size. CIT 383: Administrative Scripting

CIT 383: Administrative Scripting Serialization Converting an object into a sequence of bits. A/K/A: marshaling, deflating. Necessary to save data beyond simple strings. Converting an array to a sequence: favs = %w{Haskell OCaml Python Ruby} Marshal.dump(favs) "\004\010[\n\"\fHaskell\"\nOCaml\"\ vPython\"\tRuby" CIT 383: Administrative Scripting

CIT 383: Administrative Scripting Concurrency What if two programs access data at once? Two reads: no problem. Two writes: race condition. Race Condition Example Two payments: A for $50, B for $100, funds = $1000 A 1. data = read(file) data[‘funds’] = 1000 3. data[‘funds’] += 50 data[‘funds’] = 1050 5. write(file, data) 2. data = read(file) data[‘funds’] = 1000 4. data[‘funds’] += 100 data[‘funds’] = 1100 6. write(file, data) CIT 383: Administrative Scripting

CIT 383: Administrative Scripting Locking Files Prevent concurrent access by locking files. A: lock(file) B: lock(file) fails since A has lock A: does stuff with file A: unlock(file) B: lock(file) succeeds B: does stuff with file Ruby code for file locking: file = File.new(‘data’) file.flock(File::LOCK_EX) ... do stuff with file ... file.flock(File::LOCK_UN) CIT 383: Administrative Scripting

CIT 383: Administrative Scripting YAML YAML Ain’t a Markup Language Originally Yet Another Markup Language. Changed since primarily used to store data. Simple language for storing data. Humanly readable and writable. Less verbose than XML. Superset of JSON (JavaScript Object Notation.) CIT 383: Administrative Scripting

CIT 383: Administrative Scripting YAML Syntax Lists --- # List of languages - Ruby - Python - Perl --- # Short form of list [Ruby, Python, Perl] CIT 383: Administrative Scripting

CIT 383: Administrative Scripting YAML Syntax Hashes --- # User record username: smithj uid: 11387 gcos: John Smith --- # Short form of hash {username: smithj, uid: 11387, gcos: John Smith} CIT 383: Administrative Scripting

CIT 383: Administrative Scripting YAML Syntax Strings Regular strings need no punctuation. Long strings use --- beginning. --- | This section preserves newlines just as they appear. --- > This section ignores newlines, and their only purpose is to make text readable. CIT 383: Administrative Scripting

CIT 383: Administrative Scripting YAML vs XML XML <user id="smith"> <firstname>Al </firstname> <lastname>Smith </lastname> <phone>555-1111 </phone> <email>al@gmail.com </email> <email>smith@nku.edu </user> YAML smith: firstname: Al lastname: Smith phone: 555-1111 emails: - email: al@gmail.com - email: smith@nku.edu CIT 383: Administrative Scripting

CIT 383: Administrative Scripting YAML in Ruby Serializing objects to YAML require ‘yaml’ favs = %w{Haskell OCaml Python Ruby} puts favs.to_yaml Retrieving objects from YAML file = File.new(‘data.yaml’) favs = YAML.load(file) CIT 383: Administrative Scripting

CIT 383: Administrative Scripting References Michael Fitzgerald, Learning Ruby, O’Reilly, 2008. David Flanagan and Yukihiro Matsumoto, The Ruby Programming Language, O’Reilly, 2008. Hal Fulton, The Ruby Way, 2nd edition, Addison- Wesley, 2007. Robert C. Martin, Clean Code, Prentice Hall, 2008. Dave Thomas with Chad Fowler and Andy Hunt, Programming Ruby, 2nd edition, Pragmatic Programmers, 2005. CIT 383: Administrative Scripting