2012/03/27 DTN2 IETF 83 1 DTN2 – Improvements 2012 SQL storage Auxiliary Tables Persistent Link Names Elwyn Davies Trinity College Dublin/Folly Consulting Ltd DTN RG at IETF 83 Tuesday 27 March 2012

2012/03/27 DTN2 IETF 83 2 Agenda History and Rationale Connecting to an SQL Database Exposing the bundles Doing it better Forwarding Logs and Link Names Comparing Performance Next steps

2012/03/27 DTN2 IETF 83 3 History What DTN2.7 didn't have! SQL database as persistent store – Commented out code for MySQL, PostgreSQL Persistent forwarding logs – After restart have to guess history for reloaded bundles Some routers assume never forwarded ProPHET 'loses' bundles Must have persistent link names for persistent forwarding log to make sense

2012/03/27 DTN2 IETF 83 4 History Mitre Code for DTN2.8 Added SQL-based persistent storage – With controllable transactions as opposed to auto-commit Enable persistent forwarding logs Did not do persistent links – OK provided only uses configured links – Opportunistic links may break forwarding logs across restarts due to random order of re- establishment SQL database makes extra tables possible – needed for Mitre's Army router project

2012/03/27 DTN2 IETF 83 5 Why do more? NetInf device prototype for SAIL Bundle protocol as an instantiation of NetInf protocol using ni: URI scheme – See draft-farrell-decade-ni-00 – Use DTN2 bundle cache as info cache for NetInf device Access bundle cache with other apps.. – To search the bundles – Make them available by a filing system (FUSE)

2012/03/27 DTN2 IETF 83 6 Connecting an SQL database... There are quite a few to choose from Plan A: Write code to interface with their specialized client libraries – Did we do the right one? Plan B: Think Microsoft! – Use ODBC – Open DataBase Connectivity – Common client API – Drivers for almost all useful databases – Minimal specialized code

2012/03/27 DTN2 IETF 83 7 The ODBC Architecture DTN2 BP Agent Oasys Framework SQL Database SQLite, MySQL, etc ODBC Driver Manager ODBC SQL interface odbcinst.ini database connection specs odbc.ini Data Source specifications dtn.conf Storage type (odbc_xxx) Data Source Name odbc.ini Data Source specifications odbcinst.ini Database connection specifications SchemaPreCreationFile SQL code run before main tables are created SchemaPostCreationFile SQL code run after main tables are created

2012/03/27 DTN2 IETF 83 8 Auxiliary Tables Mitre project required independent tables for use with routing – Just add to the schema NetInf project needs certain information 'broken out' from 'data blob' – Requires extra work in DTN2 and Oases – Just need bundles_aux for now

2012/03/27 DTN2 IETF 83 9 Managing Integrity 1-to-1 correspondence between records in bundles & bundles_aux Insertions and deletions in bundles_aux handled by triggers Fields to be written to bundles_aux 'configured' by creating a class (BundleDetail) that generates a list fed to the database 'put' method. bundles_aux is 'write only' for DTN2

2012/03/27 DTN2 IETF What's in bundles_aux? Just a small subset of bundle data Currently interested in some basic data plus BPQ fields for NetInf Very easy to change Add/Remove single call from BundleDetail constructor for each item

2012/03/27 DTN2 IETF Doing it better... Discovered a couple of issues in Mitre code – now fixed Treated globals specially and only handled other tables with integer keys – Didn't work for prophet or links tables. Too much state held in ODBC (maybe) – Needed to free bound parameters Access serialization broken

2012/03/27 DTN2 IETF Completion of making Forwarding Logs Persistent Need to ensure links created after a restart are 'consistent' with ones created before Based on – Name – database key for links – Main configuration parameters

2012/03/27 DTN2 IETF Link Consistency Two cases to consider... Configured links.. – All types except OPPORTUNISTIC – Matching – warning if RemoteEID differs Name Nexthop address Type Convergence layer Opportunistic links – If a link went to same remote_eid before, use the same link name on 'reincarnation'.

2012/03/27 DTN2 IETF Implementation Store link information on creation – Don't permit link info deletion once a link name features in a bundle forwarding log On restart, read stored link info into 'previous links' list When new configured links created check consistency with any links with same name When a new OPPORTUNISTIC link is opened use the same name as used for the remote_eid if there was one previously

2012/03/27 DTN2 IETF New 'link dump' format weee-pc2 dtn% link dump Active links: link_stewie [ dtn:none ONDEMAND tcp state=AVAILABLE] link_basil [ dtn:none ONDEMAND tcp state=AVAILABLE] link-1 [ dtn://quark.dtn OPPORTUNISTIC tcp state=OPEN] Previously active links: link_stewie [ dtn:none ONDEMAND tcp state=AVAILABLE] link_basil [ dtn:none ONDEMAND tcp state=AVAILABLE] link-0 [ dtn://mightyatom.dtn OPPORTUNISTIC tcp state=OPNEN] link-1 [ dtn://quark.dtn OPPORTUNISTIC tcp state=OPEN] weee-pc2 dtn% Configured links OPPORTUNISTIC link Created first after restart but uses same name as when created second previously

2012/03/27 DTN2 IETF Performance Averages of 100 DTNpings Logging level: debug` Logging level: info Local pings on two machines and pings from one to other both ways Sample Results

2012/03/27 DTN2 IETF Results - 1 Four sets of tests (two local pings, two from one node to other) Five different storage mechanisms with transactions enabled Two logging levels The tests gave consistent results: For all four cases and either choice of logging, the order was - Memory - File system (Slower by factor ) - Berkeley DB (Slower by factor ) - ODBC MySQL/SQLite with aux table (Slower by factor ) MySQL was slightly slower than SQLite3 on the single core atom processor but was significantly faster on the dual core quark processor. This is not too surprising as the MySQL server runs in a separate process, offsetting the extra comunication 'costs'.

2012/03/27 DTN2 IETF Results - 2 Ran six extra tests for local pings on quark only (since the ordering was consistent across the various test scenarios). Checks how changing the configuration options affected the results. The following three options were exercised with both info and debug level logging: - Berkeley DB with auto-commit, - ODBC/MySQL with transactions but no auxiliary table, and - ODBC/MySQL with auto commit and no auxiliary table. The results indicate that: - turning on auto-commit slows Berkeley DB by about 10% - turning off auxiliary tables speeds up MySQL by about 10% - turning on auto-commit slows down MySQL by about 20%.

2012/03/27 DTN2 IETF Conclusions from Tests Filesystem works pretty well!. – Adding fsync for filesysdb is definitely worthwhile. BerkeleyDB is pretty efficient. The ODBC/SQL code could do with some optimization – Would have hoped for better performance OTOH, the addition of auxiliary tables is not a major drain. The transactional mechanism is quite worthwhile for SQL but is not particularly significant for BerkeleyDB.

2012/03/27 DTN2 IETF Next Steps Publish code and documentation Finish FUSE filing system/NetInf device Optimize SQL 'put' routine – Ample scope as it uses three database round trips when one would do – Don't worry about 'get' as it is hardly used apart from restarts Try out BerkeleyDB SQL shim Provide automatic reincarnation of links configured after startup

2012/03/27 DTN2 IETF Thank you... Questions?