2012/03/27 DTN2 Improvements @ IETF 83 1

DTN2 – Improvements 2012SQL storage

Auxiliary TablesPersistent Link Names

Elwyn DaviesTrinity College Dublin/Folly Consulting Ltd

DTN RG at IETF 83Tuesday 27 March 2012

2012/03/27 DTN2 Improvements @ IETF 83 2

Agenda

• History and Rationale• Connecting to an SQL Database• Exposing the bundles• Doing it better• Forwarding Logs and Link Names• Comparing Performance• Next steps

2012/03/27 DTN2 Improvements @ IETF 83 3

HistoryWhat DTN2.7 didn't have!

• SQL database as persistent store– Commented out code for MySQL, PostgreSQL

• Persistent forwarding logs– After restart have to guess history for reloaded

bundles• Some routers assume never forwarded• ProPHET 'loses' bundles

• Must have persistent link names for persistent forwarding log to make sense

2012/03/27 DTN2 Improvements @ IETF 83 4

HistoryMitre Code for DTN2.8

• Added SQL-based persistent storage–With controllable transactions

as opposed to auto-commit

• Enable persistent forwarding logs• Did not do persistent links– OK provided only uses configured links– Opportunistic links may break forwarding logs across

restarts due to random order of re-establishment

• SQL database makes extra tables possible– needed for Mitre's Army router project

2012/03/27 DTN2 Improvements @ IETF 83 5

Why do more? • NetInf device prototype for SAIL• Bundle protocol as an instantiation of NetInf protocol

using ni: URI scheme– See draft-farrell-decade-ni-00– http://sourceforge.net/projects/netinf

• Use DTN2 bundle cache as info cache for NetInf device

• Access bundle cache with other apps..– To search the bundles–Make them available by a filing system (FUSE)

2012/03/27 DTN2 Improvements @ IETF 83 6

Connecting an SQL database...

• There are quite a few to choose from• Plan A: Write code to interface with their

specialized client libraries– Did we do the right one?

• Plan B: Think Microsoft!– Use ODBC – Open DataBase Connectivity– Common client API– Drivers for almost all useful databases– Minimal specialized code

2012/03/27 DTN2 Improvements @ IETF 83 7

The ODBC Architecture

DTN2 BP Agent

Oasys Framework

SQL DatabaseSQLite, MySQL, etc

ODBCDriver Manager

ODBCSQL interface

odbcinst.inidatabase

connection specs

odbc.iniData Source

specifications

dtn.confStorage type (odbc_xxx)

Data Source Name

odbc.iniData Source

specifications

odbcinst.iniDatabase connection

specifications

SchemaPreCreationFile SQL code run before

main tables are created

SchemaPostCreationFile SQL code run after

main tables are created

2012/03/27 DTN2 Improvements @ IETF 83 8

Auxiliary Tables

• Mitre project required independent tables for use with routing– Just add to the schema

• NetInf project needs certain information 'broken out' from 'data blob'– Requires extra work in DTN2 and Oases– Just need bundles_aux for now

2012/03/27 DTN2 Improvements @ IETF 83 9

Managing Integrity

• 1-to-1 correspondence between records in bundles & bundles_aux

• Insertions and deletions in bundles_aux handled by triggers

• Fields to be written to bundles_aux 'configured' by creating a class (BundleDetail) that generates a list fed to the database 'put' method.

• bundles_aux is 'write only' for DTN2

2012/03/27 DTN2 Improvements @ IETF 83 10

What's in bundles_aux?

• Just a small subset of bundle data• Currently interested in some basic data plus

BPQ fields for NetInf• Very easy to change• Add/Remove single call from BundleDetail

constructor for each item

2012/03/27 DTN2 Improvements @ IETF 83 11

Doing it better...

• Discovered a couple of issues in Mitre code – now fixed

• Treated globals specially and only handled other tables with integer keys– Didn't work for prophet or links tables.

• Too much state held in ODBC (maybe)– Needed to free bound parameters

• Access serialization broken

2012/03/27 DTN2 Improvements @ IETF 83 12

Completion of makingForwarding Logs Persistent

• Need to ensure links created after a restart are 'consistent' with ones created before

• Based on– Name – database key for links–Main configuration parameters

2012/03/27 DTN2 Improvements @ IETF 83 13

Link ConsistencyTwo cases to consider...

• Configured links.. – All types except OPPORTUNISTIC– Matching – warning if RemoteEID differs

• Name• Nexthop address• Type• Convergence layer

• Opportunistic links– If a link went to same remote_eid before,

use the same link name on 'reincarnation'.

2012/03/27 DTN2 Improvements @ IETF 83 14

Implementation

• Store link information on creation– Don't permit link info deletion once a link name features

in a bundle forwarding log

• On restart, read stored link info into 'previous links' list

• When new configured links created check consistency with any links with same name

• When a new OPPORTUNISTIC link is opened use the same name as used for the remote_eid if there was one previously

2012/03/27 DTN2 Improvements @ IETF 83 15

New 'link dump' format

weee-pc2 dtn% link dumpActive links:link_stewie [192.168.2.2 dtn:none ONDEMAND tcp state=AVAILABLE]link_basil [134.226.36.138 dtn:none ONDEMAND tcp state=AVAILABLE]link-1 [81.187.254.246 dtn://quark.dtn OPPORTUNISTIC tcp state=OPEN]

Previously active links:link_stewie [192.168.2.2 dtn:none ONDEMAND tcp state=AVAILABLE]link_basil [134.226.36.138 dtn:none ONDEMAND tcp state=AVAILABLE]link-0 [81.187.254.250 dtn://mightyatom.dtn OPPORTUNISTIC tcp state=OPNEN]link-1 [81.187.254.246 dtn://quark.dtn OPPORTUNISTIC tcp state=OPEN]weee-pc2 dtn%

Configuredlinks

OPPORTUNISTIC linkCreated first after restart

but uses same name as when created second previously

2012/03/27 DTN2 Improvements @ IETF 83 16

PerformanceAverages of 100 DTNpings

Logging level: debug`

Logging level: info

Local pings on two machines and pings from one to other both waysSample Results

2012/03/27 DTN2 Improvements @ IETF 83 17

Results - 1Four sets of tests (two local pings, two from one node to other)Five different storage mechanisms with transactions enabledTwo logging levels

The tests gave consistent results:For all four cases and either choice of logging, the order was - Memory- File system (Slower by factor 1.1 - 1.2)- Berkeley DB (Slower by factor 1.4 - 2)- ODBC MySQL/SQLite with aux table (Slower by factor 2.5 - 6)

MySQL was slightly slower than SQLite3 on the single core atom processor but was significantly faster on the dual core quark processor.

This is not too surprising as the MySQL server runs in a separate process, offsetting the extra comunication 'costs'.

2012/03/27 DTN2 Improvements @ IETF 83 18

Results - 2

Ran six extra tests for local pings on quark only (since theordering was consistent across the various test scenarios).Checks how changing the configuration options affected the results. The

following three options were exercised with both info and debug level logging:

- Berkeley DB with auto-commit,- ODBC/MySQL with transactions but no auxiliary table, and- ODBC/MySQL with auto commit and no auxiliary table.

The results indicate that:- turning on auto-commit slows Berkeley DB by about 10%- turning off auxiliary tables speeds up MySQL by about 10%- turning on auto-commit slows down MySQL by about 20%.

2012/03/27 DTN2 Improvements @ IETF 83 19

Conclusions from Tests

• Filesystem works pretty well!.– Adding fsync for filesysdb is definitely worthwhile.

• BerkeleyDB is pretty efficient.• The ODBC/SQL code could do with some optimization– Would have hoped for better performance

• OTOH, the addition of auxiliary tables is not a major drain.

• The transactional mechanism is quite worthwhile for SQL but is not particularly significant for BerkeleyDB.

2012/03/27 DTN2 Improvements @ IETF 83 20

Next Steps• Publish code and documentation• Finish FUSE filing system/NetInf device• Optimize SQL 'put' routine– Ample scope as it uses three database round trips

when one would do– Don't worry about 'get' as it is hardly used apart

from restarts

• Try out BerkeleyDB SQL shim• Provide automatic reincarnation of links

configured after startup

2012/03/27 DTN2 Improvements @ IETF 83 21

Thank you...

Questions?