1. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.1 TDB: A Trusted Database System on Untrusted Storage Bill Shapiro Strategic Technologies and Architectural Research Lab, InterTrust Technologies February, 2001 In Collaboration with Umesh Maheshwari and Radek Vingralek

2. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.2 Motivating Application: DRM Untrusted consumer Trusted server database ŸDigital rights management ŸDatabase: account balances, contracts ŸKeep database on client for –offline commerce –performance: local aggregation net

3. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.3 Application: DRM ŸInterRights Point (IRP) – a secure process –Trusted processing environment, small amount of trusted storage –May use software obfuscation, self-checking or secure co-processor ŸProtected Database (PDB) –Secures a bulk of data on an untrusted store –May only be accessed through IRP InterRights Point PDB trusted app Trusted Server untrusted app Untrusted Consumer

4. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.4 Application: Network Storage storage Trusted PC/PDA Untrusted storage server net TDB

5. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.5 What does TDB do? trusted processing CPU, RAM, NVRAM untrusted storage ŸGiven a trusted processing environment –runs only trusted programs –protects volatile state –has small amount of trusted storage ŸTDB extends trust to scalable storage –secrecy –tamper-detection trusted processing CPU, RAM, NVRAM trusted storage TDB

6. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.6 Design Constraints: Integrity ŸResist accidental corruption (like most Database Systems) –transactions –backups –type checking –index maintenance ŸResist malicious corruption, e.g., replay attack save a copy of database (credit = $100) purchase content(credit = $0) restore saved copy(credit = $100) –Attack works even if data is encrypted and signed

7. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.7 Target Environment ŸMay run in embedded environment –small footprint (~300 KB) –no need for sophisticated query processing / query languages –can be tightly integrated with programming environment (C++) ŸSupport concurrent transactions ŸOptimized for single user –low concurrency and load –modest working set –response time more important than throughput

8. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.8 Tamper Detection Basis ŸLeverage persistent storage in trusted environment collision-resistant hash trusted environment trusted storage secret key, hash value X Y ZW

9. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.9 Tamper Resistance Basis ŸLeverage persistent storage in trusted environment Merkle Tree, 1980 h(X)h(Y)h(Z)h(W) h(S)h(T) X Y ZW S T R trusted environment trusted storage secret key, h(R)

10. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.10 Potential Architecture ŸCrypto over existing database system existing database system write read ŸProblems –unprotected metadata –no sorted indexes on encrypted data metadata encryption + hash-tree

11. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.11 TDB Architecture Trusted storage Untrusted storage Chunk Store encryption, hashing atomic updates Object Store object cache concurrency control Collection Store index maintenance scan, match, range Chunk byte sequence 100B--100KB Object abstract type 1 object / chunk Backup Store full / incremental validated restore Archival storage Collections of Objects basic querys

12. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.12 Chunk Store ŸInterface –allocate()  ChunkId –write(ChunkId, Bytes) –read(ChunkId)  Bytes –deallocate(ChunkId) ŸCrash atomicity –commit = [write|deallocate]* ŸTamper detection and secrecy –uniform protection of data and metadata

13. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.13 Implementation ŸLog-structured ( ala Sprite LFS) –no statically located chunks –log in untrusted store ŸAdvantages –traffic analysis can’t link updates to same object –supports variable-size chunks –atomic, efficient updates –more... X Y Z X

14. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.14 Chunk Map ŸCombines hash tree and location map Map: ChunkId  Handle Handle = ‹Hash, Location› MetaChunk = Array[Handle] trusted storage h(R) X Y meta chunks data chunks T S R

15. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.15 Read ŸBasic scheme: Dereference handles from root to X ŸDerefence use location to fetch use hash to validate trusted storage h(R) X Y T S R cached ŸOptimized trusted cache: ChunkId  Handle look for cached handle upward from X derefence handles down to X avoids validating entire path

16. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.16 Write ŸBasic: write chunks from X to root ŸOptimized: –buffer dirty handle of X in cache –defer upward propagation –as in LFS, but extended to hashing trusted storage h(R) X Y T S R dirty

17. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.17 Checkpointing the Map ŸWhen dirty handles fill cache –write affected meta chunks to log –write root chunk last X... X S R T meta chunks trusted storage h(R)

18. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.18... Y Crash Recovery ŸProcess log from last root chunk –residual log –checkpointed log ŸMust validate residual log crash X... X S R T trusted storage h(R) residual log

19. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.19 Validating the Log ŸKeep sequential hash of residual log in trusted storage –updated after each commit ŸHash protects all current chunks –in residual log: directly –in checkpointed log: through chunk map... Y crash X... X S R T trusted storage h(residual-log) residual log

20. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.20... c.c. 74 X c.c. 73 Counter-Based Log Validation ŸA commit chunk is written with each commit –contains a sequential hash of commit set –authenticated with system secret key ŸIncrement-only counter used to prevent replays ŸBenefits: –allows bounded discrepancy between TR store and untrusted store –doesn’t require writing to TR store after each transaction crash X... X S R T residual log hash

21. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.21 Log Cleaning ŸSegments –Log divided into fixed-sized regions called segments ( ~100 KB) –Segments are threaded in the residual log for recovery ŸLog cleaner reclaims obsolete chunk versions, creating free segments ŸCleaning step –read 1 or more segments –write current chunk versions to end of log –mark segments as free ŸMay not clean segments in residual log

22. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.22 Log Cleaning ŸCleans segments based on statistics (e.g., segment utilization) ŸPerformance –cleaning occurs mostly during idle periods –can trade-off DB size (utilization) for performance

23. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.23 Multiple Partitions ŸPartitions may use separate crypto parameters (algorithms, keys) ŸEnables fast snapshots and efficient backups Ÿe.g., partition copies: Partition Map Position Maps Data chunks P Q Partition Map Position Maps Data chunks P Q DD2

24. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.24 Backup Store ŸGuarantees backup consistency without locking ŸSupports full and incremental backups of partitions ŸFull backup: –create a partition snapshot ŸIncremental backup –Created with respect to a previous snapshot (base) ŸA trusted program must authorize backup restores

25. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.25 Cleaning and Partition Copies Q&P P.a P.bP.c PQ P.aP.b PQ P.c P.aP.bP.c Copy P  Q P updates cCleaner moves Q’s c P.aP.bP.c...P.c...P.c... Checkpoint Crash!! Residual log

26. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.26 Object Store ŸProvides type-safe access to named (C++) objects ŸImplements full transactional semantics, concurrency control ŸObjects provide pickle and unpickle methods for persistence ŸProvides an in-memory cache of C++ objects ŸImplements no-steal buffer management policy

27. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.27 Collection Store ŸSupports indexed collections of C++ objects ŸProvides automatic index maintenance during updates ŸUses functional indices –An extractor function is used to obtain a key from an object ŸSupports exact match, range and scan queries ŸCurrently includes B-tree, dynamic hash table and list indices

28. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.28 Performance Evaluation ŸCompared TDB to Berkeley DB using TPC-B ŸUsed TPC-B because: –Included with Berkeley DB –Berkeley DB functionality limited choice of benchmarks (e.g., 1 index per collection) ŸTPC-B Table sizes : 252000History 100Branch 1000Teller 100000Account

29. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.29 Performance Evaluation: Platform ŸTDB -- C++ implementation, 6000 semicolons ŸStatic library (.text section 297 KB) ŸEvaluation platform –733 MHz Pentium II, 256 MB –Windows NT 4.0, NTFS files – EIDE disk, 8.9 ms (read), 10.9 ms write seek time – 7200 RPM (4.2 ms avg. rot. latency) –One-way counter: file on NTFS ŸBoth systems used a 4 MB cache ŸCrypto Params (for secure version of TDB): –SHA-1 for hashing –3DES for encryption

30. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.30 Performance Results ŸResponse Time (avg over 100,000 transactions in steady state): ŸTDB utilization was set to 60% 6.8 3.8 5.8 0 1 2 3 4 5 6 7 8 BerkeleyDBTDBTDB-S avg. response time (ms)

31. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.31 Response Time vs. Utilization ŸMeasured response times for different TDB utilizations:

32. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.32 Related Work ŸTheoretical work –Merkle Tree 1980 –Checking correctness of memory (Blum, et. al. 1992) ŸSecure audit logs, Schneier & Kelsey 1998 –append-only data –read sequentially ŸInformation dispersal to multiple storage servers –Rabin 1989, Castro & Liskov 1999 –assumes weakly protected servers, honest quorum –provides tamper recovery ŸRead-only SFS, Fu et al. 2000 –offline update –supports many readers

33. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.33 Conclusions ŸArchitecture –To protect data and metadata alike, integrate crypto with low-level storage management ŸImplementation –Industrial-strength implementation of a Trusted Database System –Synergies between log-structured storage and trust management ŸPerformance –TDB performs well compared to Berkeley DB –Can maintain up to 80% utilization with little performance degradation ŸFor More Information: –OSDI 2000 -- “How to Build a Trusted Database System on Untrusted Storage.” Umesh Maheshwari, Radek Vingralek, William Shapiro –Technical Report available at http://www.star-lab.com/

34. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.34 Database Size vs. Utilization

35. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.35 Alternative Architectures Untrusted storage trusted file system existing DBMS Untrusted storage trusted chunk store new file system existing DBMS Untrusted storage trusted chunk store new DBMS (1) (2)(3)

36. Feb, 2001Bill Shapiro, © Copyright, InterTrust Technologies Corporation, 2001. All rights reserved.36 Space Overhead ŸChunk descriptor –hash12 –location6 ŸChunk header –chunk id6 –size4 –pad6 ŸChunk body –pad 4 ŸLog utilization