1 DiSTiL : A Transformation Library for Data Structures Yannis Smaragdakis

2 Overview u DiSTiL is a project to add GenVoca Components to Microsoft’s IP u Domain: container data structures

3 Overview (Continued) u Goal: increase productivity by programming in high-level data structure abstractions cursorcontainer

4 Overview (Continued) u Create complex container data structures by composing DiSTiL components u Design rule checking to ensure validity of component compositions u Automatic selection of data structures according to retrieval predicates

5 Implementation Platform u IP Transformation System (Microsoft Research) u Handling of code with tree primitives u User can define new language primitives ( intentions ) and write code to transform them at compilation ( reduction ) time

6 DiSTiL Library (Realms) u Data Structures (arrays, linked lists, red- black trees, hash-tables) u Storage (Persistent and Transient storage) u Architectural (create code in functions) u Special purpose layers (LRUTree) u Various data-structure related layers (garbage collection, bound checks) u Hidden layers (out-of-bounds, predicate, order)

7 DiSTiL Type Expressions u Typex construct: the composition of components (layers) that defines the target container data structure u Huge number of distinct container data structures can be generated from compositions of DiSTiL components

8 Cursors u Cursors define retrieval predicates over containers cursor(emp_cont, name “Akers”, no_order) u Predicates used to generate efficient code for retrievals u The “smart” part of the library

9 A Composite Data Structure Jones Keen Lam Mann Guy Koch Land Smith Ajit typex Example = malloc[transient]

10 A Composite Data Structure Jones Keen Lam Mann Guy Koch Land Smith Ajit typex Example = tree[malloc[transient]] key=name

11 typex Example = hash [ tree [ malloc [ transient]]] A Composite Data Structure Jones Keen Lam Mann Guy Koch Land Smith Ajit key=name key=phone

12 How to Use

13 What to Notice u Programming in domain-specific abstractions u makes code easier to understand u makes code more reliable (simpler and checked at a higher level) u makes code easier to evolve (can modify typex without modifying program) u should result in very good code

14 What to Notice u DiSTiL actually u can increase programmer productivity (simpler/shorter programs) u removes burden of coding & debugging data structures u can accommodate architectural styles

15 Example Application u LRU memory policy simulation

16 What is different about DiSTil? u Different from static libraries (e.g. STL): u meta-level optimizations (retrieval structure selection, updates) u more advanced static checking (design-rule checking) u level of programming significantly raised (“what” vs “how”) u more general, powerful approach u larger selection of structures/mechanisms

17 What is different about DiSTil? u Even at the usual level of abstraction u static libraries suffer from performance loss u not easy to integrate such a variety of components u example: templates and memory allocation u would have the added cost of a virtual dispatch u static consistency checking is a domain- specific mechanism (type checking)

18 Internal organization Quote Package Static Library Components Exported Symbols Predicate engine Transformatio n engine User Application Generated Application

19 Next Steps u Improve design-rule checking u Clarify design, possibly develop tools for writing generators u Explore ideas (explicit, generation-time scoping) examined in the Quote package u Broaden applications

20 Contributions of DiSTiL u Example of factored libraries (Biggerstaff 1994) u Primitive data structures factored in new ways (out-of-bounds checking, predicates on retrievals, inlined/functional implementation etc.) u Extensive background for software generator development in non-scoped environments (Quote package)

21 Contributions (cont’d) u Layered generator: u analysis of layer interaction u possible mechanisms to facilitate construction u determining pattern of information flow among layers