Combining Compile-Time and Run-Time Components Arjen van Weelden - Rinus Plasmeijer University of Nijmegen www.cs.kun.nl/~clean

Compile-time versus Run-time At compile-time, programming languages offer nice tools for abstraction: (higher order) functions, OO, overloading, generic functions … composition: function application, procedure calls, method application, … verification: type systems, abstract interpretation, formal proofs, … At run-time, operating systems offer limited tools for abstraction: files, processes, dll, xml format, IDL, composition: shell commands (application arguments > file) verification: file extensions, version numbers, xml parser Distributed and mobile components become more important better abstraction, composition and verification run-time tools better integration of compile-time and run-time concepts

Functions are nice components Lazy and pure functional languages (Haskell, Clean) Functions are flexible components with very nice properties Self-contained (like objects) Side-effect free: makes it easier to reason about the functionality Functions that perform I/O can be recognized by their (uniqueness) types Interface is completely defined by the type Data structures can be recursive, polymorphic, infinite High expressive power: higher order functions, rank-n polymorphism, overloading generic functions, high order types, … Function application: type safe way to combine components How can we use this both at compile-time and at run-time ?? Make functions and their types available both at compile-time and at run-time!

Packing an expression into a dynamic (based on work of Cardelli) Clean: uses compiled code, strongly typed Static typing: warn for type errors as early as possible: at compile time Dynamic typing: warn for type errors when things are being used: at run-time Clean: Hybrid type system: both static as well as dynamic typing. With the keyword dynamic one can pack any expression of any static type t into a "dynamic“. Statically, all dynamics have type Dynamic. True :: Bool fib 3 :: Int fib :: Int  Int reverse :: A.a: [a]  [a]

Packing an expression into a dynamic (based on work of Cardelli) Clean: uses compiled code, strongly typed Static typing: warn for type errors as early as possible: at compile time Dynamic typing: warn for type errors when things are being used: at run-time Clean: Hybrid type system: both static as well as dynamic typing. With the keyword dynamic one can pack any expression of any static type t into a "dynamic“. Statically, all dynamics have type Dynamic. (dynamic True :: Bool ) :: Dynamic (dynamic fib 3 :: Int ) :: Dynamic (dynamic fib :: Int  Int ) :: Dynamic (dynamic reverse :: A.a: [a]  [a] ) :: Dynamic

Packing an expression into a dynamic (based on work of Cardelli) Clean: uses compiled code, strongly typed Static typing: warn for type errors as early as possible: at compile time Dynamic typing: warn for type errors when things are being used: at run-time Clean: Hybrid type system: both static as well as dynamic typing. With the keyword dynamic one can pack any expression of any static type t into a "dynamic“. Statically, all dynamics have type Dynamic. (dynamic True :: Bool ) :: Dynamic (dynamic fib 3 :: Int ) :: Dynamic (dynamic fib :: Int  Int ) :: Dynamic (dynamic reverse :: A.a: [a]  [a] ) :: Dynamic In principle, expressions of any Clean type can be packed into a Dynamic

Unpacking an expression from a dynamic Use pattern matching to examine if an argument of dynamic type :: Dynamic is of a particular static type f :: Dynamic  Int f (0 :: Int) = 0 f (n :: Int) = n * n + 1 f else = 1

Unpacking an expression from a dynamic Use pattern matching to examine if an argument of dynamic type :: Dynamic is of a particular static type f :: Dynamic  Int f (0 :: Int) = 0 f (n :: Int) = n * n + 1 f else = 1 dynApply :: Dynamic Dynamic  Dynamic dynApply (f :: a  b) (x :: a) = dynamic f x :: b dynApply _ _ = dynamic abort "dynamic type error"

Unpacking an expression from a dynamic Use pattern matching to examine if an argument of dynamic type :: Dynamic is of a particular static type f :: Dynamic  Int f (0 :: Int) = 0 f (n :: Int) = n * n + 1 f else = 1 dynApply :: Dynamic Dynamic  Dynamic dynApply (f :: a  b) (x :: a) = dynamic f x :: b dynApply _ _ = dynamic abort "dynamic type error" With a pattern match one can: check the type of the expression in a dynamic check values stored in the dynamic Dynamic type pattern variables force unification between dynamics One can only use a dynamic expression after a successful dynamic pattern match Run-time type checking, type unification, and type errors !

Functions to perform I/O of Dynamics (M. Vervoort, IFL paper) read and write a Dynamic from/to disk with just one single function writeDynamic :: String Dynamic *World  *(Bool, *World) readDynamic :: String *World  (Bool, Dynamic, *World ) very easy to use

Example of the use of Dynamic I/O producer :: *World  *(Bool, *World) producer world = writeDynamic “primes” (dynamic sieve [2..]) world where sieve :: [Int]  [Int] sieve [prime:rest] = [prime : sieve filter ] filter = [ h \\ h <- rest | h mod prime <> 0 ] consumer :: *World  [Int] consumer world = case readDynamic “primes” world of (_, list :: [Int], _)  take 100 list other  [ ]

Example of the use of Dynamic I/O producer :: *World  *(Bool, *World) producer world = writeDynamic “primes” (dynamic sieve [2..]) world where sieve :: [Int]  [Int] sieve [prime:rest] = [prime : sieve filter ] filter = [ h \\ h <- rest | h mod prime <> 0 ] consumer :: *World  [Int] consumer world = case readDynamic “primes” world of (_, list :: [Int], _)  take 100 list other  [ ]

What can one do with Dynamics ? Advantages: Type safe exchange of any data and code between independent applications No more boring I/O handling to program (typical 30% of program code) Ouput: conversion to some (string) format, Input: parser required Persistent applications easier to make: store and retrieve any state Adding plug-ins in a type-safe way: dynamic type checking, linking Combining mobile components in a flexible and (type-) safe way but not so easy to implement ….

What to store if we write a dynamic ? To plug-in a dynamic: run-time generated information is needed: the stored dynamic (graph) expression + its type compile-time generated information is needed compiled code ! of function definitions + all type definitions a dynamic linker is needed to plug-in code in a running application.

Sharing between dynamics on disk applyExample world let (fun, _, world) = readDynamic “function" world (arg, _, world) = readDynamic “argument" world in writeDynamic "result" (dynApply fun arg) world dynApply :: Dynamic Dynamic  Dynamic dynApply (f :: a  b) (x :: a) = dynamic f x :: b dynApply df dx = abort ”dynamic type error” Dynamic on disks are not self contained but they can refer to other dynamics Due to nested dynamics one can have references to partitions in other files Due to lazy evaluation one never knows what is actually stored Dynamics on disk are snapshots of the graph rewriting process: DON’T TOUCH

Requirements for dynamic I/O Avoid duplication of work: preserve sharing in expressions Avoid unnecessary work: reconstruct a dynamic lazy when needed for evaluation Only allow type safe Plug-ins: only add new code when the types are OK No loss in efficiency compared to a static Clean application once a running application has been extended with function definitions. Dynamics on disk are “typed”-files which one should be able to use (move, copy, or delete) like any other ordinary file. The whole system should also work in a distribute environment and be robust ! A careful designed architecture is required.

Static Clean System Architecture at compile-time ..dcl ..icl Clean Sources My Application Clean Compile-Time Run-Time User Space System Space

Static Clean System Architecture at compile-time ..prj ..dcl ..icl Clean Compiler Clean Integrated Development Environment Clean ..abc Clean Sources Code Generator C Platform Independent ..obj Static Linker Clean Platform Dependent My Application Clean Compile-Time Run-Time User Space System Space

Static Clean System Architecture at run-time My Application Clean Compile-Time Run-Time User Space System Space

Dynamic Clean System Architecture at compile-time ..prj ..dcl ..icl Clean Compiler Clean Integrated Development Environment Clean ..abc Clean Sources Code Generator C Platform Independent ..obj Static Linker Clean Platform Dependent My Application Clean Compile-Time Run-Time User Space System Space

Dynamic Clean System Architecture at compile-time ..prj ..dcl ..icl Clean Compiler Clean Integrated Development Environment Clean ..abc Clean Sources Code Generator C Platform Independent ..obj Static Linker Clean Platform Dependent ..lib ..typ Symbolic Code Type Definitions My Application Clean Compile-Time Run-Time User Space System Space

Dynamic Clean System Architecture at run-time ..lib ..typ Symbolic Code Type Definitions My Application Clean Client Binary Application (.exe) Compile-Time Run-Time System Space

Dynamic Clean System Architecture at run-time ..lib ..typ Symbolic Code Type Definitions My Application Clean Client Binary Application (.exe) Dynamic Linker Clean Server Compile-Time Run-Time System Space

Dynamic Clean System Architecture at run-time ..lib ..typ Symbolic Code Type Definitions My Application Clean Client ..sysdyn Binary Application (.exe) Dynamic Linker Clean .dyn Server Links to Dynamics Dynamics Compile-Time Run-Time User Space System Space

Dynamic Clean System Architecture at run-time ..lib ..typ Symbolic Code Type Definitions My Application Clean Client ..sysdyn Binary Application (.exe) Dynamic Linker Clean .dyn Server Links to Dynamics Dynamics Your Application Clean Client Compile-Time Run-Time User Space System Space

Dynamic Clean System Architecture at run-time ..lib ..typ Symbolic Code Type Definitions My Application Clean Client ..sysdyn Binary Application (.exe) Dynamic Linker Clean .dyn Server Links to Dynamics Dynamics Your Application Clean Plug In Clean Client Compile-Time Run-Time User Space System Space

Robustness and user friendliness How to make such a system robust and user friendly ? All dynamics on disk and all repositories are completely hidden from the user and maintained by the Clean run-time system that preserves their integrity (MD5). Instead of the actual "system" dynamic the user sees a "user" dynamic which is just a direct reference to the actual dynamic. A "user" dynamic can be renamed, copied, and deleted, as any other file. There is a special garbage collector that removes unused dynamics and code and type repositories. There is a special copy function that can transfer a dynamic (with all the dynamics and repositories it is depending on) to another processor.

Current State + Future work Prototype of a typed Operating System (Arjen van Weelden) A typed shell with all basic features of a functional language Not a common interpreter: it combines compiled code in a type safe way Freely mix compile-time and run-time definitions Future work Investigate suitability architecture as general purpose middleware layer Investigate dynamic use of statically defined generic functions Combine with generic graphical user interfaces

Conclusion Conclusion: Dynamics & Dynamic I/O: enable type safe communication of any expression to any other application. The actual implementation is rather complicated: it includes dynamic unification, dynamic linking and gives rise to complex data structures on disk. All these things can be hidden from the user: working with dynamics is easy. Good integration of flexible run-time and compile-time components: functions