A Type System for Higher-Order Modules Derek Dreyer, Karl Crary, and Robert Harper Carnegie Mellon University POPL 2003

Type Theory for Module Systems Lots of work on module system design Theory has had impact on real language design: –Harper-Lillibridge 94, Leroy 94 ) SML ’97 –Leroy 95 ) Objective Caml –Russo 00 ) Moscow ML No general semantic framework for understanding relationships between designs

A Unifying Type Theory High-level semantic analysis ) Unifying type theory Previous designs can be seen as subsystems Key idea: Explain semantics of abstract data types in terms of purity and effects

Projectibility When is a module expression M projectible? –When can we project out M’s type components? “Non-projectible” module expression: if buttonIsPressed() then struct type t = int... end else struct type t = bool... end “Projectible” module expression: struct type t = int; val x = 3 end

Projectibility When is a module expression M projectible? –When can we project out M’s type components? “Non-projectible” module expression: if buttonIsPressed() then struct type t = int... end else struct type t = bool... end “Projectible” module expression: struct type t = int; val x = 3 end

Projectibility When is a module expression M projectible? –When can we project out M’s type components? “Non-projectible” module expression: if buttonIsPressed() then struct type t = int... end else struct type t = bool... end “Projectible” module expression: struct type t = int; val x = 3 end

Projectibility When is a module expression M projectible? –When can we project out M’s type components? “Impure” module expression: if buttonIsPressed() then struct type t = int... end else struct type t = bool... end “Pure” module expression: struct type t = int; val x = 3 end

Projectibility When is a module expression M projectible? –When can we project out M’s type components? “Impure” module expression: if buttonIsPressed() then struct type t = int... end else struct type t = bool... end “Pure” module expression: struct type t = int; val x = ref 3 end

Purity M is soundly projectible, M is pure (w.r.t. type components), Type components of M are the same every time M is evaluated M is impure ) Meaning of M.t not statically well-determined

Second-Class Modules Second-class modules admit “phase separation” –Type components can’t depend on run-time conditions SML and O’Caml modules are second-class because of syntactic restrictions All second-class modules are pure –But should they all be projectible?

Sealing Principal means of creating abstract data types –M :> , aka “opaque signature ascription” Treating sealed modules as projectible violates abstraction: –A = (M :>  ) and B = (M :>  ) –If (M :>  ) is projectible, then A.t = (M :>  ).t = B.t –But “A.t = B.t” not observable in 

You Can’t Handle the Truth! In truth, sealing doesn’t affect a module’s purity But sealing obstructs our knowledge about a module’s purity Projectibility is a judgment of knowledge, not truth: –Sealed modules treated as impure/non-projectible

Total and Partial Functors To track purity in the presence of functors: –Need to know whether applying a functor will unleash an effect or not Distinguish types of total and partial functors: –F :  tot s:  1.  2, body of F is known to be pure –F :  par s:  1.  2, body of F could be impure

Total, Applicative F :  tot s:  1.  2, M :  1, F and M are pure structure Res 1 = F(M) structure Res 2 = F(M) F(M) known to be pure ) projectible Res 1.t = F(M).t = Res 2.t

Partial, Generative F :  par s:  1.  2, M :  1, F and M are pure structure Res 1 = F(M) structure Res 2 = F(M) F(M) possibly impure ) non-projectible Res 1.t  Res 2.t

Functors with Sealing If body of a functor contains sealing, then: –Body is impure –Functor is generative Can be both a good and bad thing: –Gives correct semantics of abstraction for functors that use imperative features –Overly restrictive for purely functional functors

Importance of Generativity functor MakeSymbolTable() = (struct... (* creates new mutable hash table *) end :> sig type symbol val string_to_symbol : string -> symbol val symbol_to_string : symbol -> string end) ) Generativity ties the symbol type to the run-time state of the module defining it

Purely Functional Abstraction functor MakeSet (Elem : COMPARABLE) = (struct... end :> sig type elem = Elem.elem type set val insert : elem * set -> set end) What if a sealed module is purely functional? –Abstract types not tied to any run-time state –Only care about hiding type identity

Hoisting the Sealing Instead of sealing the body of the functor: s:  1. (M :>  2 ) Seal the functor itself (with a total signature): ( s:  1. M) :>  tot s:  1.  2 Problem: Only works if M is pure –Not true if M contains sealed substructures, such as datatype definitions

Module Classifications Impure, non-projectible Pure, projectible Sealing M :> 

Static and Dynamic Effects Split effects into two kinds: static and dynamic Module with any kind of effects is impure Dynamic effects occur “during execution” Static effects occur “during typechecking”

Weak and Strong Sealing Pure, projectible Statically impure, dynamically pure Statically and dynamically impure Weak Sealing M ::  Strong Sealing M :>  Non-projectible Impure,

Set Functor Revisited functor MakeSet (Elem : COMPARABLE) = (struct... end :: sig type elem = Elem.elem type set val insert : elem * set -> set end) We expand totality to allow body of a total functor to contain static (but not dynamic) effects

A Unifying Framework Standard ML –Only has strong sealing, all functors are partial/generative Objective Caml / Leroy (1995) –Only has weak sealing, all functors are total/applicative Shao (1999) –Distinguishes two kinds of functor signatures –Only tracks dynamic effects and strong sealing Russo (2000) –Two languages, one like SML and one like O’Caml –Moscow ML combines them, but language is unsound

Modules as First-Class Values Packaging modules as first-class values: –Add a new “package type” Coercions between modules and terms: –If M : , then pack M as : –If e :, then unpack e as  : 

Modules as First-Class Values structure X = struct type t = int... end structure Y = struct type t = bool... end M = (if buttonIsPressed() then X else Y) Type components of M actually depend on run-time conditions Unpacking induces a truly dynamic effect

Modules as First-Class Values signature S = sig type t... end structure X = struct type t = int... end structure Y = struct type t = bool... end M = unpack (if buttonIsPressed() then pack X as else pack Y as ) as S Type components of M actually depend on run-time conditions Unpacking induces a truly dynamic effect

The Rest of the Paper Formalism: –Synthesis of previous work on dependent types and singleton kinds Fully expressive higher-order functors –Via “static” module equivalence Decidable typechecking algorithm Avoidance problem –Restrict type theory (as little as possible) to avoid it –Unrestricted language definable by elaboration

Conclusion Future Work: –Recursive modules –Using monads instead of total/partial We’ve provided a framework for understanding: –Alternative module system designs –Semantics of abstraction, via a framework of module-level effects