Evolving the ML Module SystemEvolving the ML Module System

Derek Dreyer

Toyota Technological Institute at Chicago

April 15, 2004

2

Data AbstractionData Abstraction

• Should be able to restrict how much clients of a program module know about its implementation

• Enforcement of program invariants

• Protect clients from implementation changes

3

Object-Oriented Approach (Java)Object-Oriented Approach (Java)

• Classes/objects encapsulate code with data

• Private fields/methods are inaccessible to clients

• Semantics of data abstraction tied up with other features of OOP– Inheritance, subtyping, dynamic dispatch

4

Module-Oriented Approach (ML)Module-Oriented Approach (ML)

• Modules are units of “core-language” code

• Interface of a module describes what other modules get to know about it

• Implementor-side data abstraction via sealing

• Client-side data abstraction via functors

5

Evolving the ML Module System Evolving the ML Module System

• Want to make the ML module system even better:– e.g. Add support for recursive modules

6

Evolving the ML Module System Evolving the ML Module System

• Want to make the ML module system even better:– e.g. Add support for recursive modules

• But where do we start?– Several variants of the ML module system:

• Standard ML, Objective Caml, Moscow ML, etc.

– Relationships/tradeoffs between them are unclear

7

The Goal of This WorkThe Goal of This Work

• Develop a unifying account of existing variations on the ML module system

• Build upon this foundation with support for new features (e.g. recursive modules)

8

OverviewOverview

• Concrete examples of data abstraction in ML• High-level analysis of data abstraction in ML• Extending ML with recursive modules• Future work

9

IntSet ModuleIntSet Module

• Module implementing sets of integers:

module IntSet =

mod

type set = int list

val emptyset : set = []

fun insert (x:int,S:set) : set = x::S

fun member (x:int,S:set) : bool = ...

...

end

10

Using the IntSet ModuleUsing the IntSet Module

• Clients use “dot notation” to refer to components of the IntSet module:

module IntSet = mod ... end

val S : IntSet.set =

IntSet.insert(3, IntSet.emptyset)

11

Using the IntSet ModuleUsing the IntSet Module

• Clients use “dot notation” to refer to components of the IntSet module:

module IntSet = mod ... end

val S : int list =

IntSet.insert(3, IntSet.emptyset)

12

Abstract IntSet InterfaceAbstract IntSet Interface

• Hide definition of set type in interface of IntSet:

interface INT_SET =

iface

type set

val emptyset : set

fun insert : int * set -> set

fun member : int * set -> bool

... end

13

Data Abstraction via SealingData Abstraction via Sealing

• Seal the implementation with the interface:

module IntSet =

(mod ... end) :> INT_SET

• Clients of IntSet can’t see definition of IntSet.set

• IntSet.set is an abstract type

14

Modules Can Have EffectsModules Can Have Effects

• IntSet module was purely functional– Body just defined values and functions

• But modules can have side effects– E.g. creation of mutable state

15

Symbol Table ModuleSymbol Table Module

• When evaluated, generates a new symbol table:

module SymbolTable =

mod

val table = HashTable.create(...)

type symbol = int

fun string_to_symbol (x:string) = ...

fun symbol_to_string (n:symbol) = ...

end

16

Sealing the Symbol TableSealing the Symbol Table

module SymbolTable =

(mod ... end) :>

iface

type symbol

fun string_to_symbol : string -> symbol

fun symbol_to_string : symbol -> string

...

end

17

Making Sets More GenericMaking Sets More Generic

• IntSet module only supports integer sets

• Implementation of sets basically the sameregardless of what the type of items in the set is

• Functors allow you to implement generic sets– Can be instantiated with different item types

– Similar to templates in C++, but more powerful

18

FunctorsFunctors

• A functor is a function from modules to modules:

Some Comparable

Item Type

Set Functor

Sets of

That Item

input output

19

Interface of Comparable ItemsInterface of Comparable Items

interface COMPARABLE =

iface

type item

fun compare : item * item -> bool

end

20

The Set FunctorThe Set Functor

module Set =

functor (Item : COMPARABLE) ->

mod

type set = Item.item list

fun member (x,S) =

...Item.compare(x,y)...

...

end

21

Applying the Set FunctorApplying the Set Functor

module IntItem =

mod

type item = int

fun compare(x,y) = Int.compare(x,y)

end

module IntSet = Set(IntItem)

22

Applying the Set FunctorApplying the Set Functor

module StringItem =

mod

type item = string

fun compare(x,y) = String.compare(x,y)

end

module StringSet = Set(StringItem)

23

Two Forms of Data AbstractionTwo Forms of Data Abstraction

• Can seal a module with an abstract interface– Implementor-side abstraction

• Can use functors to make a module more generic– Client-side abstraction

24

OverviewOverview

• Concrete examples of data abstraction in ML• High-level analysis of data abstraction in ML• Extending ML with recursive modules• Future work

25

Type Components of ModulesType Components of Modules

• Modules in ML have type components

• Can “project out” type components of modules:– e.g. IntSet.set, SymbolTable.symbol.

• In all the examples so far, we’ve only projected types out of module variables (or names)

26

Module ExpressionsModule Expressions

• Examples of module expressions:

mod ... end

(mod ... end) :> (iface ... end)

Set(IntItem)

27

QuestionQuestion

• Why not be able to project types from arbitrary module expressions?

mod ... end

(mod ... end) :> (iface ... end)

Set(IntItem)

28

ExampleExample

interface I =

iface type t ... end

module A =

(mod type t = int ... end) :> I

module B =

(mod type t = float ... end) :> I

29

Non-Projectible ModuleNon-Projectible Module

• Suppose M = if button_is_selected() then A else B

30

Non-Projectible ModuleNon-Projectible Module

• Suppose M = if button_is_selected() then A else B

module C = M

module D = M

31

Non-Projectible ModuleNon-Projectible Module

• Suppose M = if button_is_selected() then A else B

module C = M

module D = M

• If M.t is a valid type, then C.t = M.t = D.t • But C.t might be int and D.t might be float!!• Unsound for M to be projectible

32

Projectible ModuleProjectible Module

• Suppose M = mod type t = int; val x = 3 end

module C = M

module D = M

• Fine if M is projectible, since C.t = M.t = D.t = int

33

PurityPurity

• “Impure” module expression: if button_is_selected() then A else B

• “Pure” module expression: mod type t = int; val x = 3 end

34

PurityPurity

• “Impure” module expression: if button_is_selected() then A else B

• “Pure” module expression: mod type t = int; val x = ref 3 end

35

PurityPurity

• “Impure” module expression: if button_is_selected() then A else B

• “Pure” module expression: mod type t = int; val x = ref 3 end

• Sound for M to be projectible , M is pure (w.r.t. type components)

36

PlanPlan

• Consider how purity and projectibility relate to:– Sealing– Functors– The way that sealing and functors interact

37

SealingSealing

• Suppose M = (mod ... end) :> (iface type t ... end)

module C = M

module D = M

• Sealing has no run-time effect, so M is pure• But if M is projectible, then C.t = M.t = D.t• This violates abstraction!

38

Big PictureBig Picture

if button_is_selected() then A else B

Impure Modules (all non-projectible)

Pure Modules

Projectible Non-projectible

A mod

type t = int val x = ref 3

end

B

M :> I

39

FunctorsFunctors

• To track purity in the presence of functors:– Need to know whether applying a functor will

unleash an effect or not

• Distinguish types of total and partial functors:

– F : I1 ! I2 , body of F is pure

– F : I1 ! I2 , body of F is impure

tot

par

40

Total Total ,, Applicative Applicative

• F : I1 ! I2, M : 1, F and M are pure

module C = F(M)

module D = F(M)

• F(M) is pure ) projectible

C.t = F(M).t = D.t

tot

41

Partial Partial ,, Generative Generative

• F : I1 ! I2, M : 1, F and M are pure

module C = F(M)

module D = F(M)

• F(M) is impure ) non-projectible

C.t D.t

par

42

Set Functor ExampleSet Functor Example

module Set = functor (Item : COMPARABLE) ->

(mod ... end :>

iface

type set

fun member : Item.item * set -> set

...

end)

• Body is pure, so Set functor is total

43

Set Functor ExampleSet Functor Example

module Set = functor (Item : COMPARABLE) ->

(mod ... end :>

iface

type set

fun member : Item.item * set -> set

...

end)

module IntSet1 = Set(IntItem)

module IntSet2 = Set(IntItem)

• IntSet1.set and IntSet2.set are compatible

44

SymbolTable ModuleSymbolTable Module

module SymbolTable =

mod ... end :>

iface

type symbol

fun string_to_symbol : string -> symbol

fun symbol_to_string : symbol -> string

end

45

SymbolTable FunctorSymbolTable Functor

module SymbolTable = functor () ->

(mod ... end :>

iface

type symbol

fun string_to_symbol : string -> symbol

fun symbol_to_string : symbol -> string

end)

• Body is pure, so SymbolTable functor is total

46

SymbolTable FunctorSymbolTable Functor

module SymbolTable = functor () -> (mod ... end :> iface type symbol fun string_to_symbol : string -> symbol fun symbol_to_string : symbol -> string end)

module ST1 = SymbolTable()module ST2 = SymbolTable()

• But ST1.symbol and ST2.symbol are not compatible!

47

Pure vs. Impure SealingPure vs. Impure Sealing

if button_is_selected() then A else B

Impure Modules (all non-projectible)

Pure Modules

Projectible Non-projectible

A mod

type t = int val x = ref 3

end

B

M :> I

M :>> I Two forms

of sealing

48

SymbolTable Functor RevisitedSymbolTable Functor Revisited

module SymbolTable = functor () ->

(mod ... end :>>

iface

type symbol

fun string_to_symbol : string -> symbol

fun symbol_to_string : symbol -> string

end)

• Body is impure, so SymbolTable functor is partial

49

SummarySummary

• Analysis in terms of purity and projectibility

• Previous systems make “applicative” or “generative” a design choice, but we support both

• Previous systems just employ one or the other form of sealing and call it “sealing”

50

Unifying Previous SystemsUnifying Previous Systems

• Standard ML ‘97– Only has impure sealing, all functors are partial/generative

• Objective Caml / Leroy (1995)– Only has pure sealing, all functors are total/applicative

• Shao (1999)– Supports both total and partial functors

– Only has impure sealing, can’t write applicative Set functor

• Russo (2000)– Two languages, one like SML and one like O’Caml

– Moscow ML combines them, but language is unsound

51

OverviewOverview

• Concrete examples of data abstraction in ML• High-level analysis of data abstraction in ML• Extending ML with recursive modules• Future work

52

Recursive ModulesRecursive Modules

• Existing proposals fall into two categories:– “Units” or “Mixin modules” meant to replace ML modules

(Flatt-Felleisen 98, Ancona-Zucca 96, Duggan-Sourelis 98)

– Extend ML module system with recursive module construct(Crary et al. 99, Russo 01)

• Issues provoked by recursive module construct:– Recursion over general expressions with effects (see

[Dreyer 04] for details)

– Interaction of recursion and data abstraction

53

Recursive Module ExampleRecursive Module Example

rec (X : iface

module A : IA

module B : IB

end.

mod

module A = MA :> IA

module B = MB :> IB

end

)

54

Abstract InterfaceAbstract Interface

iface

module A : iface

type t

...

end

module B : iface

type u

fun g : A.t -> u

end

end

X :

55

Recursive Module BodyRecursive Module Body

mod

module A = mod

type t = int

fun f(x) = ... X.B.g(3) ...

end :> IA

module B = ...

end

56

The Two The Two A.tA.t’s’s

mod

module A = mod

type t = int

fun f(x) = ... X.B.g(3) ...

end :> IA

module B = ...

end

But X.B.g’s argument type is X.A.t,which is an abstract type!

57

Moscow ML SolutionMoscow ML Solution

iface

module A : iface

type t = int

...

end

module B : iface

type u

fun g : A.t -> u

end

end

X :

58

Our SolutionOur Solution

mod

module A = mod

type t = int

fun f(x) = ... X.B.g(3) ...

end :> IA

module B = ...

end

At this point, we know that A.t = int,so we know that X.A.t = int as well.

59

Our SolutionOur Solution

mod

module A = mod

type t = int

fun f(x) = ... X.B.g(3) ...

end :> IA

module B = mod ... end :> IB

end

At this point, A.t is abstract, so all we know is that X.A.t = A.t.

60

Current and Future WorkCurrent and Future Work

• Designed an ML dialect based on our analysis, following [Harper-Stone 97]– Currently implementing it in TILT compiler for SML

• Future Work:– Further extensions to ML module system

• E.g. type classes

– Generalizations of ML-style data abstraction

Thank you!Thank you!