type inference - princeton university computer science...language design for type inference we call...

TypeInference

COS326DavidWalker

PrincetonUniversity

slidescopyright2017DavidWalkerpermissiongrantedtoreusetheseslidesfornon-commercialeducaFonalpurposes

MidtermExam

WedOct25,2017InClass(11:00-12:20)

MidtermWeek

Bethereorbesquare!

2

TYPEINFERENCE

3

LanguageDesignforTypeInferenceTheMLlanguageandtypesystemisdesignedtosupportaverystrongformoftypeinference.

LanguageDesignforTypeInferenceTheMLlanguageandtypesystemisdesignedtosupportaverystrongformoftypeinference.MLfindsthistypeformap:

let rec map f l = match l with [ ] -> [ ] | hd::tl -> f hd :: map f tl

map : ('a -> 'b) -> 'a list -> 'b list

LanguageDesignforTypeInferenceTheMLlanguageandtypesystemisdesignedtosupportaverystrongformoftypeinference.MLfindsthistypeformap:whichisreallyanabbreviaFonforthistype:

let rec map f l = match l with [ ] -> [ ] | hd::tl -> f hd :: map f tl


map : forall 'a,'b.('a -> 'b) -> 'a list -> 'b list

LanguageDesignforTypeInferenceWecallthistypetheprincipletype(scheme)formap.AnyotherML-styletypeyoucangivemapisaninstanceofthistype,meaningwecanobtaintheothertypesviasubs3tu3onoftypesforparametersfromtheprincipletype.Eg:

('a -> 'a) -> 'a list -> 'a list


(bool -> int) -> bool list -> int list

('a -> int) -> 'a list -> int list

LanguageDesignforTypeInferencePrincipletypesaregreat:•  thetypeinferenceenginecanmakeabestchoiceforthetypeto

giveanexpression•  theenginedoesn'thavetoguess(andwon'thavetoguesswrong)

ThefactthatprincipletypesexistissurprisinglybriYle.IfyouchangeML'stypesystemaliYlebitineitherdirecFon,itcanfallapart.

LanguageDesignforTypeInferenceSupposewetakeoutpolymorphictypesandneedatypeforid:Thenthecompilermightguessthatidhasone(andonlyone)ofthesetypes:

id : bool -> bool

let id x = x

id : int -> int

LanguageDesignforTypeInferenceSupposewetakeoutpolymorphictypesandneedatypeforid:Thenthecompilermightguessthatidhasone(andonlyone)ofthesetypes:Butlateron,oneofthefollowingcodesnippetswon'ttypecheck:Sowhateverchoiceismade,adifferentonemighthavebeenbeYer.

id true

id : bool -> bool

let id x = x

id : int -> int

id 3

LanguageDesignforTypeInferenceWeshowedthatremovingtypesfromthelanguagecausesafailureofprincipletypes.Doesaddingmoretypesalwaysmaketypeinferenceeasier?

LanguageDesignforTypeInferenceOCamlonlyhasuniversaltypesontheoutside:Considerthisprogram:Itwon'ttypecheckinOCaml.Wemightwanttogiveitthistype:NoFcethattheuniversalquanFfierappearsunderan->.

f : (forall a.a->a) -> bool * int

forall 'a,'b. ('a -> 'b) -> 'a list -> 'b list

let f g = (g true, g 3)

LanguageDesignforTypeInferenceSystemFisalotlikeOCaml,exceptthatitallowsuniversalquanFfiersinanyposiFon.Itcouldtypecheckf.Unfortunately,typeinferenceinSystemFisundecideable..



LanguageDesignforTypeInferenceSystemFisalotlikeOCaml,exceptthatitallowsuniversalquanFfiersinanyposiFon.Itcouldtypecheckf.Unfortunately,typeinferenceinSystemFisundecideable.Developedin1972bylogicianJeanYves-Girardwhowasinterestedintheconsistencyofalogicof2nd-orderarithemeFc.RediscoveredasprogramminglanguagebyJohnReynoldsin1974..



LanguageDesignforTypeInferenceEvenseeminglysmallchangescaneffecttypeinference.Suppose"+"operatedonbothfloatsandints.Whattypeforthis?

let f x = x + x


f : int -> int ?

let f x = x + x

f : float -> float ?


f : int -> int ?

let f x = x + x


f : 'a -> 'a ?

LanguageDesignforTypeInferenceEvenseeminglysmallchangescaneffecttypeinference.Suppose"+"operatedonbothfloatsandints.Whattypeforthis?NotypeinOCaml'stypesystemworks.InHaskell:

f : int -> int ?

let f x = x + x


f : 'a -> 'a ?

f : Num 'a => 'a -> 'a

INFERRINGSIMPLETYPES

20

TypeSchemesAtypeschemecontainstypevariablesthatmaybefilledinduringtypeinference

s::=a|int|bool|s->s

Atermschemeisatermthatcontainstypeschemesratherthanpropertypes.eg,forfuncFons:

fun(x:s)->e

letrecf(x:s):s=e

TheGenericTypeInferenceAlgorithm1)AdddisFnctvariablesinallplacestypeschemesareneeded

22

TheGenericTypeInferenceAlgorithm1)AdddisFnctvariablesinallplacestypeschemesareneeded2)Generateconstraints(equaFonsbetweentypes)thatmustbesaFsfiedinorderforanexpressiontotypecheck

•  NoFcethedifferencebetweenthisandthetypecheckingalgorithmfromlastFme.LastFme,wetriedto:•  eagerlydeducetheconcretetypewhencheckingeveryexpression•  rejectprogramswhentypesdidn'tmatch.eg:

•  ThisFme,we'llcollectupequaFonslike:

23

fe--f'sargumenttypemustequale

a->b=c

TheGenericTypeInferenceAlgorithm1)AdddisFnctvariablesinallplacestypeschemesareneeded2)Generateconstraints(equaFonsbetweentypes)thatmustbesaFsfiedinorderforanexpressiontotypecheck

•  NoFcethedifferencebetweenthisandthetypecheckingalgorithmfromlastFme.LastFme,wetriedto:•  eagerlydeducetheconcretetypewhencheckingeveryexpression•  rejectprogramswhentypesdidn'tmatch.eg:

•  ThisFme,we'llcollectupequaFonslike:

3)SolvetheequaFons,generaFngsubsFtuFonsoftypesforvar's

24

fe--f'sargumenttypemustequale

a->b=c

Example:Inferringtypesformap

let rec map f l = match l with

[] -> [] | hd::tl -> f hd :: map f tl

Step1:Annotate

let rec map (f:a) (l:b) : c = match l with

[] -> [] | hd::tl -> f hd :: map f tl

Step2:GenerateConstraints


[] -> [] | hd::tl -> f hd :: map f tl b = d list

a = d -> f

...

Step2:GenerateConstraints


[] -> [] | hd::tl -> f hd :: map f tl

b = b’ list b = b’’ list

b = b’’’ list

a = a

b = b’’’ list

a = b’’ -> a’

c = c’ list

a’ = c’

d list = c’ list

d list = c

finalconstraints:

Step3:SolveConstraints


[] -> [] | hd::tl -> f hd :: map f tl

b = b’ list b = b’’ list

b = b’’’ list

a = a

b = b’’’ list

a = b’’ -> a’

c = c’ list

a’ = c’

d list = c’ list

d list = c

finalconstraints:[b' -> c'/a] [b' list/b]

[c' list/c]

finalsoluFon:



[] -> [] | hd::tl -> f hd :: map f tl

[b' -> c'/a] [b' list/b]

[c' list/c]

finalsoluFon:

let rec map (f:b' -> c') (l:b' list) : c' list = match l with

[] -> [] | hd::tl -> f hd :: map f tl



[] -> [] | hd::tl -> f hd :: map f tl

let rec map (f:a -> b) (l:a list) : b list = match l with

[] -> [] | hd::tl -> f hd :: map f tl

renamingtypevariables:

Step4:GeneratetypesGeneratetypesfromtypeschemes

–  OpFon1:pickaninstanceofthemostgeneraltypewhenwehavecompletedtypeinferenceontheenFreprogram

•  map:(int->int)->intlist->intlist

–  OpFon2:generatepolymorphictypesforprogrampartsandconFnue(polymorphic)typeinference

•  map:foralla,b,c.(a->b)->alist->blist

TypeInferenceDetailsTypeconstraintsaresetsofequaFonsbetweentypeschemes

–  q::={s11=s12,...,sn1=sn2}

–  eg:{b=b’list,a=b->c}

ConstraintGeneraFonSyntax-directedconstraintgeneraFon

–  ouralgorithmcrawlsoverabstractsyntaxofuntypedexpressionsandgenerates

•  atermscheme•  asetofconstraints

Algorithmdefinedassetofinferencerules:

–  G|--u=>e:t,q

contextannotatedexpressionunannotated

expression

type(scheme)

constraintsthatmustbesolved

gen:ctxt->exp->ann_exp*scheme*constraintsinOCaml:

ConstraintGeneraFonSimplerules:

–  G|--x==>x:s,{}(ifG(x)=s)

–  G|--3==>3:int,{}(sameforotherints)

–  G|--true==>true:bool,{}

–  G|--false==>false:bool,{}

Operators

G |-- u1 ==> e1 : t1, q1 G |-- u2 ==> e2 : t2, q2 ------------------------------------------------------------------------ G |-- u1 + u2 ==> e1 + e2 : int, q1 U q2 U {t1 = int, t2 = int}

G |-- u1 ==> e1 : t1, q1 G |-- u2 ==> e2 : t2, q2 ------------------------------------------------------------------------ G |-- u1 < u2 ==> e1 + e2 : bool, q1 U q2 U {t1 = int, t2 = int}

Ifstatements

G |-- u1 ==> e1 : t1, q1 G |-- u2 ==> e2 : t2, q2 G |-- u3 ==> e3 : t3, q3 ---------------------------------------------------------------- G |-- if u1 then u2 else u3 ==> if e1 then e2 else e3 : a, q1 U q2 U q3 U {t1 = bool, a = t2, a = t3}

FuncFonApplicaFon

G |-- u1 ==> e1 : t1, q1 G |-- u2 ==> e2 : t2, q2 (for a fresh a) ---------------------------------------------------------------- G |-- u1 u2==> e1 e2 : a, q1 U q2 U {t1 = t2 -> a}

FuncFonDeclaraFon

G, x : a |-- u ==> e : t, q (for fresh a) ---------------------------------------------------------------- G |-- fun x -> e ==> fun (x : a) -> e : a -> b, q U {t = b}

FuncFonDeclaraFon

G, f : a -> b, x : a |-- u ==> e : t, q (for fresh a,b) ----------------------------------------------------------------------- G |-- rec f(x) = u ==> rec f (x : a) : b = e : a -> b, q U {t = b}

SolvingConstraints

AsoluFontoasystemoftypeconstraintsisasubs3tu3onS–  afuncFonfromtypevariablestotypes–  assumesubsFtuFonsaredefinedonalltypevariables:

•  S(a)=a(foralmostallvariablesa)•  S(a)=s(forsometypeschemes)

–  dom(S)=setofvariabless.t.S(a)≠a

SolvingConstraints

AsoluFontoasystemoftypeconstraintsisasubs3tu3onS–  afuncFonfromtypevariablestotypes–  assumesubsFtuFonsaredefinedonalltypevariables:

•  S(a)=a(foralmostallvariablesa)•  S(a)=s(forsometypeschemes)

–  dom(S)=setofvariabless.t.S(a)≠a

WecanalsoapplyasubsFtuFonStoafulltypeschemes. apply:[int/a,int->bool/b] to:b->a->b returns:(int->bool)->int->(int->bool)

SubsFtuFons

WecanapplyasubsFtuFonStoafulltypescheme:eg:apply[int/a,int->bool/b]tob->a->breturns:(int->bool)->int->(int->bool)

SubsFtuFons

WhenisasubsFtuFonSasoluFontoasetofconstraints?Constraints:{s1=s2,s3=s4,s5=s6,...}WhenthesubsFtuFonmakesbothsidesofallequaFonsthesame.Eg:

a=b->cc=int->bool

constraints:

SubsFtuFons


a=b->cc=int->bool

b->(int->bool)/aint->bool/cb/b

constraints:

soluFon:

SubsFtuFons


a=b->cc=int->bool


b->(int->bool)=b->(int->bool)int->bool=int->bool

constraints:

soluFon:

constraintswithsoluFonapplied:

SubsFtuFons

WhenisasubsFtuFonSasoluFontoasetofconstraints?Constraints:{s1=s2,s3=s4,s5=s6,...}WhenthesubsFtuFonmakesbothsidesofallequaFonsthesame.AsecondsoluFon

a=b->cc=int->bool


constraints:

soluFon1:

int->(int->bool)/aint->bool/cint/b

soluFon2:

SubsFtuFons

WhenisonesoluFonbeYerthananothertoasetofconstraints?

a=b->cc=int->bool


constraints:

soluFon1:int->(int->bool)/aint->bool/cint/b

soluFon2:

b->(int->bool)

typeb->cwithsoluFonapplied:

int->(int->bool)


SubsFtuFons

SoluFon1is"moregeneral"–thereismoreflex.SoluFon2is"moreconcrete"WeprefersoluFon1.



soluFon2:

b->(int->bool)


int->(int->bool)


SubsFtuFons

SoluFon1is"moregeneral"–thereismoreflex.SoluFon2is"moreconcrete"Wepreferthemoregeneral(lessconcrete)soluFon1.Technically,wepreferTtoSifthereexistsanothersubsFtuFonUandforalltypest,S(t)=U(T(t))



soluFon2:

b->(int->bool)


int->(int->bool)


SubsFtuFons

ThereisalwaysabestsoluFon,whichwecanaprinciplesolu3on.ThebestsoluFonis(atleastas)preferredasanyothersoluFon.



soluFon2:

b->(int->bool)


int->(int->bool)


MostGeneralSoluFons

Sistheprincipal(mostgeneral)soluFonofaconstraintqif–  S|=q(itisasoluFon)–  ifT|=qthenT<=S(itisthemostgeneralone)

Lemma:IfqhasasoluFon,thenithasamostgeneraloneWecareaboutprincipalsoluFonssincetheywillgiveusthemostgeneraltypesforterms

ComposiFonofSubsFtuFonsWewillneedtocomparesubsFtuFons:T<=S.eg:

–  T<=SifTis“morespecific”/"less general"thanS–  Ifthereisa

–  Formally:T<=SifandonlyifT=UoSforsomeU

ComposiFonofSubsFtuFonsComposiFon(UoS)appliesthesubsFtuFonSandthenappliesthesubsFtuFonU:

–  (UoS)(a)=U(S(a))WewillneedtocomparesubsFtuFons

–  T<=SifTis“morespecific”thanS–  T<=SifTis“lessgeneral”thanS–  Formally:T<=SifandonlyifT=UoSforsomeU

ComposiFonofSubsFtuFonsExamples:

–  example1:anysubsFtuFonislessgeneralthantheidenFtysubsFtuFonI:

•  S<=IbecauseS=SoI–  example2:

•  S(a)=int,S(b)=c->c•  T(a)=int,T(b)=int->int•  weconclude:T<=S•  ifT(a)=int,T(b)=int->boolthenTisunrelatedtoS(neithermorenorlessgeneral)

SolvingaConstraint

S|=qifSisasoluFontotheconstraintsq

S(s1) = S(s2) S |= q ----------------------------------- S |= {s1 = s2} U q

---------- S |= { }

anysubsFtuFonisasoluFonfortheemptysetofconstraints

asoluFontoanequaFonisasubsFtuFonthatmakesletandrightsidesequal

ExamplesExample1

–  q={a=int,b=a}–  principalsoluFonS:

ExamplesExample1

–  q={a=int,b=a}–  principalsoluFonS:

•  S(a)=S(b)=int•  S(c)=c(forallcotherthana,b)

ExamplesExample2

–  q={a=int,b=a,b=bool}–  principalsoluFonS:

ExamplesExample2

–  q={a=int,b=a,b=bool}–  principalsoluFonS:

•  doesnotexist(thereisnosoluFontoq)

UnificaFonUnificaFon:AnalgorithmthatprovidestheprincipalsoluFontoasetofconstraints(ifoneexists)

–  UnificaFonsystemaFcallysimplifiesasetofconstraints,yieldingasubsFtuFon

•  StarFngstateofunificaFonprocess:(I,q)•  FinalstateofunificaFonprocess:(S,{})

UnificaFonMachineWecanspecifyunificaFonasatransiFonsystem:Basetypes&simplevariables:

(S,{bool=bool} U q) -> (S, q)

(S,{a=a} U q) -> (S, q)

(S,{int=int} U q) -> (S, q)

(S1, q1) -> (S2, q2)

UnificaFonMachineFuncFons:VariabledefiniFons

(S, {s11 -> s12 = s21 -> s22} U q) -> (S, {s11 = s21, s12 = s22} U q)

(S, {a=s} U q) -> ([a=s] o S, [s/a]q)

(S, {s=a} U q) -> ([a=s] o S, [s/a]q)

when a is not in FreeVars(s)

breaksdownintosmallerconstraints

do S and then do [a=b]

OccursCheck

Recallthisprogram:Itgeneratesthetheconstraints:a->a=aWhatisthesoluFonto{a=a->a}?

funx->xx

OccursCheck

Recallthisprogram:Itgeneratesthetheconstraints:a->a=aWhatisthesoluFonto{a=a->a}?Thereisnone!

the"occurscheck"

(S, {s=a} U q) -> ([a=s] o S, [s/a]q)

"when a is not in FreeVars(s)"

funx->xx

IrreducibleStatesWhenalltheconstraintshavebeenprocessed,wewin!ButsomeFmeswegetstuck,withanequaFonlikethis

–  int=bool–  s1->s2=int–  s1->s2=bool–  a=s(scontainsa)–  orissymmetrictooneoftheabove

Stuckstatesarisewhenconstraintsareunsolvable&theprogramdoesnottypecheck.

(S1, q1) -> (S2, q2) -> (S3, q3) ... -> (Sn, { })

TerminaFonWewantunificaFontoterminate(togiveusatypereconstrucFonalgorithm)Inotherwords,wewanttoshowthatthereisnoinfinitesequenceofstates

–  (S1,q1)->(S2,q2)->...

TerminaFon

Weassociateanorderingwithconstraints–  q<q’ifandonlyif

•  qcontainsfewervariablesthanq’•  qcontainsthesamenumberofvariablesasq’butfewertypeconstructors(ie:feweroccurrencesofint,bool,or“->”)

–  Thisisalexacographicordering•  wecanprove(bycontradicFon)thatthereisnoinfinitedecreasingsequenceofconstraints

TerminaFon

Lemma:Everystepreducesthesizeofq–  Proof:Bycases(ie:inducFon)onthedefiniFonofthereducFonrelaFon.

-------------------------------- (S,{int=int} U q) -> (S, q) ------------------------------------ (S,{bool=bool} U q) -> (S, q)

----------------------------- (S,{a=a} U q) -> (S, q)

---------------------------------------------- (S,{s11 -> s12= s21 -> s22} U q) -> (S, {s11 = s21, s12 = s22} U q)

------------------------ (a not in FV(s)) (S,{a=s} U q) -> ([a=s] o S, [s/a]q)

CompleteSoluFonsAcompletesoluFonfor(S,q)isasubsFtuFonTsuchthat

–  T<=S–  T|=q

ProperFesofSoluFonsLemma1:

–  Everyfinalstate(S,{})hasacompletesoluFon.•  ItisS:

–  S<=S–  S|={}

ProperFesofSoluFonsLemma2

–  NostuckstatehasacompletesoluFon(oranysoluFonatall)•  itisimpossibleforasubsFtuFontomakethenecessaryequaFonsequal

–  int≠bool–  int≠t1->t2–  ...

ProperFesofSoluFonsLemma3

–  If(S,q)->(S’,q’)then•  Tiscompletefor(S,q)iffTiscompletefor(S’,q’)•  proofby?•  intheforwarddirecFon,thisisthepreservaFontheoremfortheunificaFonmachine!

Summary:UnificaFonByterminaFon,(I,q)->*(S,q’)where(S,q’)isirreducible.Moreover:

–  Ifq’={}then•  (S,q’)isfinal(bydefiniFon)•  SisaprincipalsoluFonforq

–  ConsideranyTsuchthatTisasoluFontoq.–  NownoFce,Siscompletefor(S,q’)(bylemma1)–  Siscompletefor(I,q)(bylemma3)–  SinceSiscompletefor(I,q),T<=SandthereforeSisprincipal.

Summary:UnificaFon(cont.)...Moreover:

–  Ifq’isnot{}(and(I,q)->*(S,q’)where(S,q’)isirreducible)then

•  (S,q)isstuck.Consequently,(S,q)hasnocompletesoluFon.Bylemma3,even(I,q)hasnocompletesoluFonandthereforeqhasnosoluFonatall.

MORETYPEINFERENCE

76

letGeneralizaFon

Wheredoweintroducepolymorphicvalues?Andplacex:foralla1,...,an.sinthecontext.

let x = v ==> let x : forall a1,..,an.s = v

if v : s and a1,...,an are the variables of s

letGeneralizaFon

Wheredoweintroducepolymorphicvalues?Andplacex:foralla1,...,an.sinthecontext.Whereandhowdoweuseapolymorphicvalue?

let x = v ==> let x : forall a1,..,an.s = v

G |- x ==> x : s[b1/a1,...,bn/an), {}

if v : s and a1,...,an are the variables of s

when G(x) = forall a1,...,an.s and b1,...,bn are fresh

Whatisthecostoftypeinference?

InpracFce?LinearinthesizeoftheprogramIntheory,DEXPTIME-complete.Why?BecausewecangenerateaprogramthathasatypethatisexponenFallylarge:

let f1 x = x f1 : a -> a let f2 = f1 f1 f2 : (a -> a) -> (a -> a) let f3 = f2 f2 f3 : ((a -> a) -> (a -> a)) ->

((a -> a) -> (a -> a)) let f4 = f3 f3 ...

Summary:TypeInferenceGivenacontextG,anduntypedtermu:

–  Finde,t,qsuchthatG|-u==>e:t,q

–  FindprincipalsoluFonSofqviaunificaFon•  ifnosoluFonexists,thereisnoreconstrucFon

–  ApplyStoe,ieoursoluFonisS(e)•  S(e)containsschemaFctypevariablesa,b,c,etcthatmaybeinstanFatedwithanytype

–  SinceSisprincipal,S(e)characterizesallreconstrucFons.

–  Ifdesired,usethetypecheckingalgorithmtovalidate

type inference - princeton university computer science...language design for type inference we call...

Documents