from monadic second-order definable string transformations to transducers

From Monadic Second-Order Definable StringTransformations to Transducers

Rajeev Alur1 Antoine Durand-Gasselin2 Ashutosh Trivedi3

1University of Pennsylvania

2LIAFA, Universite Paris Diderot

3Indian Institute of Technology Bombay

June 21, 2013

Alur, Durand-Gasselin, Trivedi Streaming String Transducers June 21, 2013 1 / 31

Regular Word Analysis

Qualitative properties over wordsϕ : Σ∞ → {0, 1}

MSO formulas Finite state automata

Logically Computational model

Regular Word Transformation

Transformations of wordsϕ : Σ∞ → Σ∞

Definition using MSO Streaming Transducers

Equivalence [Buchi, 1960]

Equi-expressiveness

Monadic Second Order Logic (MSO)We deal about MSO over the linear order:

The structure is( N , > , P1, . . . ,Pk )

The domain: N or [1, n] The order relation Some unary predicates

Words are interpreted structures: e.g. ([1, 10], >,Pa,Pb,Pc)

w = a b b a b c a b c c

Pa = { 1, 4, 7}Pb = { 2, 3, 5, 8}Pc = { 6, 9, 10}

Formulas are defined inductively:I Atomic: x1 < x2, P(x1), X (x), ...I Boolean connectives: ϕ1 ∧ ϕ2, ¬ϕ3, ...I First-order quantification: ∃x .ϕI Second-order quantification: ∃X .ϕ

An MSO formula with no free variables defines a language

Pa = { 1, 4, 7}Pb = { 2, 3, 5, 8}Pc = { 6, 9, 10}

Appealing computational model: Finite State Automata

Theorem [Buchi, 1960]

A language is MSO definable iff it is accepted by a finite-state automaton.

Deterministic automata are a computational model to analyse words:process sequentially a word input by jumping from state to state

Can be efficiently manipulatedI Automata can be determinizedI LSPACE algorithm to check if a word is accepted by an automatonI Minimization (equivalence in time O(n log log n))I Product of automata (language union, intersection,...)

Equi-expressiveness

Contents

1 Regular TransformationsLogical definitionStreaming Transducers

2 Contribution: Equivalence with a direct logic-based reductionSome logical considerationsProof Walkthrough

3 Decision proceduresFunctional equivalenceTypechecking Problem

4 Conclusion

Contents

4 Conclusion

MSO-definable Transformations

[Courcelle, 1994] Defining Graph Transformations using MSO

A labeled graph transformation using MSO is specified by:

input and output alphabets;

an MSO formula specifying the domain of the transformation;

output is specified using a finite number of copies of nodes of inputgraph;

the node labels are specified using MSO formulas; and

the existence of edges between nodes of various copies is specifiedusing MSO formulas

Example

Let Σ = {a, b,#}. Consider a transformation f1 : Σ∞ → Σ∞

u1#u2# . . . un−1#un#v 7→ u1u1# . . .#unun#v .

where u is reverse of u.

[Courcelle, 1994] Defining Graph Transformations using MSO

A labeled graph transformation using MSO is specified by:

input and output alphabets;

an MSO formula specifying the domain of the transformation;

output is specified using a finite number of copies of nodes of inputgraph;

the node labels are specified using MSO formulas; and

the existence of edges between nodes of various copies is specifiedusing MSO formulas

Example

Let Σ = {a, b,#}. Consider a transformation f1 : Σ∞ → Σ∞

u1#u2# . . . un−1#un#v 7→ u1u1# . . .#unun#v .

where u is reverse of u.Alur, Durand-Gasselin, Trivedi Streaming String Transducers June 21, 2013 8 / 31

input: a b c b b # a a b # c c a a # b

copy 1: a b c b b a a b c c a a

copy 2: a b c b b # a a b # c c a a # b

Σ = Γ = {a, b, c ,#}, C = {1, 2}, and

Node Label Formulas (|Γ|.|C | formulas)

I Labelc1α (x) = Labelinpα (x) ∧ ¬Labelinp# (x) ∧ reach#(x)

I Labelc2α (x) = Labelinpα (x)

Edge Label Formulas (|C |2 formulas)

I Edgec1,c1(x , y) = Edgeinp(y , x) ∧ ¬Labelinp# (x) ∧ ¬Labelinp# (y).I Edgec2,c2(x , y) =

Edgeinp(x , y) ∧ (¬Labelinp# (x)∨(Labelinp# (x)∧¬reach#(x)))

I Edge1,2(x , y)=(x=y)∧(first(x)∨∃z(Labelinp# (z)∧Edgeinp(z , x)))

I Edge2,1(x , y)=Labelinp# (x) ∧ reach#(x) ∧ (∃z(Edgeinp(y , z) ∧Labelinp# (z))) ∧ (∀z((path(x , z) ∧ path(z , y))→ ¬Labelinp# (z)))

Σ = Γ = {a, b, c ,#}, C = {1, 2}, and

Streaming Transducers [Alur and Cerny, 2011]

A streaming transducer is an automaton:

( Σ,Q, δ, q0 , F , X , (D, f1, . . . , fk) , ρ )

extended with a finite set of registers X

which will store values from domain D

with an update function ρ : Q × Σ→ X → T (X , f1, . . . , fk)

T (X , f1, . . . , fk) denotes terms obtained with functions f1, . . . , fk andregisters.

Its configurations will be a state together with a valuation of eachregister Q × [X → D]

An output function F :I Finite word input this is a function from Q to T (X , f1, . . . , fk).

The image of w is the value of the term F(δ(w))I in the case of infinite word input, this is a function from 2Q to X .

The image of w is the limit of the value of register F(δ(w))

( Σ,Q, δ, q0 , F , X , (D, f1, . . . , fk) , ρ )

An example

A = ( Σ,Q, δ, q0 , F , X , (D, f1, . . . , fk) , ρ )

Here D = Σ∗ , and we have binary function · (concatenation) and

constants ε, a, b, c,#

qstart

∣∣∣∣∣∣X := X · Z ·#Y := εZ := ε

∣∣∣∣∣∣X := XY := Y · αZ := α · Z · α

F(q) = X · Y

a b # a # a b

X ε ε baab# baab# baab#aa# baab#aa# baab#aa#

Y a ab ε a ε a ab

Z aa baab ε aa ε aa baab

An example

A = ( Σ,Q, δ, q0 , F , X , (D, f1, . . . , fk) , ρ )

qstart

∣∣∣∣∣∣X := X · Z ·#Y := εZ := ε

∣∣∣∣∣∣X := XY := Y · αZ := α · Z · α

F(q) = X · Y

a b # a # a b

Y a ab ε a ε a ab

An example

A = ( Σ,Q, δ, q0 , F , X , (D, f1, . . . , fk) , ρ )

qstart

∣∣∣∣∣∣X := X · Z ·#Y := εZ := ε

∣∣∣∣∣∣X := XY := Y · αZ := α · Z · α

F(q) = X · Y

a b # a # a b

Y a ab ε a ε a ab

An example

A = ( Σ,Q, δ, q0 , F , X , (D, f1, . . . , fk) , ρ )

qstart

∣∣∣∣∣∣X := X · Z ·#Y := εZ := ε

∣∣∣∣∣∣X := XY := Y · αZ := α · Z · α

F(q) = X · Y

a b # a # a b

Y a ab ε a ε a ab

An example

A = ( Σ,Q, δ, q0 , F , X , (D, f1, . . . , fk) , ρ )

qstart

∣∣∣∣∣∣X := X · Z ·#Y := εZ := ε

∣∣∣∣∣∣X := XY := Y · αZ := α · Z · α

F(q) = X · Y

a b # a # a b

Y a ab ε a ε a ab

An example

A = ( Σ,Q, δ, q0 , F , X , (D, f1, . . . , fk) , ρ )

qstart

∣∣∣∣∣∣X := X · Z ·#Y := εZ := ε

∣∣∣∣∣∣X := XY := Y · αZ := α · Z · α

F(q) = X · Y

a b # a # a b

Y a ab ε a ε a ab

An example

A = ( Σ,Q, δ, q0 , F , X , (D, f1, . . . , fk) , ρ )

qstart

∣∣∣∣∣∣X := X · Z ·#Y := εZ := ε

∣∣∣∣∣∣X := XY := Y · αZ := α · Z · α

F(q) = X · Y

a b # a # a b

Y a ab ε a ε a ab

An example

A = ( Σ,Q, δ, q0 , F , X , (D, f1, . . . , fk) , ρ )

qstart

∣∣∣∣∣∣X := X · Z ·#Y := εZ := ε

∣∣∣∣∣∣X := XY := Y · αZ := α · Z · α

F(q) = X · Y

a b # a # a b

Y a ab ε a ε a ab

An example

A = ( Σ,Q, δ, q0 , F , X , (D, f1, . . . , fk) , ρ )

qstart

∣∣∣∣∣∣X := X · Z ·#Y := εZ := ε

∣∣∣∣∣∣X := XY := Y · αZ := α · Z · α

F(q) = X · Y

a b # a # a b

Y a ab ε a ε a ab

A restriction on the update function

qstart α|X = X · α · X F = X

This leads to an exponential output

We want to forbid this behaviour:I Copylessness: each register appear at most once on the r.h.s.I Restricted copy: copies allowed but recombining is not possibleI Bounded copy: in the end the content of any register is never copies

more than a bounded number of times

Streaming transducers have to satisfy this syntactic restriction

Contents

4 Conclusion

A computational modelfor some restricted Courcelle transformations

to {0, 1} Σ∗ Σω T ∗ T ω graphsfrom

finite words

infinite words

finite trees

infinite trees

graphs

[Alur and Cerny, 2011] (POPL) Streaming transducers for algorithmic verification of single-pass list-processing programs

[Alur et al., 2012] (LICS) Regular Transformations of Infinite Strings

[Alur and D’Antoni, 2012] (ICALP) Streaming Tree Transducers

[Alur et al., 2013] (LICS) From Monadic Second-Order Definable String Transformations to Transducers

finite words

infinite words

finite trees

infinite trees

graphs

finite words

infinite words

finite trees

infinite trees

graphs

[AC11]

Existing proof, through a two way transducerCase of transformations from finite strings to finite strings [Alur and Cerny, 2011]

MSOTransformation

Two-Way transducerw/ look-ahead

Two-Way TransducerStreamingTransducer

[Engelfriet and Hoogeboom, 2001]

[Chytil and Jakl, 1977]

[Alur and Cerny, 2011]

finite words

infinite words

finite trees

infinite trees

graphs

[AC11]

[AFT12]

Existing proof, of a through way transducerCase of transformations from infinite strings to infinite strings[Alur et al., 2012]

MSOTransformation

Two-Way transducerw/ look-ahead

Functional NSSTw/ look-ahead

Functional NSSTStreaming Transducerw/ bounded copy

StreamingTransducer

[Engelfriet and Hoogeboom, 2001][Alur et al., 2012]

[Miyano and Hayashi, 1984]

[Alur et al., 2012][Alur et al., 2012]

[Alur and Cerny, 2011]

finite words

infinite words

finite trees

infinite trees

graphs

[AC11]

[AFT12]

[Ad’A12] [Ad’A12]

[Ad’A12]

finite words

infinite words

finite trees

infinite trees

graphs

[AC11]

[AFT12]

[Ad’A12] [Ad’A12]

[Ad’A12]

finite words

infinite words

finite trees

infinite trees

graphs

[AC11]

[AFT12]

[Ad’A12] [Ad’A12]

[Ad’A12]

[ADT13]

A direct proof

MSOTransformation

Streaming Transducerw/ restricted copy

[Alur, DG and Trivedi 13]

[Alur and Cerny, 2011][Alur et al., 2012]

Finiteness of MSO formula up to some quantifier depth

Remark

The number of MSO sentences of quantifier depth at most k is finite

We define an equivalence relation over wordsI Two words are k-equivalent iff no formula of q.d. k can distinguish themI This equivalence relation has finite indexI We denote k-types these equivalence classes

Remark: This equivalence relation is a monoıd congruence

The k-type of u · v is determined by the k-types of u and v

Remark

Formulas with quantifier depth k and 2 first-order free variables:

ϕ(x , y) w : x yw1 w2 w3

The validity of ϕ only depends on w [x ],w [y ] and the k-types of w1,w2,w3

Remark

The number of MSO sentences of quantifier depth at most k is finiteBy induction over formulas with r FV and quantifier depth at most k :

I true when k = 0I if true for some k , notice that an MSO formula with qd k + 1 and r FV

is a boolean combination of formulas of the form ∃X .ϕ where ϕ has qdk and r + 1 FV.

I Thus a finitely generated (by induction) boolean algebra

Remark

ϕ(x , y) w : x yw1 w2 w3

Remark

ϕ(x , y) w : x yw1 w2 w3

Remark

ϕ(x , y) w : x yw1 w2 w3

Remark

ϕ(x , y) w : x yw1 w2 w3

Remark

ϕ(x , y) w : x yw1 w2 w3

Crossings

• • •

• ••

• •

• • •

A crossing at position x is an edge which connects two nodes which are noton the same side w.r.t. x .

Boundedly many crossings

1 yb ya x y ′a y ′b

w : α α • • • · · ·

c1 ◦ · · · ◦ · · · ◦ · · · ◦ · · · ◦ · · · ◦ · · ·

c2 ◦ · · · ◦ · · · ◦ · · · ◦ · · · ◦ · · · ◦ · · ·

......

◦◦

c ◦ · · · ◦ · · · ◦ · · · ◦ · · · ◦ · · · ◦ · · ·

· · ·· · ·· · ·

u vu′ v ′

w w ′inp

Theorem

At any given position there are at most 2C |k-types| crossingsOtherwise in the image, two distinct nodes have an outgoing edge to the same node

Boundedly many crossings means boundedly many registers

• • •

••

• •

••

• • •

One register for each triple k-type, letter, k-type would be enough

We will also need to handle all possible behaviours

Boundedly many crossings means boundedly many registers

• • •

••

• •

••

• • •

? ? ? ? ? ? ? ? ? ?

One register for each triple k-type, letter, k-type would be enough

We will also need to handle all possible behaviours

Handling all possible behaviours with Regular Look-Ahead

Regular look-ahead: guards on transitions (and updates)MSO queries over the suffix.

w : y x ? ? ? ? ? ? ? ? ? ?

Is there in the image some subword that starts at a position y (labeledby α), before x such that the k-type of w [0:y) is τ1 and the k-type ofw(y :x) is τ2 ?

This is an MSO query with quantifier depth K = k + |C |+ 3 !

Thus all the possible cases are handled by “guessing” the K -type ofw(x :|w |]The set of registers will be k-types ×Σ× k-types ×K -types

The set of states will be the set of K -types (the state will state whichis the K -type of the prefix read so far).

w : y x ? ? ? ? ? ? ? ? ? ?

Thus all the possible cases are handled by “guessing” the K -type ofw(x :|w |]

The set of registers will be k-types ×Σ× k-types ×K -types

w : y x ? ? ? ? ? ? ? ? ? ?

Finite word case

At the end of the input, we output the non-empty registercorresponding to the regular-look ahead ε.

The reduction does not go through a two-way model.

Finite word case

At the end of the input, we output the non-empty registercorresponding to the regular-look ahead ε.

The reduction does not go through a two-way model.

Infinite word input case

The main difficulty lies in that we have to guess correctly and infinitelyoften the regular look-ahead.

The image is defined as the limit of the content of some register,depending on the set of infinitely occuring states (Muller condition)

With the Muller output condition, we can have some MSO propertyover the whole word.

We need to effectively find a factorization τ(τ ′)ω of the input. Thiscan be found in Shelah’s alternative proof of Buchi Theorem, using afinite additive coloring (Ramsey’s Theorem)

Thus we can output infinitely often some increasing prefixes of theimage

We converge toward the output

Contents

4 Conclusion

Functional equivalence is decidableDo two transformations have the same image on any input ?

A hard problem: different logical ways to define the same transformation

w : a0 a1 a2 . . . an−1 ano1 : a1 a2 . . . an−1 ano2 : a1 a2 a3 . . . an

Reduction to reachability in a counter system (no test, no decrement):I Idea: finding a conflicting position (say a in first image, b in the second)I Two counters tracking the number of letters before the conflicting

postion in each imageI Set of states: (states of the transducer ×4registers of the transducer)2

F 0: the value of this register does not appear in the outputF 1: its value appears before the conflicting positionF 2: its value contains the conflicting positionF 3: its value is after the conflicting position

I Erase the letters in the transitions, increment corresponding to theregisters updates

I Find a reachable configuration where the two counters are equal

The typechecking problem

Definition

Given formulas ϕ,ψ do we have ∀w .w � ϕ =⇒ T (w) � ψ

We can perform this check by some automatic construction

Contents

4 Conclusion

Conclusion

Contributions

Direct proof of equivalence between Courcelle transformations andStreaming Transducers

Previously unexplored setting of ω-words to trees

Syntactically ensuring convergence of the output

Equivalence and type-checking problems are decidable

Perspectives

More expressive transformations (relaxing the restriction on copies)

Less expressive transformations (First-order fragment)

Extension to regular cost functions

Conclusion

Contributions

Direct proof of equivalence between Courcelle transformations andStreaming Transducers

Previously unexplored setting of ω-words to trees

Syntactically ensuring convergence of the output

Equivalence and type-checking problems are decidable

Perspectives

More expressive transformations (relaxing the restriction on copies)

Less expressive transformations (First-order fragment)

Extension to regular cost functions

Thank you for your attention !

Alur, R. and Cerny, P. (2011).

Streaming transducers for algorithmic verification of single-pass list-processingprograms.

In POPL, pages 599–610.

Alur, R. and D’Antoni, L. (2012).

Streaming tree transducers.

In ICALP (2), volume 7392 of Lecture Notes in Computer Science, pages42–53. Springer.

Alur, R., Durand-Gasselin, A., and Trivedi, A. (2013).

From monadic second-order definable string transformations to transducers.

In LICS. IEEE.

Alur, R., Filiot, E., and Trivedi, A. (2012).

Regular transformations of infinite strings.

In LICS, pages 65–74.

Buchi, J. R. (1960).

Weak second-order Arithmetic and Finite Automata.

Zeitschrift fur Mathematische Logik und Grundlagen der Mathematik, 6:66–92.

Chytil, M. and Jakl, V. (1977).

Serial composition of 2-way finite-state transducers and simple programs onstrings.

In ICALP, volume 52 of Lecture Notes in Computer Science, pages 135–147.Springer.

Courcelle, B. (1994).

Monadic second-order definable graph transductions: a survey.

Theoretical Computer Science, 126(1):53–75.

Engelfriet, J. and Hoogeboom, H. J. (2001).

MSO definable string transductions and two-way finite-state transducers.

ACM Trans. Comput. Logic, 2:216–254.

Miyano, S. and Hayashi, T. (1984).

Alternating finite automata on omega-words.

Theor. Comput. Sci., 32:321–330.

from monadic second-order definable string transformations to transducers

Documents

modular monadic semantics - yale...

monadic constraint programming - university of...

monadic programming

software components as monadic, weighted mealy...

list-based monadic computations for dynamic languages

origami, a monadic fold library for scala

introduction to transducers, introduction to transducers ......

exploiting user-definable synchronizations in graph...

monadic parser combinators - school of computer...

monadic view slides-toronto

saharon shelah- on nicely definable forcing notions

chapter xiii monadic second-order theories

on monadic domination in leibniz's metaphysics

introduction to transducers, introduction to transducers...

monadic framework for delimited continuations

testing static code analyses with monadic definitional

what is success? is success definable?

vertex-minors, monadic second-order logic, and a

unit – 3 transducers€¦ · classification of...

memoizing a monadic mixin dsl - ghent university ›...