computational models - lecture...
TRANSCRIPT
Computational Models - Lecture 3 1
Handout Mode
Iftach Haitner and Yishay Mansour.
Tel Aviv University.
March 13/18, 2013
1Based on frames by Benny Chor, Tel Aviv University, modifying frames by MauriceHerlihy, Brown University.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 1 / 43
Computational Models - Lecture 3
Non Regular Languages: Two Approaches
(1) The Pumping Lemma
(2) Myhill-Nerode Theorem (not in Sipser’s book)
Closure properties
Algorithmic questions for NFAs
Sipser’s book, 1.4, 2.1, 2.2
Hopcroft and Ullman, 3.4
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 2 / 43
Proved Last Time
Theorem 1A language, L, is described by a regular expression, R, if and only if Lis regular.
=⇒ construct an NFA accepting R.
⇐= Given a regular language, L, construct anequivalent regular expression
We have made a lot of progress understanding what finite automatacan do. But what can’t they do?
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 3 / 43
Negative Results
Is there a DFA that accepts
B = {0n1n : n ≥ 0}
C = {w : w has an equal number of 0’s and 1’s}
D = {w : w has an equal number of occurrencesof 01 and 10 substrings}
Consider B:
DFA must “remember” how many 0’s it has seen
impossible with finite state.
The others are exactly the same...
Question: Is this a proof?Answer: No, D is regular!???
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 4 / 43
Part I
Pumping Lemma
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 5 / 43
Pumping Lemma
We will show that all regular languages have a special property.
Suppose L is regular.
If a string in L is longer than a certain critical length ` (thepumping length),
then it can be “pumped” to a longer string by repeating an internalsubstring any number of times.
The longer string must be in L too.
This is a powerful technique for showing that a language is notregular.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 6 / 43
Pumping Lemma
Lemma 2If L is a regular language, then there is an ` > 0 (the pumping length),where if s is any string in L of length |s| ≥ `, then s may be divided intothree pieces s = xyz such that
1 for every i ≥ 0, xy iz ∈ L,2 |y | > 0, and3 |xy | ≤ `.
Remarks: Without the second condition, the theorem would be trivial.The third condition is technical and sometimes useful.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 7 / 43
Pumping Lemma – Proof
Let M = (Q,Σ, δ, q1,F ) be a DFA that accepts L.
Let ` be |Q|, the number of states of M.
If s ∈ L has length at least `, consider the sequence of states M goesthrough as it reads s:
s1 s2 s3 s4 s5 s6 . . . sn↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑
q1 q20 q9 q17 q12 q13 q9 q2 q5 ∈ F
Since the sequence of states is of length |s|+ 1 > `,and there are only ` different states in Q,at least one state is repeated (by the pigeonhole principle).
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 8 / 43
Pumping Lemma – Proof (cont.)
Write down s = xyz
q1
q9
q5x
y
z
By inspection, M accepts xykz for every k ≥ 0.
|y | > 0 because the state (q9 in figure) is repeated.
To ensure that |xy | ≤ `, pick first state repetition, which must occurno later than `+ 1 states in sequence.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 9 / 43
Application # 1
Corollary 3
The language B = {0n1n : n > 0} is not regular.
Proof: By contradiction. Suppose B is regular. Let 2` be the pumpinglength.
Consider the string s = 0`1`.
By pumping lemma s = xyz, where xykz ∈ B for every k ≥ 0.
If y is all 0, then xykz has too many 0’s, for k ≥ 2.
If y is all 1, then xykz has too many 1’s, for k ≥ 2.
If y is mixed, then xykz is not of right form. ♣
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 10 / 43
Using the Pumping Lemma
How to prove that a language is not regular
Given a language L
An adversary sets the parameter `.
Select a word s ∈ L.
The word s would (usually) depend on `.
The adversary selects a partition s = xyz, such that |y | ≥ 1 and|xy | ≤ `.
Show an index k , such that xykz 6∈ L.
Need to prove for any parameter ` and any partition xyz.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 11 / 43
Application # 2
Corollary 4
The language C = {w : #1(w) = #0(w)}is not regular.
Proof: By contradiction. Suppose C is regular. Let ` be the pumpinglength.
Consider the string s = 0`1`.
By pumping lemma s = xyz, where xykz ∈ C for every k ≥ 0.
Since |xy | ≤ ` then y is all 0, and xykz has too many 0’s. ♣
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 12 / 43
Reflections
What about using (01)` ∈ C?
What about D ={w : w has an equal number of occurrences of 01 and 10 substrings}?
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 13 / 43
Application # 3
Corollary 5
The language E = {0i1j : i > j} is not regular.
Proof: By contradiction. Suppose E is regular. Let ` be its pumpinglength.
Consider the string s = 0`1`−1.
By pumping lemma s = xyz, where xykz ∈ E for every k ≥ 0,|y | > 0 and |xy | ≤ `
But for k = 0 we have xz /∈ E , contradiction. ♣
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 14 / 43
Application # 4
Corollary 6
The language Primes ⊂ {1}∗, which contains all strings whose lengthis a prime number, is not regular.
Proof: By contradiction. Suppose Primes is regular, accepted by DFAM. Let ` be the pumping length.
Let s = 1p ∈ Primes, where p ≥ ` is a prime.
By pumping lemma s = xyz, where xykz ∈ Primes for every k .
For k = p + 1 we have xykz = 1p+mp, where |y | = m.
since p(m + 1) is not prime, we have a contradiction. ♣
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 15 / 43
Another Example
Consider the language L = {aibncn : n ≥ 0, i ≥ 1} ∪ {bncm : n,m ≥ 0},
For any word s ∈ L we can apply the pumping lemma:
If s = aibncn, then set x = ε and y = a.
If s = bncm, then set x = ε and y = b.
If s = cm, then set x = ε and y = c.
Is L regular?!
How can we prove it?!
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 16 / 43
Part II
Characterization of Regular Languages
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 17 / 43
The Equivalence Relation ∼L
Let L ⊆ Σ∗ be a language.
Define an equivalence relation ∼L on pairs of strings:
Let x , y ∈ Σ∗. We say that x∼L y if for every string z ∈ Σ∗, xz ∈ L ifand only if yz ∈ L.
It is easy to see that ∼L is indeed an equivalence relation (reflexive,symmetric, transitive) on Σ∗.
In addition, if x∼L y then for every string z ∈ Σ∗, xz∼L yz as well (thisis called right invariance).
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 18 / 43
The Equivalence Relation ∼L cont.
Like every equivalence relation, ∼L partitions Σ∗ to (disjoint)equivalence classes. For every string x , let [x ] ⊆ Σ∗ denote itsequivalence class w.r.t. ∼L (if x ∼L y then [x ] = [y ] – equality of sets).
Question is, how many equivalence classes does ∼L induce?
∼L finite or infinite?
Well, it could be either finite or infinite. This depends on the languageL.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 19 / 43
Three Examples
Let L1 ⊂ {0, 1}∗ contain all strings where the number of 1s isdivisible by 4. Then ∼L1 has finitely many equivalence classes.
Let L2 ⊂ {0, 1}∗ contain all strings of the form 0n1n. Then ∼L2 hasinfinitely many equivalence classes.
Let L3 = {aibncn : n ≥ 0, i ≥ 1} ∪ {bncm : n,m ≥ 0}. Then ∼L3
has infinitely many equivalence classes.
(Proof on the board)
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 20 / 43
Myhill-Nerode Theorem
Theorem 7Let L ⊆ Σ∗ be a language. Then L is regular⇐⇒ ∼L has finitely manyequivalence classes.
Three specific consequences:
L1 ⊂ {0, 1}∗ contains all strings where the number of 1s isdivisible by 4. Then L1 is regular.
L2 ⊂ {0, 1}∗ contains all strings of the form 0n1n. Then L2 is notregular.
Let L3 = {aibncn : n ≥ 0, i ≥ 1} ∪ {bncm : n,m ≥ 0}. Then L3 isnot regular.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 21 / 43
Myhill-Nerode Theorem: Proof
=⇒
Suppose L is regular. Let M = (Q,Σ, δ, q0,F ) be a DFA accepting it.The relation ∼M on pairs of strings is defined as follows: x ∼M y if
δ(q0, x) = δ(q0, y).Clearly, ∼M is an equivalence relation.
Furthermore, if x ∼M y , then xz ∼M yz for every z ∈ Σ∗.Therefore, xz ∈ L if and only if yz ∈ L.
This means that x ∼M y =⇒ x ∼L y (i.e., ∼M is a refinement of ∼L ).
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 22 / 43
Myhill-Nerode Theorem: Proof cont.
=⇒
The equivalence relation ∼M has finitely many equivalenceclasses (at most the number of states in M).
We saw that x ∼M y =⇒ x ∼L y , so the number of equivalenceclasses of ∼L is less or equal than the number of equivalenceclasses of ∼M .
Therefore, ∼L has finitely many equivalence classes. ♠
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 23 / 43
Myhill-Nerode Theorem: Proof cont.
⇐=
Suppose ∼L has finitely many equivalence classes. We’llconstruct a DFA M = (Q,Σ, δ, q0,F ) that accepts L.
Let x1, . . . , xn ∈ Σ∗ be representatives for the finitely manyequivalence classes of ∼L.
Q = {[x1], . . . , [xn]}.
δ([xi ], a) = [xia] for all a ∈ Σ.
q0 = [ε].
F = {[xi ] : xi ∈ L}.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 24 / 43
Myhill-Nerode Theorem: Proof cont.
⇐=
δ([ε], x) = [x ]
proof: Assume δ([ε], x) = [x ] = [xi ]. By right invarianceδ([ε], xa) = δ([xi ], a) = [xia] = [xa]
Therefore, M accepts x iff x ∈ L
So L is accepted by DFA, hence L is regular. ♠
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 25 / 43
Example
Construct DFA (via the above method) for L1 ⊂ {0, 1}∗ contains allstrings where the number of 1s is divisible by 5.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 26 / 43
Applications of the Proof
Let L be a regular language and let M be a DFA accepting it
The number of equivalence classes of ∼L lowerbounds thenumber of equivalence classes of ∼M , which equals the number ofstates in M.
The equivalence relation ∼M , is a refinement of ∼L (eachequivalence class of ∼L correspond to a union of states).
There is an automata whose number of states equals the numberof equivalence classes of ∼L
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 27 / 43
Minimizing Automata
Input: An automata MOutput: An automata M ′, such that L(M) = L(M ′) and M ′ has aminimal number of states.
Let S1 = F and S2 = Q − F . Set S = {S1,S2}.
While exists equivalence class Si ∈ S, q1, q2 ∈ Si and σ ∈ Σ suchthat,
δ(q1, σ) ∈ Sj1 and δ(q2, σ) ∈ Sj2 , j1 6= j2, then
let Si,1 = {q ∈ Si : δ(q, σ) ∈ Sj1}, j1 6= i .
S = S − Si ∪ Si,1 ∪ (Si − Si,1)
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 28 / 43
Minimizing Automata
Output M ′ = (Q′, δ′, q′
0,F′): where
Q′ = S,
q′
0 = S0 ∈ S such that q0 ∈ S0
F ′ = {S1, . . . ,Sk} ⊂ S, such that Si ⊂ F .
δ′(Si , σ) = Sj if for q ∈ Si then δ(q, σ) ∈ Sj .
Claim 8The algorithm terminates, and outputs a (in fact, the) minimalautomata.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 29 / 43
Example
q1
q2
a b
a
a b
b
a
b
b
a
r1
s
r2
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 30 / 43
Part III
Closure Properties of Regular Languages
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 31 / 43
Simple Closure Properties
Regular languages are closed under complement.
Let M = (Q,Σ, δ, q0,F ) be a DFA that accepts L.
Then M ′ = (Q,Σ, δ, q0,Q − F ) is a DFA that accepts L̄ = Σ∗ \ L.
NFA ?!
Regular languages are closed under intersection.
L1 ∩ L2 = L1 ∪ L2.
Proof with automata ?!
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 32 / 43
Division
L1/L2 = {x : ∃y ∈ L2, xy ∈ L1}
Examples:
L1 = (01 ∪ 1)∗ and L2 = 00, L1/L2 =?
L1/L2 = ∅.
L3 = a∗b∗c∗ and L4 = b, L3/L4 =?.
L3/L4 = a∗b∗.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 33 / 43
Division cont.
Theorem 9Regular languages are closed under division with any language.
Proof:
L1 is a regular language, so it has a DFA M = (Q,Σ, δ, q0,F ).
L2 is an arbitrary language.
For L1/L2 we build M ′ = (Q,Σ, δ, q0,F ′).
F ′ = {q : ∃y ∈ L2, δ(q, y) ∈ F}.
F ′ is well defined, but might be hard to compute – “nonconstructive proof". ♣
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 34 / 43
Homomorphism
Definition 10Homomorphism: an assignment that replaces each letter with a word,i.e., h : ∆→ Σ∗. For a word w = w1, . . . ,wn, thenh(w) = h(w1) · · · h(wn), ad for a language L, thenh(L) = {h(x) : x ∈ L}.
Example: h(1) = aba, h(0) = aah(010) = aa aba aaL1 = (01)∗, h(L1) = (aaaba)∗.
h(0) = a, h(1) = aL2 = {0n1n|n ≥ 0}, h(L2) = {a2n|n ≥ 0}.
Theorem 11Regular languages are closed under homomorphism.
Proof:Using regular expressions.Using Automata
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 35 / 43
Inverse Homomorphism
Definition 12
Inverse homomorphism: h−1(w) = {x : h(x) = w},h−1(L) = {x : h(x) ∈ L}
Example: h(1) = aba, h(0) = aaL2 = (ab ∪ ba)∗a, h−1(L2) = {1}.
Claim 13
h(h−1(L))⊆L⊆h−1(h(L)).
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 36 / 43
Inverse Homomorphism cont.
Theorem 14Regular languages are closed under inverse homomorphism.
Proof idea:
Let M = (Q,Σ, δ, q0,F ) the automata for L, and h : ∆→ Σ∗.
for each letter a ∈ ∆ we advance in M using h(a).
Formally, we define M ′ = (Q,∆, δ′, q0,F ), whereδ′(q, a) = δ(q, h(a)).
Hence, δ′(q,w) = δ(q, h(w))
w ∈ L(M ′)←→ h(w) ∈ L(M) ♣
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 37 / 43
Using Homomorphism
We know that L1 = {0n1n : n ≥ 1} is not regular.
Show that L2 = {anban : n ≥ 1} is not regular
We will prove using homomorphism and inverse homomorphism
h1(a) = a, h1(b) = b, h1(c) = a.
h2(a) = 0, h2(b) = ε, h2(c) = 1.
h2(h−11 (L2) ∩ a∗b∗c∗) = L1
h−11 (L2) = (a ∪ c)kb(a ∪ c)k
h−11 (L2) ∩ a∗b∗c∗ = {anbcn : n ≥ 1}
h2(h−11 (L2) ∩ a∗b∗c∗) = {0n1n : n ≥ 1}
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 38 / 43
Part IV
Algorithmic Questions for NFAs
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 39 / 43
Algorithmic Questions for NFAs
Q.: Given an NFA, N, and a string w , is w ∈ L(N)?
Answer: Construct the DFA equivalent to N and run it on w .
Q.: Is L(N) = ∅?Answer 1: Build a minimal size DFA. (Might be exponential time!)Answer 2: This is a reachability question in graphs: Is there a path inthe states’ graph of N from the start state to some accepting state.There are simple, efficient algorithms for this task.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 40 / 43
More Questions
Q.: Is L(N) = Σ∗?
Answer: Check if L(N) = ∅.
Q.: Given N1 and N2, is L(N1) ⊆ L(N2)?
Answer: Check if L(N2) ∩ L(N1) = ∅.
Q.: Given N1 and N2, is L(N1) = L(N2)?
Answer: Check if L(N1) ⊆ L(N2) and L(N2) ⊆ L(N1).
In the future, we will see that for stronger models of computations,many of these problems cannot be solved by any algorithm.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 41 / 43
Part V
Summary — Regular Languages
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 42 / 43
Summary - Regular Languages
So far we saw
finite automata,
regular languages,
regular expressions,
Myhill-Nerode theorem and pumping lemma for regular languages.
Next class we introduce stronger machines andlanguages with more expressive power:
pushdown automata,
context-free languages,
context-free grammars
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, 2013 43 / 43