THEORY OF COMPUTATION

UNIT-1 INTRODUCTION

Overview

This chapter gives the concepts of Finite State Automata, DFA, NFA, Regular languages and Regular Expressions, Applications of finite automata, Pumping Lemma for regular languages. Objective

� To learn about Finite Automata and its types � To learn about Applications of FA. � To know about Regular expressions and languages � To solve problems in FA � To learn about the Pumping Lemma

Finite State Automata:

An automaton is an abstract model of a digital computer. An automaton has a mechanism to read input, which is a string over a given alphabet. This input is actually written on an “input file”, which can be read by the automaton but cannot change it.

Fig: Automation Input file is divided into cells, each of which can hold one symbol. The automaton

has a temporary “storage” device, which has unlimited number of cells, the contents of which can be altered by the automaton. Automaton has a control unit, which is said to be in one of a finite number of “internal states”. The automaton can change state in a defined way.

Types of Automation (a) Deterministic Automata (b) Non-deterministic Automata

A deterministic automata is one in which each move (transition from one state to another) is unequally determined by the current configuration. If the internal state, input and contents of the storage are known, it is possible to predict the future behavior of the automaton. This is said to be deterministic automata otherwise it is nondeterministic automata. An automaton whose output response is “yes” or “No” is called an “Acceptor”.

� FA has a finite set of states and it’s control moves from state to state in response to

external inputs � State is used to remember the relevant portion of the systems history � Due to finite number of states the entire history can’t be remembered � The system can be implemented using fixed set of resources since there are only finite

number of states � One of the state is designated as start state in which the system is placed initially It is

necessary to indicate one or more states as final or accepting states � Entering on of the final state after a sequence of inputs indicates the input sequence is

accepted � The automata is deterministic if it can’t be in more than one state at any one time � If the control can be in several states at once, then the automata is non-deterministic Definition of Deterministic Finite Automation(DFA): A Deterministic Finite Automaton (DFA) is a 5-tuple M = (Q, Σ,δ, q , F ) 0 Where

The input mechanism can move only from left to right and reads exactly one symbol on each step. The transition from one internal state to another are governed by the transition function δ. If δ(q0 , a) q1 , = then if the DFA is in state q0 and the current input symbol is a, the DFA will go into state q1.

DFA as a Recognizer of strings

Acceptance of Strings:

A DFA accepts a string if there is a sequence of states in Q such that

1. is the start state.

2. for all .

3. .

Extended transition function:

Extend (which is function on symbols) to a function on strings, i.e.

.

That is, is the state the automation reaches when it starts from the state q and finish processing the string w. Formally, we can give an inductive definition as follows:

The language of the DFA M is the set of strings that can take the start state to one of the accepting states i.e.

L(M) = { | M accepts w }

= { | }

Example 1 :

is the start state

It is a formal description of a DFA. But it is hard to comprehend. For ex. The language of the DFA is any string over { 0, 1} having at least one 1.

Representation of DFA

FA can be represented in two ways

1. Transition table 2. State transition diagram

Transition Table:

0 1

State transition diagram

Fig: State transition diagram

Here is an informal description how a DFA operates. An input to a DFA can be

any string . Put a pointer to the start state q. Read the input string w from left to right, one symbol at a time, moving the pointer according to the transition function, . If

the next symbol of w is a and the pointer is on state p, move the pointer to . When the end of the input string w is encountered, the pointer is on some state, r. The string is said to be accepted by the DFA if and rejected if . Note that there is no formal mechanism for moving the pointer.

A language is said to be regular if L = L(M) for some DFA M.

Example:1

Example-1 DFA accepting all the strings from { 0, 1 } with a substring 01

Example :2

Strings from {0,1} with even number of 0’s and 1’s

Example :3

All strings whose binary interpretation is divisible by 5

Example 4

All strings that don't contain the substring 110

Example: 5 All strings that contain the substring 0101

Non-deterministic Finite Automata (NFA)

• For each pair of state and input symbol there may be several possible next states • Contradicting from the deterministic finite automaton, where the next possible

state is uniquely determined • Both are equivalent in terms of the language recognition power • Constructing NFA to recognize any language is much easier than constructing a

DFA for that language Formal Definition

Example:

Obtain an NFA for a language consisting of all strings over {0,1} containing a 1 in the third position from the end.

Solution: q1, q2, q3 are initial states

Non determinism is an important abstraction in computer science. Importance of

non determinism is found in the design of algorithms. For examples, there are many problems with efficient nondeterministic solutions but no known efficient deterministic solutions. ( Travelling salesman, Hamiltonean cycle, clique, etc). Behaviour of a process is in a distributed system is also a good example of nondeterministic situation. Because

the behaviour of a process might depend on some messages from other processes that might arrive at arbitrary times with arbitrary contents.

It is easy to construct and comprehend an NFA than DFA for a given regular language. The concept of NFA can also be used in proving many theorems and results. Hence, it plays an important role in this subject.

In the context of FA non determinism can be incorporated naturally. That is, an NFA is defined in the same way as the DFA but with the following two exceptions:

• multiple next state. • - transitions.

Multiple Next State :

In contrast to a DFA, the next state is not necessarily uniquely determined by the current state and input symbol in case of an NFA. (Recall that, in a DFA there is exactly one start state and exactly one transition out of every state for each symbol in ).

This means that - in a state q and with input symbol a - there could be one, more than one

or zero next state to go, i.e. the value of is a subset of Q. Thus =

which means that any one of could be the next state.

The zero next state case is a special one giving = , which means that there is no next state on input symbol when the automata is in state q. In such a case, we may think that the automata "hangs" and the input will be rejected.

- transitions :

In an -transition, the tape head doesn't do anything- it doesnot read and it doesnot move. However, the state of the automata can be changed - that is can go to zero, one or

more states. This is written formally as implying that the next

state could by any one of w/o consuming the next input symbol.

Acceptance :

Informally, an NFA is said to accept its input if it is possible to start in some start state and process , moving according to the transition rules and making choices along the way whenever the next state is not uniquely defined, such that when is completely processed (i.e. end of is reached), the automata is in an accept state. There may be several possible paths through the automation in response to an input since the start state is not determined and there are choices along the way because of multiple next

states. Some of these paths may lead to accpet states while others may not. The automation is said to accept if at least one computation path on input starting from at least one start state leads to an accept state- otherwise, the automation rejects input .

Alternatively, we can say that, is accepted iff there exists a path with label from some start state to some accept state. Since there is no mechanism for determining which state to start in or which of the possible next moves to take (including the -transitions) in response to an input symbol we can think that the automation is having some "guessing" power to chose the correct one in case the input is accepted.

Example 1:

NFA accepting all the strings from { 0, 1 } ending with 01

The Extended Transition function, :

To describe acceptance by an NFA formally, it is necessary to extend the transition

function, denoted as , takes a state and a string , and returns the set of states, S Q, that the NFA is in after processing the string if it starts in state q.

Formally, is defined as follows:

1. that is, without rending any input symbol, an NFA does not change state.

2. Let some and a . Also assume that

. Then .

That is, can be computed by first computing , and by then following any transtive from any of these stats that is labelled a.

Input

State

0 1

q0 {q0, q1} {q0}

q1 Φ {q2}

q2 Φ Φ

Equivalence of NFA to DFA

Definition Two finite accepters M1 and M2 are equivalent iff

L(M1 )= L(M2 ) i.e., if both accept the same language.

Both DFA and NFA recognize the same class of languages. It is important

to note that every NFA has an equivalent DFA.

� Every language L that can be described by some NFA (N) can also be described some DFA (D) such that L(N) = L(D) = L

� The DFA equivalent of an NFA with n states will have 2n states in the worst case and only n states in the best case

� The DFA construction from NFA involves constructing all subsets of the set of states of the NFA – subset construction

NFA with ∈∈∈∈ transitions • NFA with ∈ transition is allowed to make a transition without receiving any input

symbol, ie allows transition on ∈ • ∈ contributes nothing to the string along the path • Does not expand the class of languages that can be accepted by a finite automata

∈∈∈∈-NFA Definition

Example

ECLOSE({q0}) = { q0, q1, q2, q3, q5 } ECLOSE({q1}) = { q1, q3, q5} ECLOSE({q2}) = { q2} Regular languages and Regular Expressions: Regular Languages

The regular languages are those languages that can be constructed from the

“big three” set operations viz., (a) Union (b) Concatenation (c) Kleene star. A regular language is defined as follows. Definition :

Let Σ be an alphabet. The class of “regular languages” over Σ is defined inductively as follows:

Example

Regular Expressions

Regular expressions were designed to represent regular languages with a mathematical tool, a tool built from a set of primitives and operations. This representation

involves a combination of strings of symbols from some alphabet Σ, parentheses and the operators +, ⋅, and *.

A regular expression is obtained from the symbol {a, b, c}, empty string ∈, and empty-set ∅ perform the operations +, ⋅ and * (union, concatenation and Kleene star).

Building Regular Expressions

Any string at all:

To describe any string at all (with S = {a, b, c} you can use (a + b + c)*. Any nonempty string:

This is written any character from S = {a, b, c} followed by any string at all: (a + b + c) (a + b + c)*

Any string not containing ..........: To describe any string at all that does not contain an ‘a’ (with S = {a, b, c}), you

can use (b + c)*. Any string containing exactly one ........:

To describe any string that contains exactly one ‘a’ put “any string not containing an a”, on either side of the ‘a’ like: (b + c)* a(b + c)* .\

Languages defined by Regular Expressions There is a very simple correspondence between regular expressions and the languages they denote:

Regular Expressions to NFA

(i) For any x in S, the regular expression denotes the language {x}. The NFA (with a single start state and a single final state) as shown below, represents exactly that language.

(ii) The regular expression l denotes the language {l}that is the language

containing only the empty string.

(iii) The regular expression ∅ denotes the language ∅; no strings belong to this

language, not even the empty string.

(iv) For juxtaposition, strings in L(r1 ) followed by strings in L(r2 ), we chain the NFAs together as shown.

(v) The “+” denotes “or” in a regular expression, we would use an NFA with a choice of paths.

(vi) The star (*) denotes zero or more applications of the regular expression, hence a loop has to be set up in the NFA.

NFAs to Regular Expression

The basic approach to convert NFA, to Regular Expressions is as follows:

(i) If an NFA has more than one final state, convert it to an NFA with only one final state. Make the original final states non final, and add a λ-transition from each to the new (single) final state.

(ii) Consider the NFA to be a generalized transition graph, which is just like an NFA except that the edges may be labeled with arbitrary regular expressions. Since the labels on the edges of an NFA may be either λ or members of each of these can be considered to be a regular expression.

(iii) Removes states one by one from the NFA, re labeling edge as you go, until only the initial and the final state remain.

(iv) Read the final regular expression from the two state automation those results. The regular expression derived in the final step accepts the same language as the original NFA.

RE Examples • L(001) = {001} • L(0+10*) = { 0, 1, 10, 100, 1000, 10000, … } • L(0*10*) = {1, 01, 10, 010, 0010, …} i.e. {w | w has exactly a single 1} • L(∑∑)* = {w | w is a string of even length} • L((0(0+1))*) = { ε, 00, 01, 0000, 0001, 0100, 0101, …} • L((0+ε)(1+ ε)) = {ε, 0, 1, 01} • L(1Ø) = Ø ; concatenating the empty set to any set yields the empty set. • Rε = R • R+Ø = R • Note that R+ε may or may not equal R (we are adding ε to the language) • Note that RØ will only equal R if R itself is the empty set.

DFA to RE: State Elimination

• Eliminates states of the automaton and replaces the edges with regular expressions that includes the behavior of the eliminated states.

• Eventually we get down to the situation with just a start and final node, and this is easy to express as a RE

• Consider the figure below, which shows a generic state s about to be eliminated. The labels on all edges are regular expressions.

• To remove s, we must make labels from each qi to p1 up to pm that include the paths we could have made through s.

q1

qk

p1

pm

.

.

.

R11+Q1S*P1

Rkm+QkS*Pm

.

.

.Rk1+QkS*P1

R1m+Q1S*Pm

q1

qk

s

p1

pm

.

.

.

.

.

.

R1m

R11

SQ1

QK

Rkm

Rk1

P1

Pm

Steps: 1. Starting with intermediate states and then moving to accepting states, apply the

state elimination process to produce an equivalent automaton with regular expression labels on the edges.

• The result will be a one or two state automaton with a start state and accepting state.

2. If the two states are different, we will have an automaton that looks like the

following:

We can describe this automaton as: (R+SU*T)*SU* 3. If the start state is also an accepting state, then we must also perform a state

elimination from the original automaton that gets rid of every state but the start state. This leaves the following:

We can describe this automaton as simply R*. 4. If there are n accepting states, we must repeat the above steps for each accepting

states to get n different regular expressions, R1, R2, … Rn. For each repeat we turn any other accepting state to non-accepting. The desired regular expression for the automaton is then the union of each of the n regular expressions: R1∪ R2… ∪ RN

DFA����RE Example • Convert the following to a RE

3Start 1 21 1

0

0

0,1

Start

R

StartS

R

T

U

• First convert the edges to RE’s:

Answer: (0+10)*11(0+1)* Algebraic Laws for RE’s: • Just like we have an algebra for arithmetic, we also have an algebra for regular

expressions. – While there are some similarities to arithmetic algebra, it is a bit different

with regular expressions.

• Commutative law for union: – L + M = M + L

• Associative law for union: – (L + M) + N = L + (M + N)

• Associative law for concatenation: – (LM)N = L(MN)

• Note that there is no commutative law for concatenation, i.e. LM ≠ ML

3Start 1 21 1

0

0

0+1

• The identity for union is: L + Ø = Ø + L = L

• The identity for concatenation is: Lε = εL = L • The annihilator for concatenation is: ØL = LØ = Ø • Left distributive law: L(M + N) = LM + LN • Right distributive law: (M + N)L = LM + LN • Idempotent law: L + L = L Laws Involving Closure • (L*)*= L*

– i.e. closing an already closed expression does not change the language • Ø* = ε • ε*= ε • L+= LL* = L*L -more of a definition than a law • L*= L+ + ε • L?= ε + L -more of a definition than a law

TWO-WAY FINITE AUTOMATA

Two-way finite automata are machines that can read input string in either direction. This type of machines has a “read head”, which can move left or right over the input string. Like the finite automata, the two-way finite automata also have a finite set Q of states and they can be either deterministic (2DFA) or nondeterministic (2NFA). They accept only regular sets like the ordinary finite automata. Let us assume that the symbols of the input string are occupying cells of a finite tape, one symbol per cell as shown in fig. The left and right end markers |— and —| enclose the input string. The end markers are not included in the input alphabet Σ.

Finite Automata With Output : Definition

A finite-state machine M = (Q, S,O,d, l, q ) 0 consists of a finite set Q of states, a finite input alphabet S, a finite output alphabet O, a transition function d that assigns to each state and input pair a new state, an output function l that assigns to each state and input pair an output, and an initial state q0.

Let M = (Q, S,O,d, l, q ) 0 be a finite state machine. A state table is used to denote the values of the transition function d and the output function l for all pairs of states and input.

Mealey Machine

Usually the finite automata have binary output, i.e., they accept the string or do not accept the string. This is basically decided on the basis of whether the final state is reached by the initial state. Removing this restriction, we are trying to consider a model where the outputs can be chosen from some other alphabet.

The values of the output function F(t) in the most general case is a function of the present state q(t) and present input x(t).

where λ is called the output function. This model is called the “Mealey machine”.

A “Mealey machine” is a six-tuple (Q, Σ,O,δ, λ, q ) 0 where all the symbols except λ have the same meaning as discussed in the sections above. λ is the output function mapping Σ ×Q into O.

Moore Machine

If the output function F(t) depends only on the present state and is independent of the present input q(t), then we have the output function f(t) given by

F (t) = λ(q(t))

A Moore machine is a six-tuple (Q, S, O, d, l, q 0 ) with the usual meanings for symbols.

Example:

Give examples for Moore and Mealy Models of finite automata with outputs. State Table shown in Fig. (a) represents a Moore Machine and that of Fig. (b) shows a Mealey Machine.

Properties of Regular Sets: A regular set (language) is a set accepted by a finite automaton. 1. Closure

A set is closed under an operation if, whenever the operation is applied to members of the set, the result is also a member of the set. For example, the set of integers is closed under addition, because x + y is an integer whenever x and y are integers. However, integers are not closed under division: if x and y are integers, x/y may or may not be an integer.

There are several operations defined on languages:

Union, Concatenation, Negation, Kleene Star, Reverse The general approach is as follows:

(i) Build automata (DFA or NFA) for each of the languages involved. (ii) Show how to combine the automata in order to form a new automaton

which recognizes the desired language. (iii) Since the language is represented by NFA/DFA, we shall conclude that the

language is regular.

Union of L1 and L2 (a) Create a new start state (b) Make a λ-transition from the new start state to each of the original start states. Concatenation of L1 and L2 (a) Put a λ-transition from each final state of L1 to the initial state of L2. (b) Make the original final states of L1 non final. Intersection and Set Difference

Just as with the other operations, it can be proved that regular languages are closed under intersection and set difference by starting with automata for the initial languages, and constructing a new automaton that represents the operation applied to the initial languages.

In this construction, a completely new machine is formed, whose states are labeled with an ordered pair of state names: the first element of each pair is a state from L1 and the second element of each pair is a state from L2.

(a) Begin by creating a start state whose label is (start state of L1, start state of L2).

(b) Repeat the following until no new arcs can be added: (1) Find a state (A, B) that lacks a tran sition for some x in S.

(2) Add a transition on x from state (A, B) to state ( ( d A, x), d (B, x)). (If this state does not already exist, create it).

Nega tion of L1

(a) Start with a complete DFA, not with an NFA (b) Make every final state nonfinal and every nonfinal state final.

Kleene star of L1

(a) Make a new start state; connect it to the original start state with a l-transition. (b) Make a new final state; connect the original final state (which becomes nonfinal) to it with l-transitions.

(c) Connect the new start state and new final state with a pair of l-transitions. Reverse of L1

(a) Start with an automaton with just one final state. (b) Make the initial state final and final state initial. (c) Reverse the direction of every arc. The same construction is used for both intersection and set difference. The distinction is in how the final states are selected.

Inter section

Make a state (A, B) as final if both (i) A is a final state in L1 and (ii) B is a final state in L2

Set Difference Mark a state (A, B) as final if A is a final state in L1, but B is not a final state in L2.

Pumping Lemma Principle of Pumping Lemma

� If an infinite language is regular, it can be defined by a DFA. � The DFA has some finite number of states (say). � Since the language is infinite, some strings of the language should have

length > n. � For a string of length > n accepted by the DFA, the walk through the DFA must

contain a cycle. � Repeating the cycle an arbitrary number of times should yield another string

accepted by the DFA. Principle of Pumping Lemma for regular languages

The “pumping lemma” for regular languages is another way of showing that a given infinite language is not regular. The proof is always done by “contradiction”. The technique that is followed is as outlined below:

(i) Assume that the language L is regular. (ii) By Pigeon-hole principle, any sufficiently long string in L should repeat

some state in the DFA, and therefore, the walk contains a “cycle”. (iii) Show that repeating the cycle some number of times (“pumping” the

cycle) yields a string that is not in L. (iv) Conclude that L is not regular.

Applying the Pumping Lemma Definition of Pumping Lemma If L is an infinite regular language, then there exists some positive integer ‘m’ such that any string w∈L, whose length is ‘m’ or greater can be decomposed into three parts, xyz where

Example

Prove that L = {anbn : n ≥ 0} is not regular. Solution

(i) We don’t know m, but let us assume that there is one. (ii) Choose a string w = anbn, where n > m, so that any prefix of length ‘m’

consists only of a’s. (iii) We don’t know the decomposition of w into xyz, , but since | xy| <= m, xy

must consist entirely of a’s. Moreover, y cannot be empty. (iv) Choose i = 0. This has the effect of dropping | y | a’s out of the string,

without affectng the number of b’s. The resultant string has fewer a’s than b’s, hence does not belong to L.

Therefore L is not regular.

Summary: This chapter deals with the regular languages which are also known as type 3 languages and the automata that works on regular languages. The pumping lemma is used to prove that a particular language is not regular. The FA(finite automata) can work only on regular languages.

Key Terms: Automaton: Abstract model of a digital computer. Acceptor: Automaton whose output response is “Yes” or “No” DFA: Deterministic Finite Automata. NFA: Non-deterministic Finite Automata. Regular Language: Language that can be constructed from the set operations—Union, Concatenation and Kleene star. Regular expression: Mathematical tool built from a set of primitives and operations. Two-way Finite Automata: Machines that can read input string in either direction. Moore machine: Output function depends only on present state and independent of present input. Mealey machine: Value of the output function is a function of the present state and present input in a Mealey Machine. Pumping lemma: A way to show that an infinite language is not regular. REVIEW QUESTIONS 1. Define the term ‘Automata’ with an example. 2. What are the types of Automaton? 3. Explain Deterministic automata with an example. 4. Explain Non-deterministic automaton with an example. 5. Distinguish between DFA and NFA. 6. Explain the terms:

(a) State Table diagram (b) State Transition diagram.

7. Define Non-deterministic Finite automata. 8. Comment on the equivalence of NFA and DFA. 9. What are regular expressions? 10. Define a regular language. 11. Give examples for regular expressions. 12. Comment on the correspondence between regular expressions and the languages they denote. 13. How will you convert an NFA to a regular expression? 14. What do you mean by two way finite automata? 15. What do you mean by finite automata with output. 16. What do you mean by a Mealy machine? 17. What do you mean by a Moore machine? 18. Give examples for Moore and mealy models of finite automata with outputs. 19. State the properties of regular sets. 20. State the principle of pumping lemma. 21. Define Pumping lemma. 22. Explain the closure properties of Regular languages. 23. What is Isomorphism? 24. State the Myhill-Nerode relations.

EXERCISES: