1 Theory of Digital Computation Course material for undergraduate students on IT Department of...

1

Theory of Digital Computation

Course material for undergraduate students on IT

Department of Computer ScienceUniversity of Veszprem

Veszprem, Hungary

2002

2

T heory o f D ig ita l C om puta tio n

Navigation Tools

Navigation buttons at the bottom of each page make easy to jump to:

Previous page

Previous chapter

Table of Contents

Next chapter

Next page

The title of the chapter is shown above the title of the page.

Véges automatákNavigation

3


Table of Contents

Alphabets and Languages

Finite Automata

Context-free Languages

Turing Machines

Universal Turing Machines

Examples

Problems

Table of Contents

4



Alphabet is a finite set of symbols.

A string over an alphabet is a finite sequence of symbols from the alphabet.

A string may have no symbols at all in this case it is called the empty string and is denoted by e.

The length of a string w, w, is its length as a sequence.


E xam p le s

5


Two strings over the same alphabet can be combined to form a third by the operation concatenation. The concatenation of strings x and y, written xy or simply xy, is the string x followed by the string y:

w(j) = x(j) for j = 1,...,x

w(x+ j) = y(j) for j = 1,...,y

If w = xy then w=x+y.


E xam p le s

6


A string v is a substring of w iff there are strings x and y such that w = xvy.

The reversal of a string w, denoted by wR, is the string “spelling backwards”:

If w= 0, then wR = w = e.

If w= n+1 (>0), then w = ua for some a, and wR = auR.

Any set of string over an alphabet is called language.


E xam p le s

E xam p le s

7


Closure or Kleene star is a language operation of a single language L, denoted by L*.

L* is the set of all strings obtained by concatenating zero or more strings from L.


E xam p le s

8


DefinitionA string over alphabet ( , ) , Ø , , *} is a regular expression over alphabet if it satisfies the following conditions.

(1) Ø and each element of is a regular expression

(2) If and are regular expressions then so is ()

(3) If and are regular expressions then so is ()

(4) If is a regular expression, then so is *.

(5) Nothing is a regular expression unless it follows from (1) through (4).


E xam p le s

9


Every regular expression represents a language.

Formally, the relation between regular expressions and the corresponding languages is established by function L, where L() is the language represented by regular expression .

Thus, L is a function from strings to languages.


10


Function L is defined as follows.

(1) L(Ø) = Ø and L() = a for each = a

(2) If and are regular expressions, then L( () ) = L()L().

(3) If and are regular expressions, then L( () ) = L()L().

(4) If is a regular expression then L(*) = L()*.


E xam p le s

11


Finite Automata

Finite Automata

DefinitionDeterministic finite automaton (DFA) is a quintuple M = (K, , , s, F)

where

K is the set of states (finite set),

is an alphabet,

s K is the initial state,

F is the set of final states (F K), and

, the transition function, is a function from K to K.

E xam p le s

12


Graphical representation of a DFA: state diagram

state p:

- initial state q:

- final state r:

transition (p, b) = q:

E xam p le s

Finite Automata

13


A configuration of a DFA is determined by the current state and the unread part of input. In other words, a configuration of a deterministic finite automaton (K, , , s, F) is an element of K*.

If (q, w) and (q’, w’) are two configurations of M, then (q, w) ├─M (q’, w’) iff w = w’ for some symbol , and

(q, ) = q’. In this case, we say that configuration (q, w) yields configuration (q’, w’) in one step.

Finite Automata

E xam p le s

E xam p le s

14


We denote the reflexive, transitive closure of ├─M by ├─M*;

(q, w) ├─M* (q’, w’) is read, configuration (q, w) yields

configuration (q’, w’).

A string w * is said to be accepted by M iff there is a state qF such that (s, w) ├─M* (q, e).

The language accepted by M, L(M), is the set of all strings accepted by M.

Finite Automata

E xam p le s

E xam p le s

E xam p le s

15


DefinitionNon-deterministic finite automaton (NFA) is a quintuple M = (K, , , s, F)

where

K is the set of states (finite),

is an alphabet,

s K is the initial state,

F is the set of final states (F K),and

, the transition relation, is a finite subset of K×*×K.

Finite Automata

16


Graphical representation of a NFA: state diagram

state p:

- initial state q:

- final state r:

transition (p, ba,q) :

Finite Automata

E xam p le s

17


A configuration of M is an element of K×*.

The relation ├─M (yields in one step) between

configurations is defined as follows: (q, w) ├─M (q’, w’) iff

there is a u * such that w = uw’ and (q, u, q’) .

├─M* is the reflexive, transitive closure of ├─M.

w * is accepted by M iff there is a state q F such that (s, w)├─M*(q, e).

The language accepted by M, L(M), is the set of all strings accepted by M.

Finite Automata

E xam p le s

18


DefinitionFinite automata M1, and M2 are said to be equivalent iff

L(M1) = L(M2).

TheoremFor each non-deterministic finite automaton, there is an equivalent deterministic finite automaton.

(No proof here)

Finite Automata

19


TheoremLanguages accepted by finite automata are closed under union.

Proof:

Let NFA M1= (K1, , 1, s1, F1) and M2= (K2, , 2, s2, F2).

NFA M = (K, , , s, F) will be given such that L(M) = L(M1) L(M2).

s is a new state not in K1K2,

K = K1K2s, (K1 and K2 are disjoint.)

F = F1 F2, and

= 12(s, e, s1), (s, e, s2).

Finite Automata

20


TheoremLanguages accepted by finite automata are closed under concatenation.

Proof:

L(M) = L(M1) ○ L(M2)

Construct a NFA M = (K, , , s, F), where

K = K1K2

(K1 and K2 are disjoint.)

s = s1

F = F2

= 12(F1 {e} {s2}).

Finite Automata

21


TheoremLanguages accepted by finite automata are closed under Kleen-star.

Proof:

L(M) = L(M1)*

Construct a NFA M = (K, , , s, F), where

K = K1s

s is a new state not in K1

F = F1s

= 1(F {e} {s1})

Finite Automata

22


TheoremLanguages accepted by finite automata are closed under complementation.

Proof:

L(M) = L(M1)

Construct a DFA M = (K, , , s, F)

K = K1

s = s1

F = K1 - F1

= 1

Finite Automata

23


TheoremLanguages accepted by finite automata are closed under intersection.

Proof:

L(M) = L(M1) L(M2)

Construct a DFA M

L(M) = * - ((* - L(M1)) ( * - L(M2))).

Note:

L(M1) L(M2) = L(M1) L(M2)

Finite Automata

24


TheoremThere is an algorithm for answering the following question:Given a finite automaton M, is L(M) = * ?

Proof:

- Tracing the state diagram can answer the question wheather L(M’) = .

- L(M) = * iff L(M) = .

- L(M’) = L(M).

Finite Automata

25


TheoremThere is an algorithm for answering the following question:Given two finite automata M1 and M2, is L(M1) L(M2)?

Proof:

L(M1) L(M2) iff

(*-L(M2)) L(M1) = .

Finite Automata

26


TheoremThere is an algorithm for answering the following question:Given two finite automata M1 and M2, is L(M1) = L(M2) ?

Proof:

L(M1) = L(M2) iff

(L(M1) L(M2) and L(M2) L(M1)).

Finite Automata

27




DefinitionA context-free grammar (CFG) G is a quadruple (V, , R, S), where

V is an alphabet,

(the set of terminals) is a subset of V,

R (the set of rules) is a finite subset of (V-)V*, and

S (the start symbol) is an element of V-.

The members of V- are called non-terminals.

28


For A V- and u V*, A G u if (A, u) R.

For any strings u, v V*, u G v if there are strings

x, y, v’ V* and A V- such that u = xAy, v = xv’y and A G v’.

Relation G* is the reflexive, transitive closure of G.

Language generated by G, is L(G) = {w *: S G* w}; we

also say that G generates each string in L(G).


E xam p le s

E xam p le s

29


DefinitionA language L is a context-free language if it is equal to L(G) for some context-free grammar G.

DefinitionA context-free grammar G = (V, , R, S) is regular iff R (V-) *((V-) e).


30


TheoremContext-free languages are closed under union.

Proof:

Let G1 = (V1, 1, R1, S1) and G2 = (V2, 2, R2, S2) context-free grammars, V1-1 and V2-2 disjoint.

CF grammar G = (V, , R, S) will be given such thatL(G) = L(G1) L(G2)

V = V1 V2 {S}

= 1 2

R = R1 R2 {S S1, S S2}

S = a new symbol not in V1 V2.


31


TheoremContext-free languages are closed under concatenation.

Proof:

L(G) = L(G1) L(G2)

Construct CF grammar G = (V, , R, S)

V = V1 V2 {S}

(V1-1 and V2-2 are disjoint.)

= 1 2

R = R1 R2 {S S1S2}

S = a new symbol not in V1 V2.


32


TheoremContext-free languages are closed under Kleen-star.

Proof:

L(G) = L(G1)*

Construct CF grammar G = (V, , R, S)

V = V1

= 1

R = R1 {S1 e, S1 S1S1}

S = S1


33


DefinitionA pushdown automaton (PDA) is a sixtuple M = (K, , , , s, F), where

K is a finite set of states,

is an alphabet (the input symbols),

is an alphabet (the stack symbols),

s is the initial state (s K),

F is the set of final states (F K), and

is the transition relation, is a finite subset of (K**)(K*).


34


Graphical representation of a NFA: state diagram

state p:

- initial state q:

- final state r:

transition (p, ba, cd, q, dc) :

E xam p le s


35


Intuitively, if ((p, u, ), (q, )) , then M, whenever it is in state p with at the top of the stack, may read u from the input tape, replaced by on the top of the stack, and enter state q. ((p, u, ), (q, )) is called a transition of M.

To “push” a symbol is to add it to the top of the stack, to “pop” a symbol is to remove it form the top of the stack.

A configuration is defined to be an element of K**:

The first component is the state of the machine, the second is the partition of the input yet to be read, and the third is the content of the pushdown store.


36


For every transition ((p, u, ),(q, )) , and for every x * and *, we define (p, ux, ) ├─M (q, x, )),

moreover, ├─M (yields in one step) holds only between

configurations that can be represented in this form for some transition, some x, and some .

The reflexive, transitive closure of ├─M is denoted by ├─M*.

We say that M accepts a string w * iff (s, w, e) ├─M*

(p, e, e) for some p F.

Language accepted by M, L(M), is the set of all strings accepted by M.


E xam p le s

37


Turing Machines

Turing Machines

DefinitionA Turing machine is a quadruple (K, , , s) where

K: is a finite set of states, not containing the halt state h; (h K),

: is an alphabet, containing the blank symbol #, but not containing the symbols L and R;

s: is the initial state;

: is a function from K to (K {h})( {L, R}).

Configuration of a Turing machine M = (K, , ,s) is an element of (K {h})* (*(-{#}){e}).

38


Let (q1, w1, a1, u1) and (q2, w2, a2, u2) be configurations of Turing

machine M. Then (q1, w1, a1, u1) ├─M (q2,w2, a2, u2)

iff for some b {L, R}, (q1, a1) = (q2, b) and either

(1) b , w1 = w2, u1= u2, and a2 = b; or

(2) b = L, w1 = w2a2, and either

(a) u2 = a1u1, if (a1 # or u1 e), or

(b) u2 = e, if a1 = # and u1= e; or

(3) b = R, w2 = w1 a1 and either

(a) u1 = a2u2, or

(b) u1 = u2 = e and a2 = #.

Turing Machines

39


For any Turing machine M, relation ├─M* is the reflexive,

transitive closure of relation ├─M; We say configuration C1

yields configuration C2 if C1├─*M C2.

A computation by M is a sequence of configurations C0, C1,

C2,..., Cn (n0) such that

C0 ├─M C1 ├─M C2├─M … ├─M Cn .

Turing Machines

40


DefinitionLet f be a function from 0* to 1*. Turing machine M = (K,

, , s) , where 0, 1 , computes f if w 0* and

f(w) = u, then

(s, #w, #, e) ├─M* (h, #u, #, e).

If such a Turing machine exists, the function is called Turing computable.

Turing Machines

41


Let 0 an alphabet not containing the blank symbol. Let Y

and N be two fixed symbols not in 0. Then, language L

0* is Turing decidable iff function xL: 0* {Y, N} is Turing

computable, where for each w 0*,

xL(w) = Y if w L

N if w L

Turing Machines

{

42


Let 0 be an alphabet not containing #. M accepts string w

0* if M halts on input w.

Thus, M accepts language L 0* iff L = {w 0*: M

accepts w}.

DefinitionA language is Turing acceptable if there is some Turing machine that accepts it.

Turing Machines

43


Symbol writing Turing machines

For alphabet and symbol a ,

let Turing machine a = (K, , , s) =({p}, , {(p, b, h, a): for any b }, p)

Head moving Turing machines

For alphabet ,

let Turing machine L be defined as L = (K, , , s) =({p}, , {(p, b, h, L): for any b }, p), and

let Turing machine R be defined as R = (K, , , s) =({p}, , {(p, b, h, R): for any b }, p).

Turing Machines

44


DefinitionA machine schema is a triplet (m, M0) where

mis a finite set of Turing machines with common alphabet and disjoint sets of states;

M0 m is the initial machine; and

is a function from subset of m to m.

Turing Machines

45


Let m= {M0, M1, …, Mm} (m≥0), where

Mi = (Ki, , i, si) for i = 0, 1, …, m.

Let q0, q1, …, qm be new states not in any of the Ki.

Then machine schema (m, M0) is defined asTuring

machine M, where M = (K, , , s)

K = K0 ... Km {q0, q2, …, qm},

s = s0, and

is defined as follows.

Turing Machines

46


Definition of

(a) if q Ki (0 ≤ i ≤ m), a , i(q, a) = (p, b), and p h,

then (q, a) = i(q, a) = (p, b);

(b) if q Ki (0 ≤ i ≤ m), a , and i(q, a) = (h, b) then

(q, a) = (qi, b);

(c) if a and (Mi , a) (0 ≤ i ≤ m) is not defined, then

(q, a) = (h, a);

(d) if a and (Mi , a) = Mj (0 ≤ i ≤ m) and

j(sj, a) = (p, b), then

(qi, a) = (p, b) if p h and

(qj, b) if p = h.

Turing Machines

E xam p le s

47


TheoremEvery Turing decidable language is Turing acceptable.

Proof:

TheoremIf L Turing decidable language, then it’s complement L is also Turing-decidable.

Proof:

Turing Machines

48


A language is Turing-decidable if and only if both it and its complement are Turing acceptable .

Proof:

(only if)

L is a Turing decidable L is also Turing decidable.

L, L is Turing decidable L, L is Turing acceptable.

(if)

2-tape machine: M1 accepts L, M2 accepts L.

Parallel simulation: Which halts?

Turing Machines

49




The only feature possessed by electronic computers that missing from the capability of Turing machines is the programmability.

Difficulties:

The alphabet of any Turing machine must be finite.

The set of states of any Turing machine must be finite.


50


Hints:

Assume that there are fixed countably infinite sets

K = {q1, q2, q3, ...}, and

= {a1, a2, a3, ...}

such that for every Turing machine, the set of states is a subset of K and the alphabet is a subset of .


51


Encode the alphabet and states of the Turing machine as

()

qi Ii+1

h I

L I

R II

ai Ii+2


52


Encode the Turing machine M over the alphabet {c, I}.

p(M) = cS0cS11S12 ... S1lS21S22 ... Sk1 ... Sklc

S0 encodes the initial state, S0 = (s).

Spr encodes the values of the transition function,

Spr = cw1cw2cw3cw4, where

w1 = (qi p), w2 = (qj r), w3 = (q’), w4 = (b)

for each

(qi p, qj r) = (q’, b).


53


The symbols on the tape must also be in encoded form.

p(w) = c(b1)c(b2)c ... c(bn)c

for each

w = b1b2 ... bn.


54


Universal Turing machine U simulates Turing machine M on a three tape Turing machine.

Tape 1: encoding of the tape of Turing machine M.

Tape 2: encoding of Turing machine M (i.e., it is the “program”).

Tape 3: encoding of the current state of Turing machine M during the simulation.


55


Examples

Examples

Symbol

Every character is a symbol, e. g., a, b, and c, etc.

Alphabet

= {a, b, c} is an alphabet.

String

cab, abaca, a, and the empty string e are strings over the alphabet = {a, b, c}.

Length of a string

cab= 3, abaca= 5, a= 1, and e=0.


56

T heory o f D ig ita l C om puta tio n Examples

Concatenation of strings

Strings cab and abaca can be concatenated to form string

cababaca.

Formally:

cababaca = cababaca


57


Substring

Strings e, a, b, c, ab, ba, ac, ca, aba, bac, aca, abac, baca, and abaca are the substrings of the string abaca.

Reversal of a string

eR = e

aR = a

abR = ba

bacR = cab

abacaR = acaba


58


Language

Set L = {e, a, cab, abaca} is a language over the alphabet = {a, b, c}.

Thus, language L is a subset of * = {e, a, b, c, aa, ab, ac, ba, bb, bc, ca, cb, cc, aaa, aab, aab, aba, …}.


59


Language operation Kleene star

Closure or Kleene star of language L = {a, cab, abaca} the language L* =

{e, a, cab, abaca, aa, acab, aabaca, caba, cabcab, cababaca, …, abacaaa, abacaacab, …} =

{e, a, cab, abaca, aa, acab, aabaca, caba, cabcab, cababaca, …, abacaaa, abacaacab, …}


60


Regular expressions

Expressions

Ø, a, b,

(ab), ((ab)a),

(ab), (a(ab)), ((ab)((ab)a)), ((ab)c),

((ab)a), ((ab)(ab)),

a*, (ab)*, (ab)*,

(a*b), (a*b), ((ab)*((ab*)a)), and ((ab)*c*)

are regular expressions over the alphabet = {a, b, c}.


61


Language represented by regular expression

L(((ab)c)*) = ?

L(a) = {a}; L(b) = {b}; L(c) = {c};

L((ab)) = L(a) L(b) = {a} {b} = {a, b};

L(((ab)c)) = L((ab))L(c) = {a, b} {c} = {ac, bc};

L(((ab)c)*) = L(((ab)c))* = {ac, bc}* =

{e, ac, bc, acac, acbc, bcac, bcbc, acacac, acacbc, acbcac,

…, acbcacbcac, acbcacbcbc, … }


62


Deterministic finite automaton

M1= (K, , , s, F) =

(K= {p, q},

= {a, b},

= {(p, a, p), (p, b, q), (q, a, q), (q, b, p)},

s = p,

F = {q})

Finite Automata

63


Graphical representation of DFA

M1= (K= {p, q},

= {a, b},

= {(p, a, p), (p, b, q), (q, a, q), (q, b, p)},

s = p,

F = {q})

is the state diagram:

Finite Automata

64


Configuration of DFA M1

is: (p, babaa)

Finite Automata

65


Yields in one step

(p, babaa) ├─M (q, abaa)

Finite Automata

66


Yields

(p, babaa) ├─M* (p, aa)

Finite Automata

67


String accepted by DFA M1: abbba

(p, abbba) ├─M* (q, e)

Finite Automata

68


Language accepted by DFA

M1= (K= {p, q},

= {a, b},

= {(p, a, p), (p, b, q), (q, a, q), (q, b, p)},

s = p,

F = {q})

L(M1) = {w {a, b}*: the number of b’s in w is odd}

Finite Automata

69


Graphical representation of NFA

M2= (K= {p, q},

= {a, b},

= {(p, ab, q), (p, e, q), (q, ba, q)},

s = p,

F = {q})


Finite Automata

70


Language accepted by NFA

M2= (K= {p, q},

= {a, b},

= {(p, ab, q), (p, e, q), (q, ba, q)},

s = p,

F = {q})

is L(M2) = L( ((ab) e)(ba)*) ).

Finite Automata

71


For context-free grammar

G = (V = {a, b, A, B, S},

= {a, b},

R = {(S, aSb), (S, e)},

S)

we can write

SaSb, Se

SaSbaaSbbaaaSbbbaaabbb

S * aaSbb * aaabbb, and S * aaabbb


72


Language generated by CFG

G = (V = {a, b, A, B, S},

= {a, b},

R = {(S, aSb), (S, e)},

S)

is L(G) = {anbn: n 0}.


73


Graphical representation of PDA

M3= (K= {p, q},

= {a, b},

= {c},

= {(p, a, e, p, c), (p, e, q), (q, b, c, q, e)},

s = p,

F = {q})



74


Language accepted by PDA

M3= (K= {p, q},

= {a, b},

= {c},

= {(p, a, e, p, c), (p, e, q), (q, b, c, q, e)},

s = p,

F = {q})

is L(M2) = {anbn: n 0}.


75


Turing machine represented by the following machine schema operates as follows: (s, #w, #, e) ├─M* (h, #u, #, e),

where w {a, b}* and u is derived from w by replacing each a in w to b and each b in w to a.

Turing Machines

76


Problems

Problems


Which strings are the elements of the following languages?

- {w: for some u{a, b}{a, b}, w = uuRu}

- {w: ww = www}

- {w: for some u, wuw = uwu}

- {w: for some u, www = uu}

77

T heory o f D ig ita l C om puta tio n Problems

Is it true or false?

- baa {a}*{b}*{a}*{b}*

- {b}*{a}*{a}*{b}* = {a}*{b}*

- {a}*{b}*{c}*{d}* =

- abcd ({a} ({c}{d})* {b})*

78


Write regular expression R with

- L(R) = {w{a, b}*: there is at most three a's in w}

- L(R) = {w{a, b}*: there is no ab substring in w}

- L(R) = {w{a, b}*: w has exactly one aaa substring}

- L(R) = {w{a, b}*: in w every a is preceded and followed by a b}

- L(R) = {w{a, b, c}*: in w there is no aa, bb, cc substring in w}

79


Finite Automata

Draw state diagram of DFA M with

- L(M) = {w{a, b}*: in w the number of a's is even and the number of b's is odd}

- L(M) = {w{a}*: in w the number of a's is even, but not divisible by 4}

- L(M) = {w{a}*: in w the number of a's is either even or divisible by 3}

- L(M) = {w{a}*: in w the number of a's is divisible by 3}

80


Draw state diagram of NFA M with L(M) = L(R) where

- R = (a*b*)*(ab)

- R = (aba)*(aab)*

- R = ( (a*b*)(ab) )*

- R = ((ab)(ab))*

- R = (ab*)(a*b)

81



Which words can be derived in at most four steps in G = (V, , R, S) from S?

V = {a, b, A, B, S},

= {a, b}, and

- R = {SA, SabA, SaB, ASa, Bb}

- R = {SA, SabA, SaB, Aa, BSb}

- R = {SABS, AaA, BbB, Se, Aa, Bb}

- R = {SaSB, SbSA, Sa, Sb, AaS, BbS}

82


Give a CF grammar G with

- L(G) = {w{a, b}*: in w the number of b's is even}

- L(G) = {anbman: n, m > 0}

- L(G) = {ancbnc: n 0}

- L(G) = {bnan: n 0} {a3nbn: n 0}

- L(G) = {ambncpdr: m+n = p+r}

83


Give a pushdown automaton M with

- L(M) = {w{a, b}*: in w the number of a's is larger than the number of b's}

- L(M) = {w{a, b}*: w = wR}

- L(M) = {w{a, b}*: in w the number of b's is odd}

- = {a, +, *, (, )} L(M) = {syntactically correct arithmetic expressions involving + and * over the variable a}

- L(M) = {ambncp: m = n or m = p or n = p}

- L(M) = {ambncndm: m, n > 0}

84


Turing machines

Give a Turing machine which decides language L

- L = {w{a, b}*: in w the number of a's is the double of the number of b's}

- L = {w{a, b}*: in w the number of a's is larger than the number of b's}

- L = {w{a, b}*: in w the number of b's is even}

- L = {w{a, b}*: w = wR}

- L = {anbncn: n > 0}

1 Theory of Digital Computation Course material for undergraduate students on IT Department of...

Documents

Transcript of 1 Theory of Digital Computation Course material for undergraduate students on IT Department of...