![Page 1: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/1.jpg)
Fall 2004 COMP 335 1
Introduction to
Theoretical Computer Science
COMP 335
Fall 2004Slides by Costas Busch, Rensselaer Polytechnic Institute,
Modified by N. Shiri & G. Grahne, Concordia University
![Page 2: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/2.jpg)
Fall 2004 COMP 335 2
•This course: A study of
abstract models of computers and computation.
•Why theory, when computer field is so practical?
•Theory provides concepts and principles, for both hardware and software that help us understand the general nature of the field.
![Page 3: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/3.jpg)
Fall 2004 COMP 335 3
Mathematical Preliminaries
![Page 4: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/4.jpg)
Fall 2004 COMP 335 4
Mathematical Preliminaries
• Sets
• Functions
• Relations
• Graphs
• Proof Techniques
![Page 5: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/5.jpg)
Fall 2004 COMP 335 5
}3,2,1{AA set is a collection of elements
SETS
},,,{ airplanebicyclebustrainB
We write
A1
Bship
![Page 6: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/6.jpg)
Fall 2004 COMP 335 6
Set Representations
C = { a, b, c, d, e, f, g, h, i, j, k }
C = { a, b, …, k }
S = { 2, 4, 6, … }
S = { j : j > 0, and j = 2k for k>0 }
S = { j : j is nonnegative and even }
finite set
infinite set
![Page 7: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/7.jpg)
Fall 2004 COMP 335 7
A = { 1, 2, 3, 4, 5 }
Universal Set: all possible elements U = { 1 , … , 10 }
1 2 3
4 5
A
U
6
7
8
910
![Page 8: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/8.jpg)
Fall 2004 COMP 335 8
Set Operations
A = { 1, 2, 3 } B = { 2, 3, 4, 5}
• Union
A U B = { 1, 2, 3, 4, 5 }
• Intersection
A B = { 2, 3 }
• Difference
A - B = { 1 }
B - A = { 4, 5 }
U
A B2
31
4
5
2
3
1
Venn diagrams
![Page 9: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/9.jpg)
Fall 2004 COMP 335 9
A
• Complement
Universal set = {1, …, 7}
A = { 1, 2, 3 } A = { 4, 5, 6, 7}
12
3
4
5
6
7
A
A = A
![Page 10: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/10.jpg)
Fall 2004 COMP 335 10
02
4
6
1
3
5
7
even
{ even integers } = { odd integers }
odd
Integers
![Page 11: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/11.jpg)
Fall 2004 COMP 335 11
DeMorgan’s Laws
A U B = A B
U
A B = A U BU
![Page 12: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/12.jpg)
Fall 2004 COMP 335 12
Empty, Null Set:= { }
S U = S
S =
S - = S
- S =
U= Universal Set
![Page 13: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/13.jpg)
Fall 2004 COMP 335 13
Subset
A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 }
A B
U
Proper Subset: A B
UA
B
![Page 14: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/14.jpg)
Fall 2004 COMP 335 14
Disjoint Sets
A = { 1, 2, 3 } B = { 5, 6}
A B =
UA B
![Page 15: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/15.jpg)
Fall 2004 COMP 335 15
Set Cardinality
• For finite sets
A = { 2, 5, 7 }
|A| = 3
(set size)
![Page 16: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/16.jpg)
Fall 2004 COMP 335 16
Powersets
A powerset is a set of sets
Powerset of S = the set of all the subsets of S
S = { a, b, c }
2S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }
Observation: | 2S | = 2|S| ( 8 = 23 )
![Page 17: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/17.jpg)
Fall 2004 COMP 335 17
Cartesian ProductA = { 2, 4 } B = { 2, 3, 5 }
A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }
|A X B| = |A|*|B|
Generalizes to more than two sets
A X B X … X Z
![Page 18: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/18.jpg)
Fall 2004 COMP 335 18
FUNCTIONSdomain
12
3
a
bc
range
f : A -> B
A B
If A = domain
then f is a total function
otherwise f is a partial function
f(1) = a4
5
![Page 19: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/19.jpg)
Fall 2004 COMP 335 19
RELATIONS R = {(x1, y1), (x2, y2), (x3, y3), …}
xi R yi
e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1
![Page 20: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/20.jpg)
Fall 2004 COMP 335 20
Equivalence Relations
• Reflexive: x R x
• Symmetric: x R y y R x
• Transitive: x R y and y R z x R z
Example: R = ‘=‘
• x = x
• x = y y = x
• x = y and y = z x = z
![Page 21: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/21.jpg)
Fall 2004 COMP 335 21
Equivalence ClassesFor an equivalence relation R, we define equivalence class of x
[x]R = {y : x R y}
Example:
R = { (1, 1), (2, 2), (1, 2), (2, 1),
(3, 3), (4, 4), (3, 4), (4, 3) }
Equivalence class of [1]R = {1, 2}
Equivalence class of [3]R = {3, 4}
![Page 22: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/22.jpg)
Fall 2004 COMP 335 22
GRAPHSA directed graph G=⟨V, E⟩
• Nodes (Vertices)
V = { a, b, c, d, e }
• Edges
E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) }
node
edge
a
b
c
d
e
![Page 23: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/23.jpg)
Fall 2004 COMP 335 23
Labeled Graph
a
b
c
d
e
1 3
56
26
2
![Page 24: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/24.jpg)
Fall 2004 COMP 335 24
Walk
a
b
c
d
e
Walk is a sequence of adjacent edges
(e, d), (d, c), (c, a)
![Page 25: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/25.jpg)
Fall 2004 COMP 335 25
Path
a
b
c
d
e
A path is a walk where no edge is repeated
A simple path is a path where no node is repeated
![Page 26: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/26.jpg)
Fall 2004 COMP 335 26
Cycle
a
b
c
d
e
12
3
A cycle is a walk from a node (base) to itself
A simple cycle: only the base node is repeated
base
![Page 27: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/27.jpg)
Fall 2004 COMP 335 27
Treesroot
leaf
parent
child
Trees have no cycles
Ordered trees?
![Page 28: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/28.jpg)
Fall 2004 COMP 335 28
root
leaf
Level 0
Level 1
Level 2
Level 3
Height 3
![Page 29: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/29.jpg)
Fall 2004 COMP 335 29
PROOF TECHNIQUES
• Proof by induction
• Proof by contradiction
![Page 30: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/30.jpg)
Fall 2004 COMP 335 30
Induction
We have statements P1, P2, P3, …
If we know
• for some b that P1, P2, …, Pb are true
• for any k >= b that
P1, P2, …, Pk imply Pk+1
Then
Every Pi is true, that is, ∀i P(i)
![Page 31: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/31.jpg)
Fall 2004 COMP 335 31
Proof by Contradiction
We want to prove that a statement P is true
• we assume that P is false
• then we arrive at an incorrect conclusion
• therefore, statement P must be true
![Page 32: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/32.jpg)
Fall 2004 COMP 335 32
Example
Theorem: is not rational
Proof:
Assume by contradiction that it is rational
= n/m
n and m have no common factors
We will show that this is impossible
2
2
![Page 33: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/33.jpg)
Fall 2004 COMP 335 33
= n/m 2 m2 = n2
Therefore, n2 is evenn is even
n = 2 k
2 m2 = 4k2 m2 = 2k2m is even
m = 2 p
Thus, m and n have common factor 2
Contradiction!
2
![Page 34: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/34.jpg)
Fall 2004 COMP 335 34
Pigeon Hole Principle:
If n+1 objects are put into n boxes, then at least
one box must contain 2 or more objects.
Ex: Can show if 5 points are placed inside a square whose sides are 2 cm long at least one pair of points are at a distance ≤ 2 cm.
According to the PHP, if we divide the square into 4, at least two of the points must be in one of these 4 squares. But the length of the diagonals of these squares is 2.
the two points cannot be further apart than 2 cm.
![Page 35: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/35.jpg)
Fall 2004 COMP 335 35
Languages
![Page 36: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/36.jpg)
Fall 2004 COMP 335 36
A language is a set of strings
String: A sequence of letters/symbols
Examples: “cat”, “dog”, “house”, …
Defined over an alphabet: zcba ,,,,
![Page 37: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/37.jpg)
Fall 2004 COMP 335 37
Alphabets and StringsWe will use small alphabets:
Strings
abbaw
bbbaaav
abu
ba,
baaabbbaaba
baba
abba
ab
a
![Page 38: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/38.jpg)
Fall 2004 COMP 335 38
String Operations
m
n
bbbv
aaaw
21
21
bbbaaa
abba
mn bbbaaawv 2121
Concatenation
abbabbbaaa
![Page 39: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/39.jpg)
Fall 2004 COMP 335 39
12aaaw nR
naaaw 21 ababaaabbb
Reverse
bbbaaababa
![Page 40: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/40.jpg)
Fall 2004 COMP 335 40
String Length
Length:
Examples:
naaaw 21
nw
1
2
4
a
aa
abba
![Page 41: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/41.jpg)
Fall 2004 COMP 335 41
Length of Concatenation
Example:
vuuv
853
8
5,
3,
vuuv
aababaabuv
vabaabv
uaabu
![Page 42: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/42.jpg)
Fall 2004 COMP 335 42
The Empty StringA string with no letters:
Observations:
abbaabbaabba
www
0
![Page 43: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/43.jpg)
Fall 2004 COMP 335 43
SubstringSubstring of string:
a subsequence of consecutive characters
String Substring
bbab
b
abba
ab
abbab
abbab
abbab
abbab
![Page 44: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/44.jpg)
Fall 2004 COMP 335 44
Prefix and Suffix
Prefixes Suffixesabbab
abbab
abba
abb
ab
a
b
ab
bab
bbab
abbab uvw
prefix
suffix
![Page 45: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/45.jpg)
Fall 2004 COMP 335 45
Another Operation
Example:
Definition:
n
n wwww
abbaabbaabba 2
0w
0abba
![Page 46: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/46.jpg)
Fall 2004 COMP 335 46
The * Operation : the set of all possible strings from alphabet
*
,,,,,,,,,*
,
aabaaabbbaabaaba
ba
![Page 47: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/47.jpg)
Fall 2004 COMP 335 47
The + Operation : the set of all possible strings from alphabet except
,,,,,,,,,*
,
aabaaabbbaabaaba
ba
}{ *
,,,,,,,, aabaaabbbaabaaba
![Page 48: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/48.jpg)
Fall 2004 COMP 335 48
LanguagesA language is any subset of
Example:
Languages:
*
,,,,,,,,*
,
aaabbbaabaaba
ba
},,,,,{
,,
aaaaaaabaababaabba
aabaaa
![Page 49: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/49.jpg)
Fall 2004 COMP 335 49
Note that:
}{}{
0}{
1}{
0
Sets
Set size
Set size
String length
![Page 50: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/50.jpg)
Fall 2004 COMP 335 50
Another Example
An infinite language }0:{ nbaL nn
aaaaabbbbb
aabb
ab
L Labb
![Page 51: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/51.jpg)
Fall 2004 COMP 335 51
Operations on LanguagesThe usual set operations
Complement:
aaaaaabbbaaaaaba
ababbbaaaaaba
aaaabbabaabbbaaaaaba
,,,,
}{,,,
},,,{,,,
LL *
,,,,,,, aaabbabaabbaa
![Page 52: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/52.jpg)
Fall 2004 COMP 335 52
Reverse
Definition:
Examples:
}:{ LwwL RR
ababbaabababaaabab R ,,,,
}0:{
}0:{
nabL
nbaL
nnR
nn
![Page 53: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/53.jpg)
Fall 2004 COMP 335 53
Concatenation
Definition:
Example:
2121 ,: LyLxxyLL
baaabababaaabbaaaab
aabbaaba
,,,,,
,,,
![Page 54: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/54.jpg)
Fall 2004 COMP 335 54
Another OperationDefinition:
Special case:
n
n LLLL
bbbbbababbaaabbabaaabaaa
babababa
,,,,,,,
,,,, 3
0
0
,, aaabbaa
L
![Page 55: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/55.jpg)
Fall 2004 COMP 335 55
More Examples
}0:{ nbaL nn
}0,:{2 mnbabaL mmnn
2Laabbaaabbb
![Page 56: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/56.jpg)
Fall 2004 COMP 335 56
Star-Closure (Kleene *)
Definition:
Example:
210* LLLL
,,,,
,,,,
,,
,
*,
abbbbabbaaabbaaa
bbbbbbaabbaa
bbabba
![Page 57: Introduction to Theoretical Computer Science COMP 335 Fall 2004](https://reader035.vdocuments.us/reader035/viewer/2022062217/56815a68550346895dc7b8a5/html5/thumbnails/57.jpg)
Fall 2004 COMP 335 57
Positive Closure
Definition:
*
21
L
LLL
,,,,
,,,,
,,
,
abbbbabbaaabbaaa
bbbbbbaabbaa
bba
bba