Download - Integrating Different Ideas Together
LeongHW, SoC, NUS(UIT2201: AI) Page 1© Leong Hon Wai, 2003-2008
Integrating Different Ideas TogetherReading Materials:
Ch 3.6 of [SG]Contents:
Incrementing a Binary Counter How many subsets, Representation Printing All Subsets Problem Exponential Time Algorithms
LeongHW, SoC, NUS(UIT2201: AI) Page 2© Leong Hon Wai, 2003-2008
Four seemingly unrelated problems
P1: You are given a bit-array A = (A[n−1], A[n−2], ... A[1] A[0]), where each A[k] is 0 or 1. Given an algorithm for incrementing A by 1.
P2: Suppose you are given a set S = {s1, s2, ..., sn} of n objects, and a subset X of S. How can we use the bit-vector A to represent the subset X?
P3: Using P1 and P2 above, or otherwise, give a simple algorithm to generate and print all subsets of S.
P4: What is the running time of your algorithm? [Note: Recall that there are 2n subsets altogether. (A proof is provided by P1 and P2 above.)]
Hon Wai Leong, NUS (CS5234, 28 Aug 2007) Page L3.3Copyright © 2005-7 by Leong Hon Wai)
n-bit Binary Counterx A[4] A[3] A[2] A[1] A[0] Cost
0 0 0 0 0 0 01 0 0 0 0 1 12 0 0 0 1 0 33 0 0 0 1 1 44 0 0 1 0 0 75 0 0 1 0 1 86 0 0 1 1 0 107 0 0 1 1 1 118 0 1 0 0 0 159 0 1 0 0 1 16
10 0 1 0 1 0 1811 0 1 0 1 1 19
Hon Wai Leong, NUS (CS5234, 28 Aug 2007) Page L3.4Copyright © 2005-7 by Leong Hon Wai)
Incrementing a Binary Counter
n-bit Binary Counter: A[0..n1]
x = A[n−1] 2⋅ (n−1) + . . . + A[1] 2⋅ 1 + A[0] 2⋅ 0
INCREMENT(A)1. i 02. while i < length[A] and A[i] = 13. do A[i] 0 ⊳ reset a bit4. i i + 15. if i < length[A]6. then A[i] 1 ⊳ set a bit
€
x = A[i]⋅ 2i
i=0
n −1∑
LeongHW, SoC, NUS(UIT2201: AI) Page 5© Leong Hon Wai, 2003-2008
P1: You are given a bit-array A = (A[n−1], A[n−2], ... A[1] A[0]), where each A[k] is 0 or 1. Given an algorithm for incrementing A by 1.
Note: For UIT2201, only need to understand the first 2 slides on the problem P1 (incrementing a binary counter).
(The other slides are included for info only.)
Four seemingly unrelated problems
Hon Wai Leong, NUS (CS5234, 28 Aug 2007) Page L3.6Copyright © 2005-7 by Leong Hon Wai)
Worst-case analysis
Consider a sequence of n insertions. The worst-case time to execute one insertion is O(k). Therefore, the worst-case time for all the n insertions is n · O(k) = O(n k).
In fact, WRONG! In fact, the worst-case cost for n insertions is only Q(n) ≪ Q(n k).Let’s see why.
Hon Wai Leong, NUS (CS5234, 28 Aug 2007) Page L3.7Copyright © 2005-7 by Leong Hon Wai)
Tighter analysisx A[4] A[3] A[2] A[1] A[0] Cost
0 0 0 0 0 0 01 0 0 0 0 1 12 0 0 0 1 0 33 0 0 0 1 1 44 0 0 1 0 0 75 0 0 1 0 1 86 0 0 1 1 0 107 0 0 1 1 1 118 0 1 0 0 0 159 0 1 0 0 1 16
10 0 1 0 1 0 1811 0 1 0 1 1 19
A[0] flipped every op n
A[1] flipped every 2 ops n/2
A[2] flipped every 4 ops n/22
A[3] flipped every 8 ops n/23
… … … … …
A[i] flipped every 2i ops n/2i
Total cost of n operations
Hon Wai Leong, NUS (CS5234, 28 Aug 2007) Page L3.8Copyright © 2005-7 by Leong Hon Wai)
Tighter analysis (continued)
)(
2 21
2
1
lg
1
n
nn
n
ii
n
ii
Q
Cost of n increments
.Thus, the average cost of each increment operation is Q(n)/n = Q(1).
Hon Wai Leong, SoC, NUS(Creative Problem Solving) Page 9
Copyright © by Leong Hon Wai
P2: Using n-bit-array to represent
subsets?
LeongHW, SoC, NUS(UIT2201: AI) Page 10© Leong Hon Wai, 2003-2008
Four seemingly unrelated problems
P1: You are given a bit-array A = (A[n−1], A[n−2], ... A[1] A[0]), where each A[k] is 0 or 1. Given an algorithm for incrementing A by 1.
P2: Suppose you are given a set S = {s1, s2, ..., sn} of n objects, and a subset X of S. How can we use the bit-vector A to represent the subset X?
Note: For UIT2201, only need to understand bit-representation method for problem P2.
(The other slides are included for info only.)
Hon Wai Leong, SoC, NUS(Creative Problem Solving) Page 11
Copyright © by Leong Hon Wai
PS: (Power-Set Example) (1)
Problem: Given a set S with n elements. How many subsets of S are there?
Stage 1: Understanding the Problem
PQ: What is the unknown? [# subsets of S] PQ: What is the data? [A set S with n
elements] PQ: What is the condition? [Subsets of S. Need to count all of them.] PQ: Is it sufficient? [Yes? Can count one-by-one, but tedious]
Hon Wai Leong, SoC, NUS(Creative Problem Solving) Page 12
Copyright © by Leong Hon Wai
PS: (Power-Set Example) (2)
Stage 2: Devising a Plan PQ: Have you seen the problem before? PQ: Can you try to work out some small instances?
PQ: Can you see any pattern?
Stage 3: Carrying out the Plan. PQ: Can you prove the result?
n S P(S) #0 ϕ ϕ 1
1 {x1} ϕ, {x1} 22 {x1, x2} ϕ, {x1}, {x2}, {x1, x2} 43 {x1, x2,
x3} ϕ, {x1}, {x2}, {x1, x2} {x3}, {x1, x3},{x2, x3}, {x1, x2, x3}
8
Hon Wai Leong, SoC, NUS(Creative Problem Solving) Page 13
Copyright © by Leong Hon Wai
PS: (Power-Set Example) (3)
Another Approach? PQ: Can you solve it differently? PQ: Introduce suitable notations?
Let An be the # of subsets of a S with n element. Now, consider the set S’ = S U {x} with n+1 elements. Divide subsets of S’ into
(P1) those containing element x, and (P2) those that do not contain element x. Those in P2 are exactly all the subsets of S (and we have An of them). For every subset T of S in P2,
there is a corresponding subset that contains x, namely T U {x} in P1. Thus, there is a 1-1 correspondence betw subsets in P1 and those in P2. Therefore, An+1 = 2 An for all n ≥ 0. In addition, we know A0 = 1.
Hon Wai Leong, SoC, NUS(Creative Problem Solving) Page 14
Copyright © by Leong Hon Wai
PS: (Power-Set Example) (4)
Yet Another Approach? PQ: Draw a figure?
c'
c c'
c c c'
c c'
b b' b b'
a a'
Where are the subset?
Hon Wai Leong, SoC, NUS(Creative Problem Solving) Page 15
Copyright © by Leong Hon Wai
PS: (Power-Set Example) (5)
Yet Another Method? PQ: Look at the unknown. [Subsets! They come in different sizes]
IDEA: Let’s count those of the same size!How many of size 0? 1How many of size 1? nHow many of size 2? n(n-1)/2 = nC2 . . . How many of size k? nCk . . . How many of size n–1? nHow many of size n? 1 Total # of subsets is
€
nk ⎛ ⎝ ⎜
⎞ ⎠ ⎟
k =0
n
∑ = 2n
€
(x + y)n =nk ⎛ ⎝ ⎜
⎞ ⎠ ⎟
k =0
n
∑ x k y(n −k )
Hon Wai Leong, SoC, NUS(Creative Problem Solving) Page 16
Copyright © by Leong Hon Wai
PS: (Power-Set Example) (6)
Yet another method? PQ: How can the subsets be represented? Let the vector (b1, b2, …, bn) represent the subset T where bk=1, if the element xk is in the set T, and 0 otherwise.
PQ: How many such bit-strings are there?
Subset Bit-Representation
ϕ (0,0,…,0,0) {x1} (1,0,…,0,0) {x2} (0,1,…,0,0) {x1, x2} (1,1,…,0,0)
{x1, x2, ..., xn} (1,1,…,1,1)
LeongHW, SoC, NUS(UIT2201: AI) Page 17© Leong Hon Wai, 2003-2008
Four seemingly unrelated problems
P1: You are given a bit-array A = (A[n−1], A[n−2], ... A[1] A[0]), where each A[k] is 0 or 1. Given an algorithm for incrementing A by 1.
P2: Suppose you are given a set S = {s1, s2, ..., sn} of n objects, and a subset X of S. How can we use the bit-vector A to represent the subset X?
P3: Using P1 and P2 above, or otherwise, give a simple algorithm to generate and print all subsets of S.
P4: What is the running time of your algorithm? [Note: Recall that there are 2n subsets altogether. (A proof is provided byP1 and P2 above.)]
Hon Wai Leong, SoC, NUS(Creative Problem Solving) Page 18
Copyright © by Leong Hon Wai
Solution to P3 and P4
Outline of algorithm for P3:Represent subsets using bit vector A1. Start with A = (0, 0, . . . , 0, 0)2. Repeat these steps 3. Increment the bit vector A (use algorithm from P1)4. Print out the subset corresponding to A5. until A = (1, 1, . . . , 1, 1)
Analysis (P4):Steps 3,4 done 2n time; (there are 2n subsets!)
Hon Wai Leong, SoC, NUS(Creative Problem Solving) Page 19
Copyright © by Leong Hon Wai
Solution to P3 and P4
Analysis (P4):Steps 3,4 done 2n time; (there are 2n subsets!) Step 3 takes time at most O(n) Step 4 takes time at most O(n)Total time: 2n * O(n) = O(n 2n)This is an exponential time algorithm!
Lower Bound: Since there 2n subsets; Just printing them will take at least 2n time!
Hon Wai Leong, SoC, NUS(Creative Problem Solving) Page 20
Copyright © by Leong Hon Wai
One Final Question ?
LeongHW, SoC, NUS(UIT2201: AI) Page 21© Leong Hon Wai, 2003-2008
Exponential Time Problems… Intel designs CPU chips for most of todays’
computers (IBM-compatible, Macs, Linux, etc) If Intel wants to thoroughly verify their
multiplication circuit is correct (i.e. produces the correct product for all input combinations), what does it need to do?
Check that for every a, b compute and verify the product (a * b)
Do you think that Intel has actually done that for their 32-bit and 64-bit processors?