finding hidden cliques - university of california, san...

IntroductionAlgorithm A - Finding the Hidden Clique

The ProofGeneral Case

Finding Hidden Cliques

Craig Timmons

Craig Timmons Finding Hidden Cliques

Outline

1 Introduction

2 Algorithm A - Finding the Hidden Clique

3 The Proof

4 General Case

Outline

1 Introduction

3 The Proof

4 General Case

Outline

1 Introduction

3 The Proof

4 General Case

Outline

1 Introduction

3 The Proof

4 General Case

A hard problem

ω(G) = clique number of G

Unless NP has randomized polynomial time algorithms, forany fixed δ ∈ (0,1), there is no algorithm C that given anyn vertex graph G, C approximates ω(G) within a factor ofnδ (see Håstad, Clique is hard to approximate within n1−ε,1999).

A hard problem

ω(G) = clique number of G

Unless NP has randomized polynomial time algorithms, forany fixed δ ∈ (0,1), there is no algorithm C that given anyn vertex graph G, C approximates ω(G) within a factor ofnδ (see Håstad, Clique is hard to approximate within n1−ε,1999).

ω in G(n,1/2)

Random graphs:

Property P holds almost surely (a.s.) iflim

n→∞P(G(n,p) has P) = 1

ω(G(n,1/2)) ≈ (2 + o(1)) log2 n (a.s.)

∃ poly. time alg. that find a.s. clique of size (1 + o(1)) log2 nin G(n,1/2)

ω in G(n,1/2)

Random graphs:

ω(G(n,1/2)) ≈ (2 + o(1)) log2 n (a.s.)

ω in G(n,1/2)

Random graphs:

ω(G(n,1/2)) ≈ (2 + o(1)) log2 n (a.s.)

ω in G(n,1/2)

Random graphs:

ω(G(n,1/2)) ≈ (2 + o(1)) log2 n (a.s.)

Add big clique

An easier problem:

Define a probability space G(n,1/2, k):

Choose r.g. G ∈ G(n,1/2)

Randomly choose k -subset Q of V (G)

Make Q a clique

Add big clique

An easier problem:

Make Q a clique

Add big clique

An easier problem:

Make Q a clique

Add big clique

An easier problem:

Make Q a clique

Add big clique

An easier problem:

Make Q a clique

G(n,1/2, k)

k > c√

n log n⇒ vertices of clique are a.s. the vertices oflargest degree

What if k = o(√

n log n)?

G(n,1/2, k)

k > c√

n log n⇒ vertices of clique are a.s. the vertices oflargest degree

What if k = o(√

n log n)?

Main Result

Theorem (Alon, Krivelevich, Sudakov, 1998)For each ε > 0 there exists a polynomial time algorithm thatfinds a.s. the unique largest clique of size k in G(n,1/2, k) ifk ≥ εn1/2.

Notation

V = {1, . . . ,n}

A adjacency matrix

µ1 ≥ · · · ≥ µn eigenvalues of A

v1, . . . , vn orthonormal basis of eigenvectors

Assume k = o(√

n log n)

Notation

V = {1, . . . ,n}

A adjacency matrix

Assume k = o(√

n log n)

Notation

V = {1, . . . ,n}

A adjacency matrix

Assume k = o(√

n log n)

Notation

V = {1, . . . ,n}

A adjacency matrix

Assume k = o(√

n log n)

Notation

V = {1, . . . ,n}

A adjacency matrix

Assume k = o(√

n log n)

The Algorithm

Algorithm A

Input: A graph G

1 Find second eigenvector v2 of G

2 Sort V by decreasing order in magnitude using thecoordinates of v2 (ties broken arbitrarily). Let W be the firstk vertices in this order.

3 Let Q0 be set of all vertices with at least 3k4 neighbors in W

4 Output: Q0

The Algorithm

Algorithm A

Input: A graph G

4 Output: Q0

The Algorithm

Algorithm A

Input: A graph G

4 Output: Q0

The Algorithm

Algorithm A

Input: A graph G

4 Output: Q0

The Algorithm

Algorithm A

Input: A graph G

4 Output: Q0

Case k ≥ 10√

First we prove a special case:

If input G from G(n,1/2, k), k ≥ 10√

n into Algorithm A,then a.s. output Q0 is the hidden clique.

Case k ≥ 10√

First we prove a special case:

If input G from G(n,1/2, k), k ≥ 10√

n into Algorithm A,then a.s. output Q0 is the hidden clique.

Main Idea

Assume hidden clique Q = {1, . . . , k}

MAIN IDEA: v2 has most of its weight on QLet

{n − k 1 ≤ i ≤ k−k k + 1 ≤ i ≤ n

We prove: v2 close to z = (z1, . . . , zn)

Main Idea

MAIN IDEA: v2 has most of its weight on Q

{n − k 1 ≤ i ≤ k−k k + 1 ≤ i ≤ n

Main Idea

{n − k 1 ≤ i ≤ k−k k + 1 ≤ i ≤ n

Main Idea

{n − k 1 ≤ i ≤ k−k k + 1 ≤ i ≤ n

Main Lemma

A.s. there exists a δ such that ‖δ‖2 ≤ 160‖z‖

2 and c2v2 = z − δfor some c2.

Conclusion: v2 looks like z

z is big on Q, small on V −Q

v2 has found Q (Good job v2!)

Main Lemma

Proving Main Lemma

Let z = c1v1 + · · ·+ cnvn

Show c1, c3, . . . , cn small in comparison to ‖z‖

1 Sorting Lemma

2 Technical Lemma

Set δ = c1v1 + c3v3 + · · ·+ cnvn

Proving Main Lemma

Let z = c1v1 + · · ·+ cnvn

1 Sorting Lemma

2 Technical Lemma

Set δ = c1v1 + c3v3 + · · ·+ cnvn

Proving Main Lemma

Let z = c1v1 + · · ·+ cnvn

1 Sorting Lemma

2 Technical Lemma

Set δ = c1v1 + c3v3 + · · ·+ cnvn

Proving Main Lemma

Let z = c1v1 + · · ·+ cnvn

1 Sorting Lemma

2 Technical Lemma

Set δ = c1v1 + c3v3 + · · ·+ cnvn

Proving Main Lemma

Let z = c1v1 + · · ·+ cnvn

1 Sorting Lemma

2 Technical Lemma

Set δ = c1v1 + c3v3 + · · ·+ cnvn

Sorting Lemma

G ∈ G(n,1/2, k), k = o(n) then a.s. µ1 ≥ · · · ≥ µn satisfy(i) µ1 ≥ (1/2 + o(1))n

(ii) µi ≤ (1 + o(1))√

n for all i ≥ 3.

(i) µ1 ≥ average degree = (1/2 + o(1))n a.s.

Sorting Lemma

G ∈ G(n,1/2, k), k = o(n) then a.s. µ1 ≥ · · · ≥ µn satisfy(i) µ1 ≥ (1/2 + o(1))n

(ii) µi ≤ (1 + o(1))√

n for all i ≥ 3.

(i) µ1 ≥ average degree = (1/2 + o(1))n a.s.

Proof of Sorting Lemma (ii)

For (ii) need:

Theorem (Füredi, Komlós)

If µ1 ≥ · · · ≥ µm eigenvalues of r.g. G ∈ G(m,1/2) then a.s.maxi≥2 |µi | ≤

√m + O(m1/3 log m)

Write G = G1 ∪G2 (edge disjoint union)

G2 ∈ G(k ,1/2) is r.g. on Q

G1 = (V (G),E(G)− E(G2))

Observe G1 ∈ G(n,1/2)

For (ii) need:

√m + O(m1/3 log m)

G2 ∈ G(k ,1/2) is r.g. on Q

G1 = (V (G),E(G)− E(G2))

For (ii) need:

√m + O(m1/3 log m)

G2 ∈ G(k ,1/2) is r.g. on Q

G1 = (V (G),E(G)− E(G2))

For (ii) need:

√m + O(m1/3 log m)

G2 ∈ G(k ,1/2) is r.g. on Q

G1 = (V (G),E(G)− E(G2))

For (ii) need:

√m + O(m1/3 log m)

G2 ∈ G(k ,1/2) is r.g. on Q

G1 = (V (G),E(G)− E(G2))

For (ii) need:

√m + O(m1/3 log m)

G2 ∈ G(k ,1/2) is r.g. on Q

G1 = (V (G),E(G)− E(G2))

Ai = adj. matrix for Gi (add 0 row/col. to A2 to make n × n)

Note A = A1 + A2

Let λi = largest eigenvalue of Ai with eigenvector vi

Let S = (span(v1, v2))⊥

Recall µi = mindimS=n−i+1

maxx∈S,x 6=0

xT AxxT x

Note A = A1 + A2

maxx∈S,x 6=0

xT AxxT x

Note A = A1 + A2

maxx∈S,x 6=0

xT AxxT x

Note A = A1 + A2

maxx∈S,x 6=0

xT AxxT x

Note A = A1 + A2

maxx∈S,x 6=0

xT AxxT x

Apply Füredi, Komlós to get a.s. for all x ∈ S, x 6= 0:

1 xT A1xxT x ≤ (1 + o(1))

2 xT A2xxT x ≤ (1 + o(1))

⇒ xT AxxT x = xT A1x

xT x + xT A2xxT x ≤ (1 + o(1))

√n (k = o(n))

dim(S) ≥ n − 2⇒ µi ≤ (1 + o(1))√

n for all i ≥ 3

1 xT A1xxT x ≤ (1 + o(1))

2 xT A2xxT x ≤ (1 + o(1))

xT x + xT A2xxT x ≤ (1 + o(1))

√n (k = o(n))

dim(S) ≥ n − 2⇒ µi ≤ (1 + o(1))√

n for all i ≥ 3

1 xT A1xxT x ≤ (1 + o(1))

2 xT A2xxT x ≤ (1 + o(1))

xT x + xT A2xxT x ≤ (1 + o(1))

√n (k = o(n))

dim(S) ≥ n − 2⇒ µi ≤ (1 + o(1))√

n for all i ≥ 3

1 xT A1xxT x ≤ (1 + o(1))

2 xT A2xxT x ≤ (1 + o(1))

xT x + xT A2xxT x ≤ (1 + o(1))

√n (k = o(n))

dim(S) ≥ n − 2⇒ µi ≤ (1 + o(1))√

n for all i ≥ 3

1 xT A1xxT x ≤ (1 + o(1))

2 xT A2xxT x ≤ (1 + o(1))

xT x + xT A2xxT x ≤ (1 + o(1))

√n (k = o(n))

dim(S) ≥ n − 2⇒ µi ≤ (1 + o(1))√

n for all i ≥ 3

A Technical Lemma

Technical Lemma

‖(A− k2 I)z‖2 ≤ (1

4 + o(1))n3k a.s.

Let (A− k2 I)z = (t1, . . . , tn)

{(k/2− 1)(n − k)− kYi 1 ≤ i ≤ kk2/2 + (n − k)Xi − kYi k + 1 ≤ i ≤ n

Xi ,Yi binomial r.v.’s, use standard estimates

A Technical Lemma

Technical Lemma

‖(A− k2 I)z‖2 ≤ (1

4 + o(1))n3k a.s.

Let (A− k2 I)z = (t1, . . . , tn)

A Technical Lemma

Technical Lemma

‖(A− k2 I)z‖2 ≤ (1

4 + o(1))n3k a.s.

Let (A− k2 I)z = (t1, . . . , tn)

A Technical Lemma

Technical Lemma

‖(A− k2 I)z‖2 ≤ (1

4 + o(1))n3k a.s.

Let (A− k2 I)z = (t1, . . . , tn)

Proof of Main Lemma

Main Lemma

A.s. exists a δ with ‖δ‖2 ≤ 160‖z‖

2 such that z − δ = c2v2 forsome c2.

Let z = c1v1 + · · ·+ cnvn

‖(A− k2 I)z‖2 =

∑ni=1 c2

i (µi − k2 )2

Recall Sorting Lemma:µ1 ≥ (1

2 + o(1))n, (1 + o(1))√

n ≥ µi ∀i ≥ 3

⇒ ‖(A− k2 I)z‖2 ≥ (1 + o(1))(k

2 −√

n)2∑i 6=2 c2

Proof of Main Lemma

Main Lemma

Let z = c1v1 + · · ·+ cnvn

‖(A− k2 I)z‖2 =

∑ni=1 c2

i (µi − k2 )2

2 + o(1))n, (1 + o(1))√

n ≥ µi ∀i ≥ 3

⇒ ‖(A− k2 I)z‖2 ≥ (1 + o(1))(k

2 −√

n)2∑i 6=2 c2

Proof of Main Lemma

Main Lemma

Let z = c1v1 + · · ·+ cnvn

‖(A− k2 I)z‖2 =

∑ni=1 c2

i (µi − k2 )2

2 + o(1))n, (1 + o(1))√

n ≥ µi ∀i ≥ 3

⇒ ‖(A− k2 I)z‖2 ≥ (1 + o(1))(k

2 −√

n)2∑i 6=2 c2

Proof of Main Lemma

Main Lemma

Let z = c1v1 + · · ·+ cnvn

‖(A− k2 I)z‖2 =

∑ni=1 c2

i (µi − k2 )2

2 + o(1))n, (1 + o(1))√

n ≥ µi ∀i ≥ 3

⇒ ‖(A− k2 I)z‖2 ≥ (1 + o(1))(k

2 −√

n)2∑i 6=2 c2

Proof of Main Lemma

Main Lemma

Let z = c1v1 + · · ·+ cnvn

‖(A− k2 I)z‖2 =

∑ni=1 c2

i (µi − k2 )2

2 + o(1))n, (1 + o(1))√

n ≥ µi ∀i ≥ 3

⇒ ‖(A− k2 I)z‖2 ≥ (1 + o(1))(k

2 −√

n)2∑i 6=2 c2

Proof of Main Lemma

⇒∑i 6=2

c2i ≤‖(A− k

2 I)z‖2

(k/2−√

n)2(1 + o(1))

Technical Lemma, k ≥ 10√

n, and k = o(n):

‖δ‖2 =∑i 6=2

c2i ≤

n3k(k − 2

√n)2

(1 + o(1)) <kn(n − k)

160‖z‖2

Note c2v2 = z − δ

Proof of Main Lemma

⇒∑i 6=2

c2i ≤‖(A− k

2 I)z‖2

(k/2−√

n)2(1 + o(1))

n, and k = o(n):

‖δ‖2 =∑i 6=2

c2i ≤

n3k(k − 2

√n)2

(1 + o(1)) <kn(n − k)

160‖z‖2

Proof of Main Lemma

⇒∑i 6=2

c2i ≤‖(A− k

2 I)z‖2

(k/2−√

n)2(1 + o(1))

n, and k = o(n):

‖δ‖2 =∑i 6=2

c2i ≤

n3k(k − 2

√n)2

(1 + o(1)) <kn(n − k)

160‖z‖2

Proof of Main Lemma

⇒∑i 6=2

c2i ≤‖(A− k

2 I)z‖2

(k/2−√

n)2(1 + o(1))

n, and k = o(n):

‖δ‖2 =∑i 6=2

c2i ≤

n3k(k − 2

√n)2

(1 + o(1)) <kn(n − k)

160‖z‖2

A Corollary

Corollary

k2 −

√n2 ≤ µ2 ≤ k

2 +√

n2 a.s.

In particular, k ≥ 10√

n implies µ2 > µi for all i ≥ 3

‖z‖2 − ‖δ‖2 = c22 ≥

5960‖z‖

2 from Main Lemma

(14 + o(1))n3k ≥ c2

2(µ2 − k2 )2 ≥ 2

3kn2(µ2 − k2 )2

A Corollary

Corollary

k2 −

√n2 ≤ µ2 ≤ k

2 +√

n2 a.s.

‖z‖2 − ‖δ‖2 = c22 ≥

5960‖z‖

2 from Main Lemma

(14 + o(1))n3k ≥ c2

2(µ2 − k2 )2 ≥ 2

3kn2(µ2 − k2 )2

A Corollary

Corollary

k2 −

√n2 ≤ µ2 ≤ k

2 +√

n2 a.s.

‖z‖2 − ‖δ‖2 = c22 ≥

5960‖z‖

2 from Main Lemma

(14 + o(1))n3k ≥ c2

2(µ2 − k2 )2 ≥ 2

3kn2(µ2 − k2 )2

A Corollary

Corollary

k2 −

√n2 ≤ µ2 ≤ k

2 +√

n2 a.s.

‖z‖2 − ‖δ‖2 = c22 ≥

5960‖z‖

2 from Main Lemma

(14 + o(1))n3k ≥ c2

2(µ2 − k2 )2 ≥ 2

3kn2(µ2 − k2 )2

Apply Main Lemma

Let v2 = (a1, . . . ,an) be normalized 2nd eigenvector

Recall W = indices corresponding to largest k values of{|a1|, . . . , |an|}

Main Lemma⇒ c2v2 = z − δ, ‖δ‖2 ≤ 160kn2

Get same W if look at c2v2

Want: W ∩ [k ] big

Apply Main Lemma

Control error δ

‖δ‖2 ≤ 160kn2 ⇒ |{δi : |δi | > n/3}| ≤ k/6

k1 = number of δi ’s with |δi | > n/3, 1 ≤ i ≤ k

k2 = number of δi ’s with |δi | > n/3, k + 1 ≤ i ≤ n

k1 + k2 ≤ k/6

Control error δ

‖δ‖2 ≤ 160kn2 ⇒ |{δi : |δi | > n/3}| ≤ k/6

k1 + k2 ≤ k/6

Control error δ

‖δ‖2 ≤ 160kn2 ⇒ |{δi : |δi | > n/3}| ≤ k/6

k1 + k2 ≤ k/6

Control error δ

‖δ‖2 ≤ 160kn2 ⇒ |{δi : |δi | > n/3}| ≤ k/6

k1 + k2 ≤ k/6

W likes [k ]

c2v2 = (n− k − δ1, . . . ,n− k − δk ,−k − δk+1, . . . ,−k − δn)

v2 is "off" by at most k/6 entries from z

|W ∩ [k ]| ≥ 5k6

W likes [k ]

|W ∩ [k ]| ≥ 5k6

W likes [k ]

|W ∩ [k ]| ≥ 5k6

Q0 is the clique

Recall: Algorithm returns Q0 = vertices with at least 3k/4neighbors in W

Q ⊂ Q0 since |W ∩Q| ≥ 5k/6

If v /∈ Q:

1 v has at most (1 + o(1)) k2 neighbors in Q (a.s.)

2 v has at most k − 5k/6 = k/6 neighbors in W