Active Learning a Convex Body in LowDimensions

Sariel Har-Peled, Mitchell Jones and Saladi RahulSoCG ’19 (YRF), June 18-21, 2019

University of Illinois at Urbana-Champaign

1/13

An innocent problem

ProblemInput: P ⊂ R2, oracle for unknown convex body C.

Oracle: Query q ∈ R2, returns true ⇐⇒ q ∈ C.

Goal: Compute P ∩ C using fewest number of oracle queries.

2/13

Remarks

I Active learningI Worst case: query all pointsI Question: In what model can we do better?

3/13

Modified problem

ProblemInput: P ⊂ R2, oracle for unknown convex body C.

Oracle: Separation oracle

q

C q C

`

Goal: Compute P ∩ C using fewest number of oracle queries.

4/13

Motivation

I Slighter stronger model

I Separation oracles are well-known (OR)I Other models previously studied [Angluin, 1987] [Panahi,Adler, et al., 2013] [Har-Peled, Kumar, et al., 2016]

5/13

Motivation

I Slighter stronger modelI Separation oracles are well-known (OR)

I Other models previously studied [Angluin, 1987] [Panahi,Adler, et al., 2013] [Har-Peled, Kumar, et al., 2016]

5/13

Motivation

I Slighter stronger modelI Separation oracles are well-known (OR)I Other models previously studied [Angluin, 1987] [Panahi,Adler, et al., 2013] [Har-Peled, Kumar, et al., 2016]

5/13

PAC learning

I Allow error in classificationI Random sampling

I C has bounded complexity =⇒ finite VC dimension =⇒random sample of size ≈ O(ε−1 log ε−1) =⇒ εn error

I Scheme fails for arbitrary convex regions

6/13

PAC learning

I Allow error in classificationI Random samplingI C has bounded complexity =⇒ finite VC dimension =⇒random sample of size ≈ O(ε−1 log ε−1) =⇒ εn error

I Scheme fails for arbitrary convex regions

6/13

PAC learning

I Allow error in classificationI Random samplingI C has bounded complexity =⇒ finite VC dimension =⇒random sample of size ≈ O(ε−1 log ε−1) =⇒ εn error

I Scheme fails for arbitrary convex regions

6/13

Hard vs. easy instances

I Worst case: query all points

I Goal: design instance sensitive algorithms

7/13

Hard vs. easy instances

I Worst case: query all pointsI Goal: design instance sensitive algorithms

7/13

A lower bound

I Fin = convex polygon with fewest vertices s.t. Fin ⊆ C andC ∩ P = Fin ∩ P.

I Fout = convex polygon with fewest vertices s.t. C ⊆ Fout

and C ∩ P = Fout ∩ P.I Separation price σ(P, C) = |Fin|+ |Fout|.

LemmaAny algorithm must make at least σ(P, C) oracle queries.

8/13

A lower bound

I Fin = convex polygon with fewest vertices s.t. Fin ⊆ C andC ∩ P = Fin ∩ P.

I Fout = convex polygon with fewest vertices s.t. C ⊆ Fout

and C ∩ P = Fout ∩ P.

I Separation price σ(P, C) = |Fin|+ |Fout|.

LemmaAny algorithm must make at least σ(P, C) oracle queries.

8/13

A lower bound

I Fin = convex polygon with fewest vertices s.t. Fin ⊆ C andC ∩ P = Fin ∩ P.

I Fout = convex polygon with fewest vertices s.t. C ⊆ Fout

and C ∩ P = Fout ∩ P.I Separation price σ(P, C) = |Fin|+ |Fout|.

LemmaAny algorithm must make at least σ(P, C) oracle queries.

8/13

A lower bound

I Fin = convex polygon with fewest vertices s.t. Fin ⊆ C andC ∩ P = Fin ∩ P.

I Fout = convex polygon with fewest vertices s.t. C ⊆ Fout

and C ∩ P = Fout ∩ P.I Separation price σ(P, C) = |Fin|+ |Fout|.

LemmaAny algorithm must make at least σ(P, C) oracle queries.

8/13

Results

Problem Lowerbound Upperbound

Classify (2D) σ(P, C) O(k(P) log n) (†)

Classify (2D) σ(P, C) O(σ(P, C) log2 n)

Classify (3D) — O(k(P) log n) (†)

Verify in (2D) |Fin| O(|Fin| log n)

Verify out (2D) |Fout| O(|Fout| log n) (‡)

(†) k(P) = largest # of pts of P in convex position

(‡) Randomized, w.h.p

9/13

Results

Problem Lowerbound Upperbound

Classify (2D) σ(P, C) O(k(P) log n) (†)

Classify (2D) σ(P, C) O(σ(P, C) log2 n)

Classify (3D) — O(k(P) log n) (†)

Verify in (2D) |Fin| O(|Fin| log n)

Verify out (2D) |Fout| O(|Fout| log n) (‡)

(†) k(P) = largest # of pts of P in convex position

(‡) Randomized, w.h.p

9/13

Results

Problem Lowerbound Upperbound

Classify (2D) σ(P, C) O(k(P) log n) (†)

Classify (2D) σ(P, C) O(σ(P, C) log2 n)

Classify (3D) — O(k(P) log n) (†)

Verify in (2D) |Fin| O(|Fin| log n)

Verify out (2D) |Fout| O(|Fout| log n) (‡)

(†) k(P) = largest # of pts of P in convex position

(‡) Randomized, w.h.p

9/13

Results

Problem Lowerbound Upperbound

Classify (2D) σ(P, C) O(k(P) log n) (†)

Classify (2D) σ(P, C) O(σ(P, C) log2 n)

Classify (3D) — O(k(P) log n) (†)

Verify in (2D) |Fin| O(|Fin| log n)

Verify out (2D) |Fout| O(|Fout| log n) (‡)

(†) k(P) = largest # of pts of P in convex position

(‡) Randomized, w.h.p

9/13

Results

Problem Lowerbound Upperbound

Classify (2D) σ(P, C) O(k(P) log n) (†)

Classify (2D) σ(P, C) O(σ(P, C) log2 n)

Classify (3D) — O(k(P) log n) (†)

Verify in (2D) |Fin| O(|Fin| log n)

Verify out (2D) |Fout| O(|Fout| log n) (‡)

(†) k(P) = largest # of pts of P in convex position(‡) Randomized, w.h.p

9/13

Results

Problem Lowerbound Upperbound

Classify (2D) σ(P, C) O(k(P) log n) (†)

Classify (2D) σ(P, C) O(σ(P, C) log2 n)

Classify (3D) — O(k(P) log n) (†)

Verify in (2D) |Fin| O(|Fin| log n)

Verify out (2D) |Fout| O(|Fout| log n) (‡)

(†) k(P) = largest # of pts of P in convex position(‡) Randomized, w.h.p

9/13

The greedy algorithm: preliminaries

I Maintain approximation B ⊆ C

I Operations:

1. expand(p): Update B = CH(B+ p)2. remove(`+): Classify points P ∩ `+ as outside C

I c ∈ R2 is a centerpoint for P if for all halfspaces `+:c ∈ `+ =⇒ |P ∩ `+| > |P|/3.

10/13

The greedy algorithm: preliminaries

I Maintain approximation B ⊆ CI Operations:

1. expand(p): Update B = CH(B+ p)2. remove(`+): Classify points P ∩ `+ as outside C

I c ∈ R2 is a centerpoint for P if for all halfspaces `+:c ∈ `+ =⇒ |P ∩ `+| > |P|/3.

10/13

The greedy algorithm: preliminaries

I Maintain approximation B ⊆ CI Operations:

1. expand(p): Update B = CH(B+ p)2. remove(`+): Classify points P ∩ `+ as outside C

I c ∈ R2 is a centerpoint for P if for all halfspaces `+:c ∈ `+ =⇒ |P ∩ `+| > |P|/3.

C

Bp

`+

C

B

10/13

The greedy algorithm: preliminaries

I Maintain approximation B ⊆ CI Operations:

1. expand(p): Update B = CH(B+ p)2. remove(`+): Classify points P ∩ `+ as outside C

I c ∈ R2 is a centerpoint for P if for all halfspaces `+:c ∈ `+ =⇒ |P ∩ `+| > |P|/3.

C

Bp

`+

C

B

10/13

The greedy algorithm

U ⊆ P unclassified points. While U 6= ∅:

1. `+ = halfspace tangent to B maximizing |`+ ∩ U|2. c= centerpoint of `+ ∩ U3. Query oracle using c:

(A) c ∈ C =⇒ expand(c)(B) c 6∈ C, h is a separating line =⇒ remove(h)

11/13

The greedy algorithm

U ⊆ P unclassified points. While U 6= ∅:

1. `+ = halfspace tangent to B maximizing |`+ ∩ U|

2. c= centerpoint of `+ ∩ U3. Query oracle using c:

(A) c ∈ C =⇒ expand(c)(B) c 6∈ C, h is a separating line =⇒ remove(h)

11/13

The greedy algorithm

U ⊆ P unclassified points. While U 6= ∅:

1. `+ = halfspace tangent to B maximizing |`+ ∩ U|

2. c= centerpoint of `+ ∩ U3. Query oracle using c:

(A) c ∈ C =⇒ expand(c)(B) c 6∈ C, h is a separating line =⇒ remove(h)

11/13

The greedy algorithm

U ⊆ P unclassified points. While U 6= ∅:

1. `+ = halfspace tangent to B maximizing |`+ ∩ U|2. c= centerpoint of `+ ∩ U

3. Query oracle using c:

(A) c ∈ C =⇒ expand(c)(B) c 6∈ C, h is a separating line =⇒ remove(h)

11/13

The greedy algorithm

U ⊆ P unclassified points. While U 6= ∅:

1. `+ = halfspace tangent to B maximizing |`+ ∩ U|2. c= centerpoint of `+ ∩ U

3. Query oracle using c:

(A) c ∈ C =⇒ expand(c)(B) c 6∈ C, h is a separating line =⇒ remove(h)

11/13

The greedy algorithm

U ⊆ P unclassified points. While U 6= ∅:

1. `+ = halfspace tangent to B maximizing |`+ ∩ U|2. c= centerpoint of `+ ∩ U3. Query oracle using c:

(A) c ∈ C =⇒ expand(c)(B) c 6∈ C, h is a separating line =⇒ remove(h)

11/13

The greedy algorithm

U ⊆ P unclassified points. While U 6= ∅:

1. `+ = halfspace tangent to B maximizing |`+ ∩ U|2. c= centerpoint of `+ ∩ U3. Query oracle using c:

(A) c ∈ C =⇒ expand(c)

(B) c 6∈ C, h is a separating line =⇒ remove(h)

11/13

The greedy algorithm

U ⊆ P unclassified points. While U 6= ∅:

1. `+ = halfspace tangent to B maximizing |`+ ∩ U|2. c= centerpoint of `+ ∩ U3. Query oracle using c:

(A) c ∈ C =⇒ expand(c)(B) c 6∈ C, h is a separating line =⇒ remove(h)

11/13

The greedy algorithm

U ⊆ P unclassified points. While U 6= ∅:

1. `+ = halfspace tangent to B maximizing |`+ ∩ U|2. c= centerpoint of `+ ∩ U3. Query oracle using c:

(A) c ∈ C =⇒ expand(c)(B) c 6∈ C, h is a separating line =⇒ remove(h)

11/13

Analysis sketch

I Count visible pairs of points

I In each iteration:

(A) Pairs lose visibility(B) Classify points

Our resultThe greedy algorithm usesO(k(P) log n) queries.

(k(P) = largest # of pts of P in convexposition.)

B

12/13

Analysis sketch

I Count visible pairs of pointsI In each iteration:

(A) Pairs lose visibility(B) Classify points

Our resultThe greedy algorithm usesO(k(P) log n) queries.

(k(P) = largest # of pts of P in convexposition.)

B

12/13

Analysis sketch

I Count visible pairs of pointsI In each iteration:

(A) Pairs lose visibility

(B) Classify points

Our resultThe greedy algorithm usesO(k(P) log n) queries.

(k(P) = largest # of pts of P in convexposition.)

B

12/13

Analysis sketch

I Count visible pairs of pointsI In each iteration:

(A) Pairs lose visibility(B) Classify points

Our resultThe greedy algorithm usesO(k(P) log n) queries.

(k(P) = largest # of pts of P in convexposition.)

B

12/13

Analysis sketch

I Count visible pairs of pointsI In each iteration:

(A) Pairs lose visibility(B) Classify points

Our resultThe greedy algorithm usesO(k(P) log n) queries.

(k(P) = largest # of pts of P in convexposition.)

B

12/13

Conclusion & open problems

Problem Lowerbound Upperbound

Classify (2D) σ(P, C)O(k(P) log n)

O(σ(P, C) log2 n)Classify (3D) — O(k(P) log n)Verify in |Fin| O(|Fin| log n)Verify out |Fout| O(|Fout| log n)

I Shaving log factors?I Near-optimal solution in 3D?I Higher dimensions?I Conjecture: Greedy extends to Rd using O(k(P)bd/2c log n)queries (only interesting when k(P) � ( n

log n)2/d)

13/13

Conclusion & open problems

Problem Lowerbound Upperbound

Classify (2D) σ(P, C)O(k(P) log n)

O(σ(P, C) log2 n)Classify (3D) — O(k(P) log n)Verify in |Fin| O(|Fin| log n)Verify out |Fout| O(|Fout| log n)

I Shaving log factors?

I Near-optimal solution in 3D?I Higher dimensions?I Conjecture: Greedy extends to Rd using O(k(P)bd/2c log n)queries (only interesting when k(P) � ( n

log n)2/d)

13/13

Conclusion & open problems

Problem Lowerbound Upperbound

Classify (2D) σ(P, C)O(k(P) log n)

O(σ(P, C) log2 n)Classify (3D) — O(k(P) log n)Verify in |Fin| O(|Fin| log n)Verify out |Fout| O(|Fout| log n)

I Shaving log factors?I Near-optimal solution in 3D?

I Higher dimensions?I Conjecture: Greedy extends to Rd using O(k(P)bd/2c log n)queries (only interesting when k(P) � ( n

log n)2/d)

13/13

Conclusion & open problems

Problem Lowerbound Upperbound

Classify (2D) σ(P, C)O(k(P) log n)

O(σ(P, C) log2 n)Classify (3D) — O(k(P) log n)Verify in |Fin| O(|Fin| log n)Verify out |Fout| O(|Fout| log n)

I Shaving log factors?I Near-optimal solution in 3D?I Higher dimensions?

I Conjecture: Greedy extends to Rd using O(k(P)bd/2c log n)queries (only interesting when k(P) � ( n

log n)2/d)

13/13

Conclusion & open problems

Problem Lowerbound Upperbound

Classify (2D) σ(P, C)O(k(P) log n)

O(σ(P, C) log2 n)Classify (3D) — O(k(P) log n)Verify in |Fin| O(|Fin| log n)Verify out |Fout| O(|Fout| log n)

I Shaving log factors?I Near-optimal solution in 3D?I Higher dimensions?I Conjecture: Greedy extends to Rd using O(k(P)bd/2c log n)queries (only interesting when k(P) � ( n

log n)2/d)

13/13

References i

D. Angluin. Queries and concept learning. Machine Learning,2(4): 319–342, 1987.

F. Panahi, A. Adler, A. F. van der Stappen, and K. Goldberg. Anefficient proximity probing algorithm for metrology. Int. Conf.on Automation Science and Engineering, CASE 2013, 342–349,2013.

S. Har-Peled, N. Kumar, D. M. Mount, and B. Raichel. Spaceexploration via proximity search. Discrete Comput. Geom., 56(2):357–376, 2016.