Randomized Sensing in Adversarial Environments

Andreas Krause

Joint work with Daniel Golovin and Alex Roper

International Joint Conference on Artificial Intelligence 2011

Motivation

Want to manage sensing resources to enable robust monitoring under uncertainty

2

Roboticenvironmental

monitoring

Detectsurvivors after

disaster

Coordinatecameras to

detect intrusions

4

Select two cameras to query, in order to detect the most people.

3

People Detected:

2

Duplicates only counted

once

A Sensor Selection Problem

Set V of sensors, |V| = nSelect a set of k sensors Sensing quality model

NP-hard…

4

A Sensor Selection Problem

5

SubmodularityDiminishing returns property for adding more sensors.

Many objectives are submodular [K, Guestrin ‘07]Detection, coverage, mutual information, and others

+2

+1

For all , and a sensor ,

Greedy algorithm

6

Lets choose sensors S = {v1 , … , vk} greedily

[Nemhauser et al ‘78] If F is submodular, greedy algorithm gives constant factor approx.:

i Fi({3}) Fi({5})1 0 12 1 0

Sensing in Adversarial Environments

Set I of m intrusion scenariosFor scenario i: Fi(A) is sensing utility when selecting AIntruder chooses worst-case scenario, knowing the sensors

7

2

1

Deterministic minimax solution

One approach: Want to solve

[K, McMahan, Guestrin, Gupta ’08]:NP-hardGreedy algorithm fails arbitrarily badlySATURATE algorithm provides near-optimal solution

8

Disadvantage of minimax approach

Suppose we pick {3} and {5} with probability 1/2

Randomization can perform arbitrarily better!

9

1

2

i Fi({3}) Fi({5})1 1 02 0 1

The randomized sensing problemGiven submodular functions F1,…,Fm, want to find

NP-hard!

Even representing the optimal solution may require exponential space!

10

Existing approachesMany techniques for solving matrix games

Typically don’t scale to combinatorially large strategy sets

Security games [Tambe et al]Solve large scale Stackelberg games for security applicationsCannot capture general submodular objective functions

LP based approach [Halvorson et al ‘09]Double oracle with approximate best responseNo polynomial time convergence convergence guarantee

11

Randomized sensingDefine

12

Thus, can minimize over q instead of over p!

Distributionover sensing

actions

Distribution over intrusions

Equivalent problem: Finding q*Want to solve

Use multiplicative update algorithm [Freund & Schapire ‘99]

InitializeFor t = 1:T

13

NP-hard But submodular!

The RSENSE algorithmInitialize

For t=1:TUse greedy algorithm to compute

based on objective function

Update

Return

14

Performance guarantee

Theorem: Let Suppose RSENSE runs for iterations. For the resulting distribution it holds that

15

Handling more general constraintsSo far: wanted

Many application may require more complex constraints:

Examples:Informative path planning:Controlling PTZ cameras:Nonuniform cost:

Can replace greedy algorithm by - best response RSENSE guarantees

16

17

Example: Lake monitoringMonitor pH values using robotic sensor

Position s along transect

pH v

alue

Observations A

True (hidden) pH values

Prediction at unobservedlocations

transect

Where should we sense to minimize our maximum error?

Use probabilistic model(Gaussian processes)

to estimate prediction error

(often) submodular[Das & Kempe ’08]

Var(s | A)

Experimental results

Randomized sensing outperforms deterministic solutions 18

Running time

RSENSE outperforms existing LP based method 19

20

pSPIEL Results: Search & RescueMap from Robocup Research Challenge

Coordination of multiple mobile sensors to detect survivors of major urban disasterBuildings obstruct viewfield of cameraFi(A) = Expected # of people detected at location i

Detection Range

Detected Survivors

Experimental results

Randomization outperforms deterministic solutionRSENSE finds solution faster than existing methods

21

22

Worst- vs. average caseGiven: Possible locations V, submodular functions F1,…,Fm

Average-case score Worst-case score

Strong assumptions! Very pessimistic!

Want to optimize both average- and worst-case score!

Can modify RSENSE to solve this problem!Compute best response to

Tradeoff results

23

Wor

st c

ase

scor

e

Average case score

Knee intradeoff

curve

Search &rescue Wor

st c

ase

scor

eAverage case score

Envtl. monitoring

Can find good compromise between average- and worst-case score!

ConclusionsWish to find randomized strategy for maximizing an adversarially-chosen submodular functionDeveloped RSENSE, which provides near-optimal performancePerforms well on two real applications

Search and rescueEnvironmental monitoring

24