sequential decision aggregation: outlinemotion.me.ucsb.edu/talks/2010r-sda-13aug2010-2x2.pdf ·...

Sequential Decision Aggregation:Accuracy and Decision Time

Sandra H. Dandach, Ruggero Carli and Francesco Bullo

Center for Control,Dynamical Systems & Computation

University of California at Santa Barbara

http://motion.me.ucsb.edu

MURI FA95500710528 Project Review: Behavioral Dynamics inCooperative Control of Mixed Human/Robot Teams

Center for Human and Robot Decision Dynamics, Aug 13, 2010

Dandach, Carli, Bullo (UCSB) Sequential Decision Aggregation 13aug2010 1 / 16

Sequential decision aggregation: Outline

1 Setup & Literature Review

2 SDA: analysis of decision probabilities

3 SDA: scalability analysis of accuracy/decision time

4 Conclusions and future directions


Setup & Literature Review

Assumptions:

1 N identical individuals, arbitrary local rule

2 Independent information

3 Aggregation of individual decisions !"##$%&"'(")((

*$%+,+"',(

-( .( /( 0(

Group decision rule = SDA algorithm

q out of N rule: decision as soon as q nodes report concordant opinion

Fastest rule fastest node decides for network (q = 1)

Majority rule network agrees with majority decision (q = dN/2e)

Goal #1: characterize decision probabilities of SDAas function of: threshold and SDM decision probabilities

Goal #2: express accuracy & decision timeas function of: decision threshold × group size



Assumptions:




*$%+,+"',(

-( .( /( 0(









Assumptions:




*$%+,+"',(

-( .( /( 0(








Literature review #1

Distributed/decentralized detection

1 P. K. Varshney. Distributed Detection and Data Fusion. Signal Processing and DataFusion. Springer Verlag, 1996

2 V. V. Veeravalli, T. Basar, and H. V. Poor. Decentralized sequential detection withsensors performing sequential tests. Math Control, Signals & Systems, 7(4):292–305, 1994

3 J. N. Tsitsiklis. Decentralized detection. In H. V. Poor and J. B. Thomas, editors,Advances in Statistical Signal Processing, volume 2, pages 297–344, 1993

4 J.-F. Chamberland and V. V. Veeravalli. Decentralized detection in sensor networks. IEEETrans Signal Processing, 51(2):407–416, 2003

Social networks

1 D. Acemoglu, M. A. Dahleh, I. Lobel, and A. Ozdaglar. Bayesian learning in socialnetworks. Working Paper 14040, National Bureau of Economic Research, May 2008



For decentralized detection, with conditional independence of observations:

Tsitsiklis ’93: Bayesian decision problem with fusion center. For largenetworks identical local decision rules are asymptotically optimal

Varshney ’96: on non-identical decision rules with q out of N,1 threshold rules are optimal at the nodes levels2 finding optimal thresholds requires solving N + 2N equations

Varshney ’96: on optimal fusion rules for identical local decisions, “qout of N” is optimal at the fusion center level

Contributions today

arbitrary decision makers (rather than optimal local rules)

sequential aggregation (rather than “complete” aggregation)

scalability analysis of accuracy / decision time



For decentralized detection, with conditional independence of observations:

Tsitsiklis ’93: Bayesian decision problem with fusion center. For largenetworks identical local decision rules are asymptotically optimal

Varshney ’96: on non-identical decision rules with q out of N,1 threshold rules are optimal at the nodes levels2 finding optimal thresholds requires solving N + 2N equations

Varshney ’96: on optimal fusion rules for identical local decisions, “qout of N” is optimal at the fusion center level

Contributions today

arbitrary decision makers (rather than optimal local rules)

sequential aggregation (rather than “complete” aggregation)

scalability analysis of accuracy / decision time


Today’s Outline






Model of sequential decision maker

Sequential decision maker (SDM)

pi |j(t) := Probability “say Hi given Hj” at time t

pi |j =+∞∑t=1

pi |j(t), E [T |Hi ] =+∞∑t=1

t(p1|i (t) + p0|i (t)

)

Assume knowledge of {pi |j(t)}t∈N for individual SDM,known exactly, calculated numerically, or measured empirically

0 5 10 15 20 25 30 35 40 45 500

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

Number of observations (t)

p 1|1(t)

pi|j(t) for a Gaussian distribution

!"""#$%%#&'(#)**+,-.#/*0'1+2#

3*4*56*1#778#9::;8#<*=#>1?*@-28#AB#

#

#

#

# CD

!"#$%&$'&()"*&+$"

%+@1+,-.#=,+E#0@0*12#2F65,++*G#,-#7HHH8#=*#@1*#I**0,-.#+1@4I#'J#+E*#1*K,*=,-.#+,5*#L+,5*#6*+=**-#2F65,22,'-#G@+*#@-G#G@+*#

=E*1*#@#0F6?,4@+,'-#G*4,2,'-#,2#5@G*MN#OE*#@K*1@.*#1*K,*=,-.#+,5*#J'1#JF??#0@0*12#E@2#6**-#4'-2,2+*-+?P#@001'Q,5@+*?P#C#

5'-+E2N#R'1#0@0*12#2F65,++*G#,-",--.#+E*#1*K,*=,-.#+,5*#E,2+'.1@5#,2#2E'=-#6*?'=N#OE*#@K*1@.*#1*K,*=,-.#+,5*#1*5@,-2#@+#

@001'Q,5@+*?P#C#5'-+E2N#!+#,2#-'+*='1+EP#+E@+#@#?@1.*#-F56*1#'J#0@0*12#@1*#-'=#01'4*22*G#=,+E,-#+E*,1#J,12+#5'-+E#J1'5#

2F65,22,'-#F-G*1#+E*#S,-@001'01,@+*#4@+*.'1PTN#)@-P#'J#+E*5#@1*#*K*-+F@??P#1*2F65,++*G#@2#O*4E-,4@?#<'+*2#L2**#@?2'#

SUF6?,4@+,'-#%+@+,2+,42T#,-#+E,2#1*0'1+MN#OE*#?@1.*#-F56*1#'J#0@0*12#,-#+E,2#4@+*.'1P#,2#@#1*?@+,K*?P#-*=#0E*-'5*-'-#L?@2+#+='#

P*@12MN#

#

!""#$%&'(&)$*(+&$,-.-

/0/0"#$1$/!02/0"#

!

"!

#!

$!

%!

&!!

&"!

'&( &)"( ")*( *)#( #)+( +)$( $),( ,)%( %)-( -)

&!(

&!)

&&(

&&)

&"(

&")

&*(

.&"

##

#

Hz) and an elevated firing state (30 Hz) in a network of binaryunits (Fig. 2C). This mechanism works well when the occur-rence of each transition is equally probable at an arbitrary timepoint during a delay period. The consecutive rate distributions

of the different graded-activity types exhibited quite differentprofiles (Fig. 2, E and F). Most characteristically, the distribu-tion obtained from the stepwise rate changes in single neuronsexhibits a trough near the peak of the distribution obtainedfrom the truly graded rate changes. Thus the rate distributionenables us to examine which type of graded activity givenspike trains are more likely to represent.

Graded activity in recurrent neural networks

We then constructed a recurrent network consisting of 500excitatory neurons and 100 inhibitory neurons (see METHODS).In the network model, excitatory neurons receive excitatoryand inhibitory recurrent synaptic inputs, excitatory and inhib-itory background synaptic inputs, and an external input toinduce graded activity. Inhibitory neurons receive synapticinput from excitatory neurons as well as excitatory and inhib-itory background synaptic inputs. Each excitatory neuronprojects to 10% of randomly chosen other excitatory neuronsand to all inhibitory neurons, whereas each inhibitory neuronprojects to all excitatory neurons, but not to other inhibitoryneurons. We note that the temporal integration performancewas relatively independent of the connectivity of synapses. In

FIG. 2. Comparison between different temporal integration mechanisms. A:graded activity may be modeled as a trial- or an ensemble-average of graduallyincreasing firing rates of individual neurons. B: climbing activity (bottom) wasconstructed from nonstationary Poisson spike trains with a gradually increas-ing mean firing rate (top). C: graded activity in our model consists oftemporally organized bimodal transitions between the baseline and elevatedfiring states. In the individual neurons, the transitions should occur at arbitrarytemporal positions with equal probabilities. Both trial average and ensembleaverage give equally good representations of graded activity in the presentmodel. D: climbing activity (bottom) was constructed from artificial bimodalPoisson spike trains showing stepwise increases in the mean firing rate (top).E: consecutive firing-rate distribution (see METHODS) exhibits a single peak inthe climbing activity shown in B. F: by contrast, the firing-rate distribution isbimodal in the climbing activity shown in D.

FIG. 1. Bimodal firing states of model excitatory neuron. A: responses of asingle excitatory neuron to a brief stimulus are shown in the absence of recurrentsynaptic inputs and the fluctuating components of background synaptic inputs (the“frozen” condition). External input was set as Iext ! 0 nA (top), for which theresponse was not bistable, and Iext ! 0.025 nA (bottom), for which the responsewas bistable. In the latter case, neuronal firing was terminated by a hyperpolarizinginput. Horizontal bars show the duration of the stimuli. B: model neuron withbistability repeats noise-driven transitions between the baseline and elevated firingstates under the influences of continuous synaptic bombardments (top). Monitor-ing the intracellular calcium density enables us to distinguish the epochs of theelevated firing state (bottom, gray shades). C: presence of the 2 distinct firing statesresults in a bimodal consecutive firing-rate distribution. D: bimodal firing-ratedistribution is shown at Iext ! 0.035 nA.

3862 H. OKAMOTO, Y. ISOMURA, M. TAKADA, AND T. FUKAI

J Neurophysiol • VOL 97 • JUNE 2007 • www.jn.org

on

Au

gu

st 6

, 20

10

jn

.ph

ysio

log

y.o

rgD

ow

nlo

ad

ed

from


Sequential decision aggregation: Intermediate events

!"#$

!"#$%&#$#

'(

')

!"#$

aggregate states and divide in groups characterized by count

calculate the probability of transition between the different groups

characterize two states for network decisions H0 and H1


Sequential decision aggregation: Intermediate events

!"#$

!"#$%&#$#

'(

')

!"#$

aggregate states and divide in groups characterized by count

calculate the probability of transition between the different groups

characterize two states for network decisions H0 and H1


Sequential decision aggregation: Computational approach

Goal: as function of SDM decision probabilities {pi |j(t)}t∈N,compute SDA decision probabilities {pi |j(t;N, q)}t∈N

General result: q out of N decision probabilities

pi |j(t;N, q) =

q−1∑s0=0

q−1∑s1=0

(N

s1 + s0

)α(t − 1, s0, s1)βi |j(t, s0, s1)

+

bN/2c∑s=q

(N

2s

)α(t − 1, s)βi |j(t, s)

As function of t and sizes, formulas for α, β, α, and βcomputational complexity linear in N





pi |j(t;N, q) =

q−1∑s0=0

q−1∑s1=0

(N

s1 + s0

)α(t − 1, s0, s1)βi |j(t, s0, s1)

+

bN/2c∑s=q

(N

2s

)α(t − 1, s)βi |j(t, s)






pi |j(t;N, q) =

q−1∑s0=0

q−1∑s1=0

(N

s1 + s0

)α(t − 1, s0, s1)βi |j(t, s0, s1)

+

bN/2c∑s=q

(N

2s

)α(t − 1, s)βi |j(t, s)



Illustration of results

5 10 15 20 25 30 35

510

155

10

15

20

Nq

E[T]

Expected Decision Time

1020

305

1015

0.9

0.95

1

Nq

P[sa

y H 1|H

1]

Probability of correct decision

(H0 : µ = 0) and (H1 : µ = 1)

SPRT with pf = pm = 0.1

Gaussian noise N (µ, σ), σ = 1 and µ ∈ {0, 1}


Today’s Outline






Asymptotic results for the Fastest rule

Expected Decision Time:

limN→∞

E [T |H1,N, fastest] = earliest possible decision time

=: tmin = min{t ∈ N | either p1|1(t) 6= 0 or p0|1(t) 6= 0}

Accuracy:

limN→∞

p0|1(N, fastest) =

{0, if p1|1(tmin) > p0|1(tmin)

1, if p1|1(tmin) < p0|1(tmin)

1 SDA accuracy is function of (SDM probability at tmin),not of (SDA cumulative probability)!

2 hence, SDA accuracy is not monotonic with N

3 hence, SDA accuracy is unrelated to SDM accuracy for large N


Asymptotic results for the Fastest rule

Expected Decision Time:

limN→∞

E [T |H1,N, fastest] = earliest possible decision time

=: tmin = min{t ∈ N | either p1|1(t) 6= 0 or p0|1(t) 6= 0}

Accuracy:

limN→∞

p0|1(N, fastest) =

{0, if p1|1(tmin) > p0|1(tmin)

1, if p1|1(tmin) < p0|1(tmin)

1 SDA accuracy is function of (SDM probability at tmin),not of (SDA cumulative probability)!

2 hence, SDA accuracy is not monotonic with N

3 hence, SDA accuracy is unrelated to SDM accuracy for large N


Asymptotic results for the Majority rule

Expected Decision Time: Assume p1|1 > p0|1 and define

t< 12

:= max{t ∈ N | p1|1(0) + · · ·+ p1|1(t) < 1/2},

t> 12

:= min{t ∈ N | p1|1(0) + · · ·+ p1|1(t) > 1/2}

Then

limN→∞

E[T |H1,N,majority

]=

1

2

(t< 1

2+ t> 1

2+ 1

)Accuracy: Monotonicity with group size and, as N →∞

p0|1(N,majority) →

0, if p0|1 < 1/2

1, if p0|1 > 1/2√N/(2π) (4p0|1)

dN2e, if p0|1 < 1/4


Asymptotic results for the Majority rule

Expected Decision Time: Assume p1|1 > p0|1 and define

t< 12

:= max{t ∈ N | p1|1(0) + · · ·+ p1|1(t) < 1/2},

t> 12

:= min{t ∈ N | p1|1(0) + · · ·+ p1|1(t) > 1/2}

Then

limN→∞

E[T |H1,N,majority

]=

1

2

(t< 1

2+ t> 1

2+ 1

)Accuracy: Monotonicity with group size and, as N →∞

p0|1(N,majority) →

0, if p0|1 < 1/2

1, if p0|1 > 1/2√N/(2π) (4p0|1)

dN2e, if p0|1 < 1/4


Lessons learned about SDA

Accuracy Expected decision time

Fastest SDM accuracy at tmin earliest possible decision time tmin

Majority exponentially better than SDM average of half-times t< 12, t> 1

2

10 20 30 40 50 600

0.05

0.1Comparison: Fastest vs. Majority

Number of decision makers

Prob

abilit

y

o

f wro

ng d

ecisi

on

10 20 30 40 50 600

5

10

15

20

Number of decision makers

Expe

cted

num

ber

of o

bser

vatio

ns

Fastest ruleMajority rule


A fair comparison

to compare different thresholds, re-scale local accuracy

the group accuracy is now same (eg, low or high)

compare the decision time

2 4 6 8 10 12 14 16 180

10

20

30

40

50

N

E[T

]

network accuracy= 0.995

Fastest rule

Majority rule

2 4 6 8 10 12 14 16 180

5

10

15

N

E[T

]

Network accuracy 0.9

Fastest rule

Majority rule

for most cases majority rule is bestfor some small inaccurate networks, fastest rule is best


Conclusions and future directions

Summary fundamental understanding of “sequential aggregation”

1 applicable to broad range of agent models, eg, mixed networks

2 applicable to family of threshold-based rules

3 tradeoffs in fastest vs majority

4 role of time in sequential aggregation

Future directions

1 models with heterogeneous agents

2 models with interactions between agents

3 models with correlated information

4 how to use this analysis for design


sequential decision aggregation: outlinemotion.me.ucsb.edu/talks/2010r-sda-13aug2010-2x2.pdf ·...

Documents