computational social cognition: perception, modeling, and ...hspark/groupbehavior/...choi, eccv12,...
TRANSCRIPT
![Page 1: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/1.jpg)
Social Dynamics
![Page 2: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/2.jpg)
Mathematical Definition of Group Behaviors
Behaviors of X and Y are a group behavior if spatial/temporal predictions of Y based on X and Y jointly are more accurate than predictions based on Y alone.
Y X
ˆ ˆ∆ − ∆ ≥ ∆ − ∆ Y(t + t) Y(t + t | Y(0), ,Y(t)) Y(t + t) Y(t + t | Y(0), ,Y(t),X(0), ,X(t))
Inspired by Granger Causality
![Page 3: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/3.jpg)
Number of group members
Scen
e dy
nam
ism
Dyadic interaction Crowd interaction
Static
scen
e Dy
nami
c sce
ne
Rehg, CVPR13 Prabhaker, ECCV12 Prabhakar, CVPR12 Patron-Perez, BMVC10
Yang, CVPR12 Hoai, CVPR14
Fathi, CVPR12 Choi, ECCV14 Park, NIPS12, ICCV13
Rodriguez, ICCV11a, ICCVb Mehran, CVPR09 Alahi, CVPR14
Cristani, BMVC11 Park, CVPR15 Arev, SIGGRAPH14
Lan, CVPR12 Ramanathan, CVPR13 Antic, ECCV14
Wang, ECCV10 Gallagher, CVPR09
Ding, ECCV10 Choi, ECCV12, CVPR14 Direkoglu, ECCV12
![Page 4: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/4.jpg)
![Page 5: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/5.jpg)
Number of group members
Scen
e dy
nam
ism
Dyadic interaction Crowd interaction
Static
scen
e Dy
nami
c sce
ne
Rehg, CVPR13 Prabhaker, ECCV12 Prabhakar, CVPR12 Patron-Perez, BMVC10
Yang, CVPR12 Hoai, CVPR14
Fathi, CVPR12 Choi, ECCV14 Park, NIPS12, ICCV13
Rodriguez, ICCV11a, ICCVb Mehran, CVPR09 Alahi, CVPR14
Cristani, BMVC11 Park, CVPR15 Arev, SIGGRAPH14
Lan, CVPR12 Ramanathan, CVPR13 Antic, ECCV14
Wang, ECCV10 Gallagher, CVPR09
Ding, ECCV10 Choi, ECCV12, CVPR14 Direkoglu, ECCV12
![Page 6: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/6.jpg)
Measurements
Individual social signal
Group behavior
Social role
Social Statics
![Page 7: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/7.jpg)
Measurements
Individual social signal
Group behavior
Social role
Time
Social Dynamics
![Page 8: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/8.jpg)
Measurements
Individual social signal
Group behavior
Social role
Social Dynamics
![Page 9: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/9.jpg)
Model based Approach
Measurements
Individual social signal
Group behavior
Social role
![Page 10: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/10.jpg)
Causal Relationship
Input: a video of social interactions Output: to find a causal relationship
[Prabhakar, CVPR10]
Measurements
Individual social signal: motion
Group behavior: Causality
Social role
![Page 11: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/11.jpg)
Time Signal Representation Point Process
Point process A random process for a set of isolated points either in time or space.
events
1 1 2 2 2 3 3 4 4 5 6 7 7 7 7 7 8 8 9 N(t)
0 1 2 3 3 3 4 5 5 7 7 7 8 9 10 11 11 13 13 N(t)
Multivariate point process
[Prabhakar, CVPR10]
![Page 12: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/12.jpg)
Visual Signal Representation Point Process
[Prabhakar, CVPR10]
![Page 13: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/13.jpg)
Causal Representation Cross Correlation
Cross correlation between point processes:
Spectral matrix for the multivariate point process: auto-spectrum+cross-spectrum
where
[Prabhakar, CVPR10]
![Page 14: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/14.jpg)
Causal Relationship Detection
Green: causally related actions
[Prabhakar, CVPR10]
![Page 15: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/15.jpg)
Causal Relationship Detection
[Prabhakar, CVPR10]
![Page 16: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/16.jpg)
Causal Relationship Detection
[Prabhakar, CVPR10]
![Page 17: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/17.jpg)
Social Force Model
Input: pedestrian locations Model: Newtonian mechanics
[Helbing, Phys. Rev. 95, Johansson, Adv. Comp. Sys. 2008]
Measurements
Individual social signal: position
Group behavior: Newtonian mechanics
Social role
![Page 18: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/18.jpg)
Equation of Motion for Pedestrians
αrα
krα : destination
eα
vα : instantaneous velocity
Fα
1) He or she wants to reach a certain destination.
Acceleration
[Helbing, Phys. Rev. 95]
![Page 19: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/19.jpg)
Equation of Motion for Pedestrians
αrα
krα : destination
eα
vα
Fα β
: Potential by wall Vαβ : Potential field by βwallU
i
1) He or she wants to reach a certain destination.
Acceleration
2) Motion of a pedestrian is influenced by other pedestrian. Repulsive effect induced by β
Deceleration induced by wall
Attractive effect induced by i
[Helbing, Phys. Rev. 95]
![Page 20: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/20.jpg)
Equation of Motion for Pedestrians 1) He or she wants to reach a certain destination.
Acceleration
2) Motion of a pedestrian is influenced by other pedestrian.
αrα
krα : destination
eα
vα
Fα αβ
Repulsive effect induced by β
: Potential by wall Vαβ : Potential field by βwallUDeceleration induced by wall
i
Attractive effect induced by i
3) Line of sight effect
[Helbing, Phys. Rev. 95]
![Page 21: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/21.jpg)
Equation of Motion for Pedestrians
Equation of motion:
αrα
krα : destination
eα
vα
Fα αβ
: Potential by wall Vαβ : Potential field by βwallU
i
ddt
=(preferred velocity)
(Force / acceleration) + (Noise)
[Helbing, Phys. Rev. 95]
wα
wα : preferred velocity affected by social force model.
![Page 22: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/22.jpg)
Pedestrian Stress Analysis
Jamarat bridge
[Johansson, Adv. Comp. Sys. 2008]
![Page 23: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/23.jpg)
Pedestrian Stress Analysis
Jamarat bridge
[Johansson, Adv. Comp. Sys. 2008]
![Page 24: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/24.jpg)
Abnormal Crowd Behaviors
Input: pedestrian tracks Output: to detect abnormal behaviors
[Mehran, CVPR09]
Measurements
Individual social signal: crowd motion
Group behavior: Newtonian mechanics
Social role
![Page 25: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/25.jpg)
Generalized Social Force Model
Social force model
Adjustment for desired velocity
Interaction between pedestrians
Generalized social force model
[0,1]ip ∈where is a panic parameter.
piv
civ
civ : average velocity of the neighboring pedestrians.
[Mehran, CVPR09]
![Page 26: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/26.jpg)
Generalized Social Force Model
Interaction force Optical flow field
[Mehran, CVPR09]
![Page 27: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/27.jpg)
[Mehran, CVPR09]
![Page 28: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/28.jpg)
Crowd Activity Forecasting
Input: short term trajectories Output: long term trajectories
[Alahi, CVPR14]
Measurements
Individual social signal: motion
Group behavior: social affinity
Social role
![Page 29: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/29.jpg)
100 m / 330 ft. 10 m 33 ft.
A Corridor with 14 Origin/Destination (O/D) Top View
O/D O/D O/D O/D O/D
O/D
O/D
O/D O/D O/D O/D O/D
O/D
O/D
Origin 5
Destination 11
![Page 30: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/30.jpg)
100 m / 330 ft. 10 m 33 ft.
O/D O/D O/D O/D O/D
O/D
O/D
O/D O/D O/D O/D O/D
O/D
O/D
Origin 5
Destination 11
Study the performance of having few cameras to estimate the O/D matrices
A Corridor with 14 Origin/Destination (O/D) Top View
![Page 31: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/31.jpg)
Collect long-term trajectories
31
# People Av. duration Av distance Density (up to) # Paths (O/D)
42 million 1 minute 100m 1 pedestrian/m2 196
Density
![Page 32: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/32.jpg)
Collect long-term trajectories
32
# People Av. duration Av distance Density (up to) # Paths (O/D)
42 million 1 minute 100m 1 pedestrian/m2 196
![Page 33: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/33.jpg)
Social Affinity Map (SAM)
Encoding spatial positions of neighbors
![Page 34: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/34.jpg)
Learning Social Affinity Map
Heatmap of the most occupied regions surrounding an individual
Direction of motion
![Page 35: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/35.jpg)
Input: • X: tracklets
Time t0 Time t1 Time tn
Tracking in a Large-scale Network
![Page 36: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/36.jpg)
Tracking in a Large-scale Network
Input: • X: tracklets
Output: • T: all long term trajectories
![Page 37: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/37.jpg)
Linked Tracklets
No prior
SAM prior
![Page 38: Computational Social Cognition: Perception, Modeling, and ...hspark/GroupBehavior/...Choi, ECCV12, CVPR14 Direkoglu, ECCV12 Number of group members Scene dynamism Dyadic interaction](https://reader033.vdocuments.us/reader033/viewer/2022050605/5facf5613ab3e5474f414126/html5/thumbnails/38.jpg)
Model based Approach
Measurements
Individual social signal
Group behavior
Social role