Context-based vision system for place and object

recognition

Torralba, Murphy, Freeman & Rubin (2003)

Context is useful for object recognition

Local and global image representations

• Local representation (L)– Wavelet decomposition– – N = 24 (6 orientations, 4 scales)

• Global representation– Average across space and downsample to

M x M (M = 4)– PCA to reduce to 80 dimensions–

€

v tL (x) = v t,k (x){ }

k=1,N

€

v tG

Place recognition

• Transition matrix– Count transitions + Dirichlet smoothing

• Observation likelihood– Appearances: mixture of Gaussians– Uniform weights

Influence of HMM

Dealing with novel places

• Separately trained model for categories

Using context for object detection

• Estimate probability of object presence using global image features

(Objects independent given location and image features)

€

P(O |Qt ,v) = P(Oi |Qt ,v t )i

∏

Context for object localization

• Coarse “expectation mask”