learning collections of parts for object recognition and transfer learning university of illinois at...

Learning Collections of Parts for Object Recognition and Transfer

Learning

University of Illinois at Urbana-Champaign

Flexible Part Based Model

• Why?– Interested in learning a large number of object

categories– Avoid learning new category from scratch when

useful information can be borrowed from other categories

Flexible Part Based Model: Objectives

• Simple to train– Minimal manual initialization effort– Train each part independently– Simple spatial model

3How Can We Adapt Existing Part Models to New Categories?

Boosted Collections of Parts

4

• Simple to train– Minimal manual initialization effort– Train each part independently– Simple spatial model

ECCV 2010

How Can We Adapt Existing Part Models to New Categories?

Part Refinement

Retrain with new examples

Train Part Detector

Collect Consistent PositivesInitialize with

Single Exemplar

5

• Compute expected part position from exemplar:

• Transfer to other examples:

Encouraging Spatial Consistency

6How Can We Adapt Existing Part Models to New Categories?

• Only allow candidates with sufficient overlap with expected position

Encouraging Consistency

7

GOODBAD

How Can We Adapt Existing Part Models to New Categories?

Learned Part Models

8How Can We Adapt Existing Part Models to New Categories?

Learned Part Models

9How Can We Adapt Existing Part Models to New Categories?

Part Evaluation: Discrimination

1. How discriminative are our parts? (mean AP)Plane Bike Boat Cat Dog Sofa

Exemplar 15.2 17.4 3.5 23.6 18.1 6.6

Refined: All-in 36.5 39.7 4.0 42.3 25.8 8.0

Selective: Appearance 38.1 39.9 5.7 46.5 29.5 8.3

Selective: App.+Spatial 37.3 37.2 4.6 39.5 24.4 8.7

Part Evaluation: Spatial Consistency

1. How discriminative are our parts? (mean AP)2. How well can we localize Poselet Keypoint

annotations? (mean best AP per keypoint type)Plane Bike Boat Cat Dog Sofa

Exemplar 14.1 34.6 12.4 12.8 8.9 9.1

Refined: All-in 21.3 41.3 9.6 22.0 12.9 7.2

Selective: Appearance 23.9 41.6 13.9 22.5 14.7 11.1

Selective: App.+Spatial 27.3 42.4 14.8 22.2 13.3 10.8

Pooling Part Detections

Propose 500 candidate object regions per image(Endres and Hoiem 2010)

Pooling Part Detections

Collect highest scoring response for each part:

Pooling Part Detections

Collect highest scoring response for each part:

Scoring Object Candidates

Classify vector of scores using boosted classifier

Relocalization

Loose spatial model: Good parts can be assigned to bad regions

Relocalization

Loose spatial model: Good parts can be assigned to bad regions

Solution 1: •Region shape features to down-weight bad regions•HOG of segmentation mask

Relocalization

Loose spatial model: Good parts can be assigned to bad regions

Solution 2: •Use parts to repredict bounding box•Each part votes for box•Weighted average based on appearance score and learned reliability

Results:Beating state of the art

Our ModelFelzenszwalb et al.

Aeroplane44.3 -> 48.4 AP

Cat24.1 -> 36.9 AP

Dog8.5 -> 20.9 AP

Results:Competitive with state of the art

Bicycle49.6 -> 43.0 AP

Boat6.7 -> 5.0 AP

Sofa17.2 -> 14.1 AP

Our ModelFelzenszwalb et al.

Conclusion

• Goal: Recognition systems that can give as much detail about any object they encounter

• Consider supervised tasks that generalize across categories

• Capture shared similarities across categories and differences within categories

25