1

Quantifying and Transferring Contextual Information

in Object Detection

Professor: S. J. WangStudent : Y. S. Wang

2

OutlineBackgroundGoalDifficulties in Usage of Contextual

InformationProvided solutionsAnother method: TASExperimental Results and

DiscussionConclusion and Future Direction

3

Background (I)Only the properties of target

object used in the detection task in the past.◦Problem: Intolerable number of false

positive

4

Background (I)Only the properties of target

object used in the detection task in the past.◦Problem: Intolerable number of false

positive

5

Background (II)What else??? Contextual

information!

6

GoalEstablish a model to efficiently

utilize the contextual information to boost the performance of detection accuracy.

7

Difficulties (I)Diversity of Contextual Information

◦There are may different types of context often co-existing with different degrees of relevance to the detection for the target object(s) in different images.

◦Terminology: Things (e.g. cars and people) Stuffs (e.g. roads and sky) Scene (e.g. what happen in the image)

◦Thing-Thing, Thing-Stuff, Stuff-Stuffand Scene-Thing

8

Difficulties (II)Ambiguity of Contextual

Information◦Contextual information can be

ambiguous and unreliable, thus may not always have a positive effect on object detection.

◦Ex: Crowded Scene with constant movement and occlusion among multiple objects.

9

Difficulties (III)Lack of Data for Context Learning

◦Not enough training data : Over-fitting problem Wrong degree of relevance

◦Ex: The contextual information of people on top of sofa can be more useful than people on top of grass.

10

Training Data Preparation & Notation Representation

Base Detector(HOG)

Training Image Candidate windows

Positive sample: Red

Negative sample: Green

11

Provided SolutionsA polar geometric descriptor for

contextual representation.A maximum margin context model

(MMC) for quantifying context.A context transfer learning model for

context learning with limited data.

12

Polar Geometric Descriptor

Instead of traditional annotation based descriptor, here we use polar geometric descriptor to describe two kind of contextual information (Thing-Thing, Thing-Stuff).

r :orientationb+1 :radial binsr*b+1 :patches0.5σ, σ and 2σ :bin lengthFeature :HOGPatch representation:Bag of Words method using K-means with K = 100

13

Provided SolutionsA polar geometric descriptor for

contextual representation.A maximum margin context model

(MMC) for quantifying context.A context transfer learning model for

context learning with limited data.

14

Quantifying Context (I)Risk function:

15

Quantifying Context (II)Goal = Minimize the Risk function

Minimize L equal to fulfill the following constraint

Hard to be solved, could be replaced by

16

Quantifying Context (III)Maximum Margin Context Model

Add some extra variables and constraints

17

Provided SolutionsA polar geometric descriptor for

contextual representation.A maximum margin context model

(MMC) for quantifying context.A context transfer learning model

for context learning with limited data.

18

Context Transfer LearningTwo Cases:

◦Similar contextual information Ex: Cars and motorbikes

◦Little in common in both appearance and context, but similar level of assistance provided by contextual information. Ex: People and bikes

19

TMMC-I: Transferring Discriminant Contextual Information

Similar context provide the assistance on the learning of w.

20

TMMC-I: Transferring Discriminant Contextual Information

New Constraint:

Modified optimization function:

21

TMMC-II: Transferring the Weight of Prior Detection Score

Similar level of assistance, same weight

22

TMMC-II: Transferring the Weight of Prior Detection Score

New Constraint:

Modified optimization function:

23

Another Method: TAS

24

Another Method: TAS (I)Steps:1. Segmenting image

into regions.2. Use base-detector

to get the candidate patches.

3. Establish the relationships between candidate patches and regions.

4. Use the relationships to judge there is a target object in the patch or not.

25

Another Method: TAS (II)Region clusters:

26

Another Method: TAS (III)Examples of experiment:

27

Experimental Result and Discussion

Use four data sets for testing:◦VOC 2005◦VOC 2007◦I-LIDS◦FORECOURT

28

Experimental Result and Discussion

29

Experimental Result and Discussion

30

Experimental Result and DiscussionContext Transfer Learning

Models:

31

Experimental Result and DiscussionContext Transfer Learning

Models:

32

Conclusion and Future Direction

In this paper, the author proposes a contextual information model to quantify and select useful context information to boost the detection performance.

What can we do next?◦HOG feature not suits for stuff (e.g. sky, road)◦Automatic selection between TMMC-I, TMMC-

II◦Automatic selection between target object

category and source category

33

ReferenceWei-Shi Zheng, Member, IEEE,

Shaogang Gong, and Tao Xiang, ”Quantifying and Transferring Contextual Information in Object Detection ”, PAMI accepted.

Geremy Heitz, Daphne Koller, “ Learning Spatial Context: Using Stuff to Find Things”, ECCV 2008.

Youtube Search “Hard-Margin SVM”