Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al....
Transcript of Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al....
Hierarchical Matching with Side Information for Image Classification
Authors Qiang Chen et al.
Presenter Hyung-seok Lee (이형석 )
CVPR 2012
2
Motivation (1/2)• Bag-of-Words (BoW)• Frequency histogram of visual words• Not emphasize any elements with regard to image layout
3
Motivation (2/2)• Spatial Pyramid Matching (SPM)• Partitioning the image plane into fine sub-regions• Not optimum for object-centered recognition problem
Beyond Bags of Features : Spatial Pyramid Matching for Recognizing Natural Scene Categories, Svetlana Lazebnik et al.
4
Main Idea• Generalized Hierarchical Matching (GHM)• Extends the popular matching work• Integrate prior knowledge for enhancing feature matching
side information
5
Overview
Im-age
Hierarchi-cal cluster-
ing
Hierarchical structure
representation
Matching over each cluster
Local features &
side informa-tion
6
Side Information (1/4)• Object Confidence Map• Object position should be extremely beneficial• Shape model / Appearance model
7
Side Information (2/4)• Shape-based object detection : HOG features
• Appearance-based object detection : BoW features
Object Confidence Map
8
Side Information (3/4)• Visual Saliency Map• Aim to detect the important content in images• Explain human visual search strategies
SUN : A Bayesian Framework for Saliency Using Natural Statistics, Lingyun Zhang et el.
9
Side Information (4/4)• Use the saliency model SUN (Saliency Using Natural statistics)• Based on natural image statistics
10
Side Information Combination• Combine object confidence map & visual saliency map• Design a 3 level hierarchical structure
- Object confidence map is used in level 2
- Background area will be further utilized in level 3
11
Encoding & Matching (1/2)
12
Encoding & Matching (2/2)• Encoding is operated on each cluster• Matching over each corresponding cluster
13
Experiments (1/2)• Caltech-UCSD Birds 200 & Oxford Flowers 17 and 102
14
Experiments (2/2)• VOC 2007
• VOC 2010
15
Conclusion• Propose the Generalized Hierarchical Matching framework
- Extends the popular pyramid matching
• Two novel kinds of side information are introduced to enhance object-oriented image classification tasks
- Object confidence map- Visual saliency amp