Information Extraction, Conditional Random Fields, and Social Network Analysis
Object Segmentation Based on Multiple Features Fusion and Conditional Random Field
description
Transcript of Object Segmentation Based on Multiple Features Fusion and Conditional Random Field
Object Segmentation Based on Multiple Features Fusion and Conditional Random Field
CASIA_IGIT
National Laboratory of Pattern Recognition(NLPR)Institute of Automation, Chinese Academy of Sciences(CASIA)
Reporter: Kun Ding(丁昆)2013.10.17
Outline• System Overview
• System Characteristics
•Results and Conclusions
Outline• System Overview
• System Characteristics
•Results and Conclusions
System Overview
•Object Segmentation Pipeline
FeatureEngineering
Superpixel Segmentation
Superpixels Features Probabilistic Output Final ResultsInput Image
Feature Extraction SVM Classification GrabCut
Stage 1 : Superpixel Classification
Stage 2 : Pixel-based CRF Smoothing
System Overview
• Superpixel Classification• Superpixel Segmentation • Graph-based image segmentation
• Feature Extraction: • To be detailed in next section
• SVM Classification[1]• RBF kernel with Probabilistic Output
System Overview
•Pixel-Based CRF Smoothing• Fusing several kinds of information as data term• Solving with GrabCut with only a few iterations
SVM Probabilistic Output CRF Smoothing Output
BinarizeFirst Iteration
SecondIteration
Outline
• System Overview
• System Characteristics
•Results and Conclusions
System Characteristics • Superpixel Segmentation -- Efficient Graph-Based Image Segmentation[2]• Fast, property of edge-preserving• Speeding up the whole procedure• Improving the separability between foreground and
background
Superpixels and their edge-preserving property
System Characteristics • Feature Engineering – Superpixel-Based Multiple Features Fusion
Gradient
Texture
Color and skin
Geometrical
Saliency
Results of Object Detection
PCA
Dense SIFT[3][4] dictionary with Bag-of-Words description
Multi-scale LBP histogram
RGB histogram and HS histogram with skin detection
Position, direction and roundness
Color spatial distribution, multi-scale local and global contrast
Probability derived from AdaBoost, with manifold ranking[6] refinement
System Characteristics • Feature Engineering – Superpixel-Based Multiple Features Fusion• Illustration of object detection
Object Detection result Rectangle Density as Probability Refined with Manifold Ranking
System Characteristics •Pixel-Based CRF Smoothing – GrabCut[7]•Modified data term • Solving by maxflow iteratively
SVM Result Object Detection Result
GMM Result for Foreground and Background
CRF Smoothing Output
Outline• System Overview
• System Characteristics
•Results and Conclusions
Conclusion and Results Exhibition•Results Exhibition
Conclusion and Results Exhibition•Conclusion• Superpixel classification• Feature fusion works• CRF smoothing improves the results of SVM
• Object parts sometimes lost• Context information is inadequate
[1] C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http: //www.csie.ntu.edu.tw/˜cjlin/libsvm.[2] Felzenszwalb P F, Huttenlocher D P. Efficient graph-based image segmentation[J]. International Journal of Computer Vision, 2004, 59(2): 167-181.[3] Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International journal of computer vision, 2004, 60(2): 91-110.[4] Vedaldi A, Fulkerson B. VLFeat: An open and portable library of computer vision algorithms[C]//Proceedings of the international conference on Multimedia. ACM, 2010: 1469-1472.
Selected References
Selected References[5] Liu T, Yuan Z, Sun J, et al. Learning to detect a salient object[J]. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2011, 33(2): 353-367.[6] Chuan Yang, Lihe Zhang, Huchuan Lu, Minghsuan Yang, Saliency Detection via Graph-Based Manifold Ranking, CVPR2013, P3166-3173[7] Rother C, Kolmogorov V, Blake A. Grabcut: Interactive foreground extraction using iterated graph cuts[C]//ACM Transactions on Graphics (TOG). ACM, 2004, 23(3): 309-314.
Thank you very much!Any questions?
CASIA_IGIT
Leader: Ying Wang (王颖)Members: Kun Ding (丁昆)
Huxiang Gu (谷鹄翔)Yongchao Gong (宫永超)
E-mails: {ywang, kding, hxgu, yongchao.gong}@nlpr.ia.ac.cn