以區域二元圖樣與部分比對為基礎之 人臉辨識 Face Recognition with Local Binary...
-
Upload
branden-maxwell -
Category
Documents
-
view
227 -
download
1
Transcript of 以區域二元圖樣與部分比對為基礎之 人臉辨識 Face Recognition with Local Binary...
以區域二元圖樣與部分比對為基礎之人臉辨識Face Recognition with Local Binary
Patterns and Partial Matching
Presenter: 施佩汝Advisor: 歐陽明 教授
1
Outlines
• Motivation• Implementation• Result• Conclusion
2
MOTIVATION
3
Motivation
4
Publication
• Che-Hua Yeh, Pei-Ruu Shih, Kuan-Ting Liu, Yin-Tzu Lin, Huang-Ming Chang, Ming Ouhyoung. A Comparison of Three Methods of Face Recognition for Home Photos. ACM Siggraph, poster, 2009.
5
Problem Statement
6
Main Contribution
• Improve Local Binary Patterns by using Partial Matching Metric
• Better Performance in Home Photos
7
IMPLEMENTATION
8
System Overview
Pre-Processing
Build Descriptor
Images Descriptors
ClusteringCalculate
LBPBuild
Descriptor
9
System Overview
Pre-Processing
Build Descriptor
Images Descriptors
ClusteringCalculate
LBPBuild
Descriptor
10
Pre-Processing
11
System Overview
Pre-Processing
Build Descriptor
Images Descriptors
ClusteringCalculate
LBPBuild
Descriptor
12
System Overview
Pre-Processing
Build Descriptor
Images Descriptors
ClusteringCalculate
LBPBuild
Descriptor
13
Local Binary Patterns [PAMI2006]
• An operator to encode the relationship of a pixel and its neighbors. Z1 Z2 Z3
Z8 Z0 Z4
Z7 Z6 Z5
58 99 21
54 54 86
67 12 13
1
otherwise ,0
0 if ,1
28
10
xxs
ZZsLBP p
pp
14
Local Binary Patterns [PAMI2006]
• An operator to encode the relationship of a pixel and its neighbors.
58 99 21
54 54 86
67 12 13
1 1
otherwise ,0
0 if ,1
28
10
xxs
ZZsLBP p
pp
Z1 Z2 Z3
Z8 Z0 Z4
Z7 Z6 Z5
15
Local Binary Patterns [PAMI2006]
• An operator to encode the relationship of a pixel and its neighbors.
58 99 21
54 54 86
67 12 13
1 1 0
otherwise ,0
0 if ,1
28
10
xxs
ZZsLBP p
pp
Z1 Z2 Z3
Z8 Z0 Z4
Z7 Z6 Z5
16
Local Binary Patterns [PAMI2006]
• An operator to encode the relationship of a pixel and its neighbors.
58 99 21
54 54 86
67 12 13
1 1 0
1
otherwise ,0
0 if ,1
28
10
xxs
ZZsLBP p
pp
Z1 Z2 Z3
Z8 Z0 Z4
Z7 Z6 Z5
17
Local Binary Patterns [PAMI2006]
• An operator to encode the relationship of a pixel and its neighbors.
58 99 21
54 54 86
67 12 13
1 1 0
1
0
otherwise ,0
0 if ,1
28
10
xxs
ZZsLBP p
pp
Z1 Z2 Z3
Z8 Z0 Z4
Z7 Z6 Z5
18
Local Binary Patterns [PAMI2006]
• An operator to encode the relationship of a pixel and its neighbors.
58 99 21
54 54 86
67 12 13
1 1 0
1
0 0
otherwise ,0
0 if ,1
28
10
xxs
ZZsLBP p
pp
Z1 Z2 Z3
Z8 Z0 Z4
Z7 Z6 Z5
19
Local Binary Patterns [PAMI2006]
• An operator to encode the relationship of a pixel and its neighbors.
58 99 21
54 54 86
67 12 13
1 1 0
1
1 0 0
otherwise ,0
0 if ,1
28
10
xxs
ZZsLBP p
pp
Z1 Z2 Z3
Z8 Z0 Z4
Z7 Z6 Z5
20
Local Binary Patterns [PAMI2006]
• An operator to encode the relationship of a pixel and its neighbors.
58 99 21
54 54 86
67 12 13
1 1 0
1 1
1 0 0
otherwise ,0
0 if ,1
28
10
xxs
ZZsLBP p
pp
Z1 Z2 Z3
Z8 Z0 Z4
Z7 Z6 Z5
21
Local Binary Patterns [PAMI2006]
• An operator to encode the relationship of a pixel and its neighbors.
58 99 21
54 54 86
67 12 13
1 1 0
1 1
1 0 0
otherwise ,0
0 if ,1
28
10
xxs
ZZsLBP p
pp
Z1 Z2 Z3
Z8 Z0 Z4
Z7 Z6 Z5
LBP = 1101001122
System Overview
Prepared-Works
Build Descriptor
Images Descriptors
ClusteringCalculate
LBPBuild
Descriptor
23
Facial Image Descriptor
• They use Spatially Enhanced Histogram in original Local Binary Pattern. [PAMI2006]
24
Local Patches
• We sample a patch for every s pixels.• There are S patches for one image.
mm
s s
25
Spatial Block [CVPR2007]
• We use three concentric circles to describe a patch.
26
Descriptor
• Build a descriptor for one face.
27
KKKK
K
K
fff
fff
fff
21
22221
11211
System Overview
Pre-Processing
Build Descriptor
Images Descriptors
ClusteringCalculate
LBPBuild
Descriptor
28
System Overview
Pre-Processing
Build Descriptor
Images Descriptors
Complete-Linkage Clustering
Calculate LBP
Build Descriptor
Compute all the similarities
Clustering
29
System Overview
Pre-Processing
Build Descriptor
Images Descriptors
Complete-Linkage Clustering
Calculate LBP
Build Descriptor
Compute all the similarities
Clustering
30
Similarity
• They use the weighted Chi-Square distance in original Local Binary Pattern. [PAMI2006]
•
j ijij
ijij
iiw MS
MSwMS
2
2 ,
31
Partial Matching [ICCV2009]
• Step1:– Compute the similarity of each patch from one
image with the nearby patches in another image.
,
Image 1: I(1) Image 2: I(2)
)2()1(||;|:|,
)1( ,min klijrslsjrsksilkij ffsimfd
32
Partial Matching [ICCV2009]
• Step2:– Sort the similarities of all patches.
– dαS is the similarity of I(1) to I(2). K
jiijsS fdSortdddd1,
)1(21 ,...,,...,,
SdIId )( )2()1(
33
Partial Matching [ICCV2009]
• Step3:– Calculate the similarity of I(2) to I(1)
)( )1()2( IId
34
Partial Matching [ICCV2009]
• Step4:– Use the maximum of two similarity
)(),(max),( )1()2()2()1()2()1( IIdIIdIID
35
System Overview
Pre-Processing
Build Descriptor
Images Descriptors
Complete-Linkage Clustering
Calculate LBP
Build Descriptor
Compute all the similarities
Clustering
36
Hierarchical Clustering
• Build a tree-based hierarchical taxonomy (dendrogram) from a set of documents.
Material Selected from Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, 2008.
37
Hierarchical Clustering
• Clustering obtained by cutting the dendrogram at a desired level: each connected connected component forms a cluster.
Material Selected from Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, 2008.
38
Hierarchical Complete-Linkage Clustering
• Similarity of the “furthest” points.• Makes “tighter,” spherical clusters that are
typically preferable.
Material Selected from Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, 2008.
39
Performance Optimization
• 4 threads in Quad-Core system
Pre-Processing
Build Descriptor
Images Descriptors
Complete-Link Clustering
Calculate LBP
Build Descriptor
Compute all the similarities
Clustering
40
Performance Optimization
• 4 threads in Quad-Core system– 3 times faster than single thread.
• 73 minutes to 24 minute for 309 images.
Pre-Processing
Build Descriptor
Images Descriptors
Complete-Link Clustering
Calculate LBP
Build Descriptor
Compute all the similarities
Clustering
41
RESULT
42
FERET Result
• fa: gallery, 994 images• fb: alternative facial expression, 992 images• dup1: the photos taken after later, 736 images• dup2: the photos taken at least one year after
the gallery, 228 images
43
FERET Result
44
Accuracy fb dup1 dup2
LBP [PAMI2006] 95.67% 59.92% 45.61%
Our Result 98.89% 71.33% 68.42%
Time※ Register fb dup1 dup2
LBP [PAMI2006] 46 seconds 102 seconds 67 seconds 21 seconds
Our Result 4 minutes 224 minutes 168 minutes 5 minutes
※ The time results are computed in multithreads version.
Experiments
• Home Photo Dataset I– 309 images, 5 subjects
• Home Photo Dataset II– 838 images, 8 subjects
45
Evaluation
• Cluster Number• Unknown Number• Pair-wise Precision• Pair-wise Rand Index• Executing time
46
Unknown Number
• The number of clusters which contain only one component.
47
Precision/Rand Index
fptp
tpprecision
fntp
tprecall
tnfpfntp
tntpindexrand
48
Assigned
Same Cluster
Different Clusters
Ground Truth
Same Cluster tp fn
Different Clusters fp tn
Dataset-I Result
#Clusters #Unknown Precision Rand Index
Time ※
Picasa Web 94 73 99.92% 0.829816 10 seconds
Picasa PC 99 75 100% 0.865002 3 minutes
LBP [PAMI2006] 100 31 90.39% 0.811856 4 seconds
LID_PM
[ICCV2009]
99 37 98.78% - 11 minutes
Our Result 100 39 99.46% 0.816290 24 minutes
LID+PM
[Chang2010]
100 43 99.24% - 25 minutes
LBP: Local Binary Pattern, PM: Partial Matching, SB: Spatial Block
※ The time results are computed in multithreads version.49
Dataset-I Result
50
LBP Our Result
Dataset-I Result
• Wrong clustering result in LBP
• The clusters in our result
51
Dataset-II Result
#Clusters #Unknown Precision Rand Index
Time ※
Picasa Web 195 47 99.49% 0.876229 10 seconds
Picasa PC 253 150 100% 0.888492 10 minutes
LBP [PAMI2006] 253 45 91.70% 0.70261 17 seconds
LID_PM [ICCV2009] 253 79 97.88% - 51 minutes
Our Result 253 64 99.59% 0.871187 163 minutes
LID+PM
[Chang2010]
253 89 99.82% - 136 minutes
LBP: Local Binary Pattern, PM: Partial Matching, SB: Spatial Block
※ The time results are computed in multithreads version.52
Dataset-II Result
53
LBP Our Result
Dataset-II Result
• Wrong clustering result in LBP
• Clusters in our result
54
Demo
• Face Recognition with web camera
55
CONCLUSION
56
Conclusion
• LBP is an efficient algorithm for face recognition.
• Partial Matching is good for the different facial expression or different illumination in facial images.
• Our system has better performance than LBP.
57
Future Works
• Improve performance by GPU.
• Use other extension of LBP combined with Partial Matching.
58
Thank You for Your Attention!
59
Result
LBP: Local Binary Pattern, PM: Partial Matching, SB: Spatial Block
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.20.700000000000001
0.750000000000001
0.800000000000001
0.850000000000001
0.900000000000001
0.950000000000001
1
Home Photo Dataset I
LBP LBP+SB LBP+PM LBP+LID+SB
Recall
Pre
cisi
on
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.20.700000000000001
0.750000000000001
0.800000000000001
0.850000000000001
0.900000000000001
0.950000000000001
1
Home Photo Dataset II
LBP LBP+SB LBP+PM LBP+SB+PM
Recall
Pre
cisi
on
60