Automatic Face Recognition under Component-Based Manifolds
CVGIP 2006
Wen-Sheng Chu (朱文生 ) and Jenn-Jier James Lien (連震杰 )
Robotics Lab. CSIE NCKU
Objective
• Recognize faces using multiple face patterns rather than a single one.
Person BPerson A Person BPerson A
Single input pattern Multiple input patterns
Automatic Acquisition of Facial Components
Detected Features
Normalized Illumination IB
Normalized Pose IR
Original Image
FaceDetection
Cropped Face I
Feature Point Detection
Face +veRemoval
Facial Components
Extraction
Band-passFiltering
Registration byAffine Warping
Rejected Non-face
Extracted Facial Components
2-Class SVM
Classifiers
Training Data of Feat
ures
P. Viola and M. Jones, “Robust Real-Time Face Detection”, IJCV 2004.
Automatic Acquisition of Facial Components
Detected Features
Normalized Illumination IB
Normalized Pose IR
Original Image
FaceDetection
Cropped Face I
Feature Point Detection
Face +veRemoval
Facial Components
Extraction
Band-passFiltering
Registration byAffine Warping
Rejected Non-face
Extracted Facial Components
2-Class SVM
Classifiers
Training Data of Feat
ures
Facial Feature Detector
• 2-class SVM with feature vector v:
• Reject false positives
),()( yxIxNyvA
),()( yxIxNyvG
G
A
v
vv
x x x
o o o o
Automatic Acquisition of Facial Components
Detected Features
Normalized Illumination IB
Normalized Pose IR
Original Image
FaceDetection
Cropped Face I
Feature Point Detection
Face +veRemoval
Facial Components
Extraction
Band-passFiltering
Registration byAffine Warping
Rejected Non-face
Extracted Facial Components
2-Class SVM
Classifiers
Training Data of Feat
ures
Registration & Illumination Normalization
Affine warping
Band-pass filtering85.0 GIGII RRB
RI
I
BI
Registration
IlluminationNormalization
Automatic Acquisition of Facial Components
Detected Features
Normalized Illumination IB
Normalized Pose IR
Original Image
FaceDetection
Cropped Face I
Feature Point Detection
Face +veRemoval
Facial Components
Extraction
Band-passFiltering
Registration byAffine Warping
Rejected Non-face
Extracted Facial Components
2-Class SVM
Classifiers
Training Data of Feat
ures
Facial Components Extraction
• Effects of pose and illumination are smaller in each local region compared with those in the holistic face image.
T. K. Kim, H. Kim, W. Hwang and J. Kittler, “Independent Component Analysis in A Local Facial Residue Space for Face Recognition”, PR, 2004.
Constrained Mutual Subspace Method (CMSM)
• Similarity between i and j == θc
• Use the variation of dissimilarity between subjects
θ
θc
K. Fukui and O. Yamaguchi, “Face Recognition Using Multi-viewpoint Patterns for Robot Vision”, ISRR 2003.
project project
subspace i subspace j
ic jc
constrainedsubspace
Constrained Subspace Generation
• Take nose for explanation:
The eigenvectors, w, selected in ascending order, are the basis of the constrained subspace, Snose.
Tnosei
nose
i
nose
i
T
kkNk
nosei NC BBxx/1 1
T
iii
nosenosenosenoseL
nose2
nose1 BBP w,)wP...PP(
Constraint subspace basis PCA basis
Projection onto Constrained Subspace
1. Projection basis vectors constrained subspace Snose
2. Normalizationlength(projected vector) 1
3. Orthogonalizationapplying Gram-Schmidt process to orthogonalize the normalized vectors
Snose
nose
iB nose
jB
Comparison between Normalized Manifolds
• The similarity of nose between subject i and subject j:
where are defined as the eigenvalues of ma
trix .• Similarity(i, j) == summing up the five canonical
correlations
nose
i
ti
nose
Sj
nose
Sit 2
1 cos/1)B,B(similarity
nose
i
NNTnose
Si
nose
Si
Tnose
Sj
nose
Sj
Tnose
Si
nose
Si
BBBBBB
Experiment Setup
#individuals 16
#sequence/individual 5
#second/sequence 10
#frame/second 10
Resolution 320x240
Size of Registration Template 100x125
Size of Facial Components
Eye-braw 40x15
Eyes 28x18
Nose 44x21
Mouth 28x40
Typical Samples in 3D Principal Component Space – Holistic Image
subject 1 (․)
subject 2 (․)
subject 3 (․)
subject 4 (․)
Comparison
MethodsNN-facial
component
MSM-facial
component
CMSM-holistic
face
CMSM-facial component
Recognition Rate (%)
82.3 90.8 95.2 98.6
Top Related