以區域二元圖樣與部分比對為基礎之人臉辨識 Face Recognition with Local Binary...

以區域二元圖樣與部分比對為基礎之人臉辨識Face Recognition with Local Binary

Patterns and Partial Matching

Presenter: 施佩汝Advisor: 歐陽明教授

1

Outlines• Motivation• Implementation• Result• Conclusion

2

MOTIVATION

3

Motivation

4

Publication• Che-Hua Yeh, Pei-Ruu Shih, Kuan-Ting Liu, Yin-Tzu

Lin, Huang-Ming Chang, Ming Ouhyoung. A Comparison of Three Methods of Face Recognition for Home Photos. ACM Siggraph, poster, 2009.

5

Problem Statement

6

Main Contribution• Improve Local Binary Patterns by using Partial

Matching Metric

• Better Performance in Home Photos

7

IMPLEMENTATION

8

System Overview

Pre-Processing

Build Descriptor

Images Descriptors

ClusteringCalculate

LBPBuild

Descriptor

9

System Overview

Pre-Processing

Build Descriptor

Images Descriptors

ClusteringCalculate

LBPBuild

Descriptor

10

Pre-Processing

11

System Overview

Pre-Processing

Build Descriptor

Images Descriptors

ClusteringCalculate

LBPBuild

Descriptor

12

System Overview

Pre-Processing

Build Descriptor

Images Descriptors

ClusteringCalculate

LBPBuild

Descriptor

13

Local Binary Patterns [PAMI2006]

• An operator to encode the relationship of a pixel and its neighbors. Z1 Z2 Z3

Z8 Z0 Z4

Z7 Z6 Z5

58 99 21

54 54 86

67 12 13

1

otherwise ,00 if ,1

28

10

xxs

ZZsLBP p

pp

14


• An operator to encode the relationship of a pixel and its neighbors.

58 99 21

54 54 86

67 12 13

1 1

otherwise ,00 if ,1

28

10

xxs

ZZsLBP p

pp

Z1 Z2 Z3

Z8 Z0 Z4

Z7 Z6 Z5

15



58 99 21

54 54 86

67 12 13

1 1 0

otherwise ,00 if ,1

28

10

xxs

ZZsLBP p

pp

Z1 Z2 Z3

Z8 Z0 Z4

Z7 Z6 Z5

16



58 99 21

54 54 86

67 12 13

1 1 0

1

otherwise ,00 if ,1

28

10

xxs

ZZsLBP p

pp

Z1 Z2 Z3

Z8 Z0 Z4

Z7 Z6 Z5

17



58 99 21

54 54 86

67 12 13

1 1 0

1

0

otherwise ,00 if ,1

28

10

xxs

ZZsLBP p

pp

Z1 Z2 Z3

Z8 Z0 Z4

Z7 Z6 Z5

18



58 99 21

54 54 86

67 12 13

1 1 0

1

0 0

otherwise ,00 if ,1

28

10

xxs

ZZsLBP p

pp

Z1 Z2 Z3

Z8 Z0 Z4

Z7 Z6 Z5

19



58 99 21

54 54 86

67 12 13

1 1 0

1

1 0 0

otherwise ,00 if ,1

28

10

xxs

ZZsLBP p

pp

Z1 Z2 Z3

Z8 Z0 Z4

Z7 Z6 Z5

20



58 99 21

54 54 86

67 12 13

1 1 0

1 1

1 0 0

otherwise ,00 if ,1

28

10

xxs

ZZsLBP p

pp

Z1 Z2 Z3

Z8 Z0 Z4

Z7 Z6 Z5

21



58 99 21

54 54 86

67 12 13

1 1 0

1 1

1 0 0

otherwise ,00 if ,1

28

10

xxs

ZZsLBP p

pp

Z1 Z2 Z3

Z8 Z0 Z4

Z7 Z6 Z5

LBP = 1101001122

System Overview

Prepared-Works

Build Descriptor

Images Descriptors

ClusteringCalculate

LBPBuild

Descriptor

23

Facial Image Descriptor• They use Spatially Enhanced Histogram in

original Local Binary Pattern. [PAMI2006]

24

Local Patches• We sample a patch for every s pixels.• There are S patches for one image.

mm

s s

25

Spatial Block [CVPR2007]

• We use three concentric circles to describe a patch.

26

Descriptor• Build a descriptor for one face.

27

KKKK

K

K

fff

ffffff

21

22221

11211

System Overview

Pre-Processing

Build Descriptor

Images Descriptors

ClusteringCalculate

LBPBuild

Descriptor

28

System Overview

Pre-Processing

Build Descriptor

Images Descriptors

Complete-Linkage Clustering

Calculate LBP

Build Descriptor

Compute all the similarities Clustering

29

System Overview

Pre-Processing

Build Descriptor

Images Descriptors


Calculate LBP

Build Descriptor


30

Similarity• They use the weighted Chi-Square distance in

original Local Binary Pattern. [PAMI2006]

•

j ijij

ijij

iiw MS

MSwMS

22 ,

31

Partial Matching [ICCV2009]

• Step1:– Compute the similarity of each patch from one

image with the nearby patches in another image.

,

Image 1: I(1) Image 2: I(2)

)2()1(||;|:|,

)1( ,min klijrslsjrsksilkij ffsimfd

32


• Step2:– Sort the similarities of all patches.

– dαS is the similarity of I(1) to I(2). K

jiijsS fdSortdddd1,

)1(21 ,...,,...,,

SdIId )( )2()1(

33


• Step3:– Calculate the similarity of I(2) to I(1)

)( )1()2( IId

34


• Step4:– Use the maximum of two similarity

)(),(max),( )1()2()2()1()2()1( IIdIIdIID

35

System Overview

Pre-Processing

Build Descriptor

Images Descriptors


Calculate LBP

Build Descriptor


36

Hierarchical Clustering• Build a tree-based hierarchical taxonomy

(dendrogram) from a set of documents.

Material Selected from Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, 2008.

37

Hierarchical Clustering• Clustering obtained by cutting the dendrogram

at a desired level: each connected connected component forms a cluster.


38

Hierarchical Complete-Linkage Clustering

• Similarity of the “furthest” points.• Makes “tighter,” spherical clusters that are

typically preferable.


39

Performance Optimization• 4 threads in Quad-Core system

Pre-Processing

Build Descriptor

Images Descriptors

Complete-Link Clustering

Calculate LBP

Build Descriptor


40

Performance Optimization• 4 threads in Quad-Core system

– 3 times faster than single thread.• 73 minutes to 24 minute for 309 images.

Pre-Processing

Build Descriptor

Images Descriptors

Complete-Link Clustering

Calculate LBP

Build Descriptor


41

RESULT

42

FERET Result• fa: gallery, 994 images• fb: alternative facial expression, 992 images• dup1: the photos taken after later, 736 images• dup2: the photos taken at least one year after

the gallery, 228 images

43

FERET Result

44

Accuracy fb dup1 dup2LBP [PAMI2006] 95.67% 59.92% 45.61%

Our Result 98.89% 71.33% 68.42%

Time※ Register fb dup1 dup2LBP [PAMI2006] 46 seconds 102 seconds 67 seconds 21 seconds

Our Result 4 minutes 224 minutes 168 minutes 5 minutes

※ The time results are computed in multithreads version.

Experiments• Home Photo Dataset I

– 309 images, 5 subjects

• Home Photo Dataset II– 838 images, 8 subjects

45

Evaluation• Cluster Number• Unknown Number• Pair-wise Precision• Pair-wise Rand Index• Executing time

46

Unknown Number• The number of clusters which contain only one

component.

47

Precision/Rand Index

fptptpprecision

fntptprecall

tnfpfntptntpindexrand

48

Assigned

Same Cluster

Different Clusters

Ground Truth

Same Cluster tp fn

Different Clusters fp tn

Dataset-I Result#Clusters #Unknown Precision Rand

IndexTime ※

Picasa Web 94 73 99.92% 0.829816 10 seconds

Picasa PC 99 75 100% 0.865002 3 minutes

LBP [PAMI2006] 100 31 90.39% 0.811856 4 seconds

LID_PM [ICCV2009]

99 37 98.78% - 11 minutes

Our Result 100 39 99.46% 0.816290 24 minutes

LID+PM [Chang2010]

100 43 99.24% - 25 minutes

LBP: Local Binary Pattern, PM: Partial Matching, SB: Spatial Block

※ The time results are computed in multithreads version.49

Dataset-I Result

50

LBP Our Result

http://c.csie.org/~kana/hp01/hp01_LBP_com_100_s8_r4_res88w8_a20_str1.html


Dataset-I Result• Wrong clustering result in LBP

• The clusters in our result

51

Dataset-II Result#Clusters #Unknown Precision Rand

IndexTime ※

Picasa Web 195 47 99.49% 0.876229 10 seconds

Picasa PC 253 150 100% 0.888492 10 minutes

LBP [PAMI2006] 253 45 91.70% 0.70261 17 seconds

LID_PM [ICCV2009] 253 79 97.88% - 51 minutes

Our Result 253 64 99.59% 0.871187 163 minutes

LID+PM [Chang2010]

253 89 99.82% - 136 minutes


※ The time results are computed in multithreads version.52

Dataset-II Result

53

LBP Our Result



Dataset-II Result• Wrong clustering result in LBP

• Clusters in our result

54

Demo• Face Recognition with web camera

55

CONCLUSION

56

Conclusion• LBP is an efficient algorithm for face

recognition.

• Partial Matching is good for the different facial expression or different illumination in facial images.

• Our system has better performance than LBP.

57

Future Works• Improve performance by GPU.

• Use other extension of LBP combined with Partial Matching.

58

Thank You for Your Attention!

59

Result


0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.20.700000000000001

0.750000000000001

0.800000000000001

0.850000000000001

0.900000000000001

0.950000000000001

1

Home Photo Dataset I

LBP LBP+SB LBP+PM LBP+LID+SB

Recall

Prec

ision

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.20.700000000000001

0.750000000000001

0.800000000000001

0.850000000000001

0.900000000000001

0.950000000000001

1

Home Photo Dataset II

LBP LBP+SB LBP+PM LBP+SB+PM

Recall

Prec

ision

60

以區域二元圖樣與部分比對為基礎之人臉辨識 Face Recognition with Local Binary...

Documents

Transcript of 以區域二元圖樣與部分比對為基礎之人臉辨識 Face Recognition with Local Binary...