3DHeadPose

8/10/2019 3DHeadPose

1/9

L. Wang and Y. Jin (Eds.): FSKD 2005, LNAI 3614, pp. 293301, 2005.

Springer-Verlag Berlin Heidelberg 2005

3-D Head Pose Estimation for Monocular Image

Yingjie Pan, Hong Zhu, and Ruirui Ji

Department of Automation and Information Engineering,

Xian University of technology 710048, Xian, China Tel: [email protected] [email protected]

[email protected]

Abstract. 3-D head pose estimation plays an important role in many

applications, such as face recognition, 3-D reconstruction and so on. But it is

very difficult to estimate 3-D head pose only from a single monocular imagedirectly without other auxiliary information. This paper proposes a new human

face pose estimation algorithm using a single image based on pinhole imaging

theory and 3-D head rotation model. The key of this algorithm is to obtain 3-D

head pose information based on the relations of projections and the positions

changing of seven facial points. Experiments show the proposed method has

good performance in both accuracy and robustness for human face pose

estimation using only a single monocular image.

1 Introduction

Human face pose estimation is a key problem in many practical applications of

computer vision and image processing. Some research achievements have been made

on this problem, however, most of these solutions need additional information such as

front face view, stereo vision, 3D head model and so on. Considering that it is too

hard and too expensive to get these additional information in practical application

system, a novel algorithm to analyze human face pose directly from a single image

should be designed.This paper proposes a new human face pose estimation approach from a single face

image under arbitrary pose using face geometry characteristic based on pinhole

imaging theory[1,2]and 3-D head rotation model.

2 Human Face Pose Estimation

Human face pose estimation is to calculate the rotation angles of the head with respect

to front face along the X, Y, and Z-axis respectively. As human head is a non-rigidobject in three-dimensional space, the face picture presents us only two-dimensional

information, it can be considered from the imaging theory that face image is formed

by human head project onto X-Y plane along Z-axis. So the image plane coincides

with the X-Y plane. We can use (pitch, yaw and roll, respectively ) to denote

the three rotation angles of the head about the X, Y and Z-axis respectively.


2/9

294 Y. Pan, H. Zhu, and R. Ji

Fig. 1.Facial points

From anthropometry, as Fig. 1 shows, human face is basically bilateral symmetry,

two eyes are the same both in size and shape, and the four eye corners are

approximately collinear in three-dimension. While =0, i.e. the rotation angle around

Z-axis is zero, this line is parallel to X-Y plane, and also parallel to X-axis of the face

image. Furthermore, in order to describe human head position, the symmetric points

(em, nm and mm shown in Fig.1) of eyes, mouth and nose also must be given, as

shown in Fig. 1. According to the position change of the seven points listed above, we

can calculate the pose of human face directly.

Since human face pose can be described as three rotation angles of the head about

the X, Y and Z-axis respectively in three-dimension, we can get human face pose

from these angles.

2.1 Roll (The Rotation Angle of Z-Axis, )

As Fig.2 shows, roll is the rotation angle of Z-axis, and roll () can be straightforward

recovered by the eye corners in the image.

X

X X

YY

Z

symmetricalline

symmetrical

line

?

Fig. 2.The rotation angle of Z-axis


3/9

3-D Head Pose Estimation for Monocular Image 295

1 2 3( ) / 3 = + + (1)

1 arctan( )ell elr

ell elr

y y

x x

=

(2)

2 arctan( )erl err

erl err

y y

x x

=

(3)

3

( ) / 2 ( ) / 2arctan( )

( ) / 2 ( ) / 2

ell elr erl err

ell elr erl err

y y y y

x x x x

+ +=

+ +

(4)

Set (xell, yell), (xelr, yelr), (xerl, yerl), and (xerr, yerr) denote left corner of left eye, right

corner of left eye, left corner of right eye, right corner of right eye respectively.

2.2 Yaw (The Rotation Angle of Y-Axis )

As Fig.3 shows, yaw is the rotation angle of Y-axis. From anthropometry, it can be

known that the four eye corners are collinear and the two eyes have the same size, that

is: || Ell Elr || || Erl Err ||.

Y

X

Z

X

Zsymmetrical

line

symmetrical

line

Fig. 3.The rotation angle of Y-axis

Based on pinhole imaging theory, Fig.4 shows the principal of calculating the

Yaw-. From the projective invariance of the cross-ratios, the following can be

derivated:2

1 2

1

( )( )

( )( ) (2 )

erl err ell elr

ell erl elr err

x x x x DI

x x x x D D

= =

+

(5)

1D =DQ/2 (6)

1

1Q= 1

I

(7)


4/9


Fig. 4.The principle for calculating

Fig. 5.Head rotation

Y

X X

Z

symmetrical

line

symmetrical

line

r

xm2l xm2r

Fig. 6.3-D head rotation model


5/9

3-D Head Pose Estimation for Monocular Image 297

1

2M

Q

=

+

(8)

1 01elr

1 1 0

cosfXx =Z+Z sin

fDZ D

=+

(9)

1 02ell

2 1 0

( )cosfXx =

Z+Z ( )sin

f D D

Z D D

+=

+ +

(10)

1 01erl

1 1 0

cosfXx =

Z-Z sin

fD

Z D

=

(11)

1 02err

2 1 0

( ) cosfXx =

Z-Z ( )sin

f D D

Z D D

+=

+

(12)

11 1

1

( 1)elr

fXZ Z Z S

x Z= = (13)

S can be got from equation (14).

( 1)( (1 2 / ))( 1)( (1 2 / ))

ell elr

erl err

x x S S Qx x S S Q

+= + + +

(14)

2 2( )( ) ( ) ( )( )

( )( ( ) ( ))

ell elr erl err elr erl ell err elr erl

ell elr elr erl erl err

x x x x M x x M x x x xu

x x M x x x x

=

(15)

Where f is the focus of the camera, D D1 Z Z1 Z2 X1 X2 are distance

parameters shown in Fig. 4. From formulation (5)-(15), the formulation to calculate 0

can be got:

0

f=arctan

(S-1)u

(16)

However, this method is influenced easily and badly by the point positions,

especially great deviation will occur when >450. For example, as Fig. 5 shows, 0

calculated by this method is 40.530, the actual yaw is 60

0, and the error is 19.47

0. To

solve this problem, 3-D head rotation model is introduced. Human head is

approximately regarded as a round cylinder, because human face is bilateral

symmetry, the symmetric line of face will be coincident with the center line of the

cylinder in front human face view; and when human head rotates around Y-axis, there

will be offset between the symmetric line and the center line, shown as Fig.6. With

this offset, the rotation angle around Y-axis can be calculated.


6/9


At first, calculating the rotation angle around Y-axis at eyes, nose and mouth

respectively according to the formulation followed:

xx [ min(x 2 , x 2 )]

2

lm l m r =

(17)

x

xsin( )

x / 2acr

l

=

(18)

Where lx could be le, ln and lm, which correspond to the face width at eyes, noseand mouth points respectively; xm2l and xm2r could be em2l, em2r, nm2l, nm2r,mm2l and mm2r respectively, which is the distance from the symmetric points ateyes, nose and mouth to the left edge and right edge of face respectively.

Considering practical applications, the rotation angle around Y-axis could be set in

the range of [-90o,+90o]. In the situation of heavy incline, such as is near -90o or+90

o, one eye is covered partly or totally, the symmetric points are near the face

contour, the value of min(xm2l,xm2r) is near 0, andx

is near 90o.

Then get the mean value of yaws at eyes, nose and mouth according to theformulation as follows:

1 ( ) / 3e n m = + + (19)

Finally, yaw is determined with different formulations in different cases.

00 1 1

0 0

1 1

| || 45

45 || || 90

+

3DHeadPose

Documents

Transcript of 3DHeadPose