Chin-Hsien Fang(方競賢 ), Ju-Chin Chen(陳洳瑾 ), Chien-Chung Tseng(曾建中 ),and Jenn-Jier James Lien(連震杰 )

Department of Computer Science and Information Engineering,National Cheng Kung University

HUMAN ACTION RECOGNITION IN TEMPORAL-VECTOR TRAJECTORY LEARNING FRAMEWORK

1

+ Motivation+ System flowchart+ Training Process+ Testing Process+ Experimental Results + Conclusions

2

+ Traditional Manifold classification (ex: LDA , LSDA…) *Only spatial information *The input data are continuous sequences *Temporal information should be considered

3

ASM

h*w

d

d*(2t+1)

d*(2t+1)

h*w

d

d*(2t+1)

d*(2t+1)

4

LPP

Temporal data

Metric Learning

5

+ Why dimension reduction?– To reduce the calculation cost

+ Why LPP (Locality Preserving Projections)?– Can handle non-linear data with linear transformation matrix

– Local structure is preserved

6

Try to keep the local structure while reducing the dimension

7

ijij

jT

iT Wxaxa

2)(minarg

ij

TTijj

Ti

T aXLXaWxaxa 2)(2

1

1aXDXa TTSubject to

Where L = (D - W)

Objective function:

L : Laplacian matrixD : Diagonal matrixW : Weight matrix

8

+ Three kinds of temporal information1. LTM(Locations temporal motion of Mahalanobis

distance)2. DTM(Difference temporal motion of Mahalanobis

distance)3. TTM(Trajectory temporal motion of Mahalanobis

distance)

9

LTM

An input sequence: },......,{ 21 nxxxX LPP

},......,{ 21 nyyyY

Temporal

}',......','{' 21 nyyyY

],...,,,,...,[' 11 tiiiitii yyyyyy where

10

DTM

}',......','{' 21 nyyyY

],...,,,,...,[' 11 tiiiiiiitiii yyyyyyyyyy

where

1iy

1iy

iy1 ii yy1 ii yy

11

TTM

}',......','{' 21 nyyyY

],...,,,,...,[' 1111 titiiiiiititii yyyyyyyyyy

where

1iy 1iy

iy

ii yy 1

1 ii yy

2iy21 ii yy

12 ii yy

2iy 12

+ Mahalanobis distance1. Preserving the relation of the data

2. Doesn’t depend on the scale of the data

13

yiyi

yj

yl

yj

yi

yl

yi

yj

yl

LME Space

LMNN

LPP+Temporal Space

)12*( tdiy

ii yy E

iy

)12*( tdiy

14

ijk

ijkikijij

jiT

jiij YYMYY )1()()( ''''

Minimize :

Subject to :

ijkjiT

jikiT

ki YYMYYYYMYY 1)()()()( ''''''''

0ijk

(i)

(ii)

(iii) M has to be positive semi-definite

LPP

Metric Learning

Temporal data

K-NN

15

Test data

Training data

3

1

1

K=5

The winner takes all~~

Labeled as

16

The number of nearest neighbor

17

18

+ Our TVTL framework makes impressive progress compared to other traditional methods such as LSDA

+ Temporal information do have positive influence

+ DTM , TTM are better than LTM because they consider the correlation of the data

19