Intelligent Database Systems Lab Advisor ： Dr. Hsu Graduate ： Yu Cheng Chen Author ： Yongqiang...

Intelligent Database Systems Lab

Advisor ： Dr. HsuGraduate ： Yu Cheng ChenAuthor ： Yongqiang Cao

Jianhong Wu

國立雲林科技大學National Yunlin University of Science and Technology

Projective ART for clustering data sets in high dimensional spaces

Neural Networks , 2003. Proceedings. 2002 Elsevier Science Ltd


Outline Motivation Objective Introduction Projective adaptive resonance theory Algorithms Simulation and comparisons Conclusions Personal Opinion Review

N.Y.U.S.T.I.M.


Motivation Most clustering algorithms do not work efficiently for da

ta sets in high dimensional spaces because of the inherent sparsity of data.

Consequently, a clustering algorithms is often preceded by feature selection, but a feature selection procedure can lead to a significant loss of information.

N.Y.U.S.T.I.M.


Objective PART and the resulting algorithms are proposed to find

projected clusters for data sets in high dimensional spaces.

N.Y.U.S.T.I.M.


Introduction Projected Clustering The goal of Projected clustering is to find projected

clustering, each of which consists of a subset C of data points together with a subset D of dimensions such that the points in C are closely correlated in the subspace of dimensions D.

N.Y.U.S.T.I.M.


Introduction ART1 architecture

N.Y.U.S.T.I.M.


PART architecture

N.Y.U.S.T.I.M.Introduction


Introduction The principal difference between PART and ART is

selectively sends signals in F1 layer to nodes in the F2 layer.

In other words, a node in the F1 layer can be active relative to some F2 nodes, but inactive relative to other F2 nodes.

An F1 node is active is determined by a similarity test between the corresponding top-down weight and the signal generated in the F1 node.

N.Y.U.S.T.I.M.


We define the selective output signal of node vi to node v

j by

N.Y.U.S.T.I.M.Projective adaptive resonance

(1) )()),((),,( 1 ijjiijiijiij zlzxfhzzxhh

(2) b)d(a, if 0

b)d(a, if 1),(

bah

(3) z if 0

z if 1)(

ij

ij

ijzl

We say that vi is active to vj if hij=1,and inactive to vj if hij=0


STM equations


iii Ix

dtdx

(4) 1 0

(5) )()1( jjjjjj JCxBJAxx

dtdx

(7) )(J

(6) )(J

2,

-j

j

Fvjkk

jj

k

xg

Txgwhere

user


STM equations


(8) ),,(11

Fvi

jiijiijFv

ijijj zzxhzhzTi

(9) otherwise 0

winnera is vnode if 1)( j

2

jxf

F2 layer makes a choice by winner-take –all paradigm


LTM equations


1,

2

(10) )],,(

),,()1)[((

Fvikjkkjkij

jiijiijjij

k

zzxhz

zzxLhzxfdt

dz

nactiveif vj is i

jve to v is inactii, but v is activejif v

to v is activei and v is activejif v

ijzX

XijzLijzdt

dz

j

ij

0

||

)1|(|)1(


LTM equations


)11()( 12 ted is commit)] if v(xfz[xfdt

dzjijij

ji

)12()( 12 mitted is noncom)] if v(xfz[xfdt

dzjijij

ji

(14) inactive is vif 0

active is vif )(

j

j1

ijiji xfzdt

dz

(15) inactive is vif 0active is vif )(

j

i1 ijiij xfzdt

dz


Vigilance and reset

and we reset the winner vj if and only if


(16) i

ijj hr

(17) jr


The extension of PART architecture: PART tree



Algorithms F1 activation and computation of hij

Here, we take f1(xi)=xi, and by Eq. (4), xi=Ii

N.Y.U.S.T.I.M.

(1) )()),((),,( 1 ijjiijiijiij zlzxfhzzxhh

(18) )(),( ijjiiij zlzIhh

(19) /),( bebabad


F2 activation and selection of winner

We compute the input Tj to the committed F2 node vj

by Eq. (8), and then select the winner.

N.Y.U.S.T.I.M.Algorithms

(8) ),,(11

Fvi

jiijiijFv

ijijj zzxhzhzTi


Vigilance and resetWe use the vigilance and reset mechanism show in Eqs. (16) and (17). Namely winner vj is reset if and only if


(16) i

jj hir

(17) jr

jr


LearningFor the committed winning F2 node vj which has passed the vigilance test, we have


(21) 0

) 1/(

j v toinactive is i vnode 1F if

j v toactive is i vnode 1F ifXLLz new

ij


LearningFor the committed winning F2 node vj which has passed the vigilance test, we have


(22) )1( ioldji

newji αIz-αz


LearningFor a noncommitted winner vj, and for every F1 node vi we have


(24)

(23) )1/(

inewji

newij

Iz

mLLz


PART tree algorithm



N.Y.U.S.T.I.M.Simulations and comparisons


Simulations and comparisonsN.Y.U.S.T.

I.M.


Simulations and comparisons

Data set 1 with 10,000 data points and number of clusters k=5

N.Y.U.S.T.I.M.


Simulations and comparisons N.Y.U.S.T.

I.M.


Conclusions

PART provides a solution to the feasibility-reliability dilemma in clustering data sets in high dimensional spaces.

N.Y.U.S.T.I.M.


Personal OpinionN.Y.U.S.T.

I.M.


ReviewN.Y.U.S.T.

I.M.

Intelligent Database Systems Lab Advisor ： Dr. Hsu Graduate ： Yu Cheng Chen Author ： Yongqiang...

Documents

Transcript of Intelligent Database Systems Lab Advisor ： Dr. Hsu Graduate ： Yu Cheng Chen Author ： Yongqiang...