L7-Mang Noron Nhan Tao
Transcript of L7-Mang Noron Nhan Tao
-
8/13/2019 L7-Mang Noron Nhan Tao
1/68
Hc My(IT4866)
Nguy n Nh t Quang
Trng i hc Bch Khoa H Ni
Vin Cng nghthng tin v truyn thng
Nm hc 2013-2014
-
8/13/2019 L7-Mang Noron Nhan Tao
2/68
ung m n c:
Gii thiu chung
nh gi hiu nng h thng hc my
Cc phng php hc c gim st
Cc phng php hc khng gim st
2Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
3/68
Mng n-ron nhn to Gii thiu (1)
Mng n-ron nhn to (Artificial neural network ANN)
ANN l mt cu trc (structure/network) c to nn bi mt slng cc n-ron (artificial neurons) lin kt vi nhau
Mi n-ron
C mt c tnh vo/ra c n m n o n cc m m cc
Gi tru ra ca mt n-ron c xc nh bi
Cc lin kt ca n vi cc n-ron khc
(C th) cc u vo bsung
3Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
4/68
Mng n-ron nhn to Gii thiu (2)
ANN c thc xem nhmt cu trc xl thng tin mtcch hn tn v son son mc cao
ANN c khnng hc (learn), nhli (recall), v khi qut ha(generalize) tcc dliu hc bng cch gn v iu chnh(thch nghi) cc gi trtrng s(mc quan trng) ca cclin kt gia cc n-ron
c n ng m mc u c a m c x c n
Kin trc (topology) ca mng n-ron
- Chin lc hc (hun luyn)
Dli u h c
4Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
5/68
ANN Cc ng dng in hnh (1)
Xl nh v Computer vision , , , ,
nn nh, xl v hiu cc nh thay i theo thi gian Xl tn hiu
: n t c t n u v n t a c n, ng t
Nhn dng mu V d : Trch ch n thu c tnh hn lo i v hn tch tn hi u ra-a nh n
dng v hiu ging ni, nhn dng du vn tay, nhn dng k t(chhoc s), nhn dng mt ngi, v phn tch chvit tay
V d: Phn tch v hiu tn hiu in tim, chn on cc loi bnh, v xl cc nh trong lnh vc y t
5Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
6/68
ANN Cc ng dng in hnh (2)
Cc h thng qun s- ,
Cc h thng ti chnh V d: Phn tch th trng chng khon, nh gi gi tr btng sn,
,
Lp k hoch, iu khin, v tm kim V d: Cit song song cc bi ton tha mn rng buc, tm li gii
c o o n ng a ng, u n v oa c ng n c u vngi my (robotics)
Cc h thng nng lng V d: nh gi trng thi h thng, pht hin v khc phc s c, d
on ti (khi lng) cng vic, vnh gi mc an ton
... v nhiu lnh v c bi ton khc!
6Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
7/68
Cu trc v ho
t
ng c
a m
t n
-ron Cc tn hiu u vo (input
signals) ca n-ron (xi, x0=1..
Mi tn hiu u voxignvi mt trng swi Trng siu chnh (bias)
x1
x2w0
w1w
Gi tr
w0(vix0=1)
u vo tng th(Net
input) l mt hm tch hpx
m
wm
u ra
ca
n-ronc a c c n u u v o Net(w,x)
Hm tc ng/truynCc tn u vo Hm tc
function) tnh gi tru raca n-ron f(Net(w,x))
Gi tru ra (Output) ca
hiu uvo can-ron
tng th(Net)
ng(truyn)
fn-ron: Out=f(Net(w,x)) (x)
7Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
8/68
u vo tn thv dch chu nmm
u vo tng th(net input) thng c tnh ton bi mthm tuyn tnh
==
=+=++++=i
ii
i
iimm xwxwwxwxwxwwNet01
022110 1....
ngha ca tn hiu dch chuyn (bias) w0 c c m p n c e =w1x1 ng p n c cc c
v dthnh 2 lp (two classes)
Nhng: hcc hm Net=w1x1+w0 c th!
Net = w1x
1
Net Net
Net = w1x
1+ w0
x1 x1
8Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
9/68
Hm tc ng: Gii hn cng (Hard-limiter)
Cn c gi l hm ngng(threshold function)
==
nu,1),(1)(
NetNethlNetOut
Gi tr u ra l y mt trong 2 gi tr
l gi trngng
Nh c im: khn lin t c, o
,
),(),(2)( NetsignNethlNetOut ==
Out
hm khng lin tc
hard-limiter
11
Bipolar
hard-limiter
0 Net Net0
-1
9Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
10/68
Hm tc ng: Logic ngng (Threshold logic)
< 1
if,0 Net
>
1if,1
,,,
Net
>0u)))(,1min(,0max( += Net
Cn c gi l hm tuyn tnh
0 Net- (1/)-
o a sa ura ng near unc on
Kt hp ca 2 hm tc ng:tuyn tnh v gii hn cht
1/ xc nh d c ca khong
tuyn tnh
Nhc im: Lin tc nhngo m ng n c
10Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
11/68
Hm tc
n : Xch-ma Si moidal1
),,()(
+== NetNetsfNetOut
Out
e
c dng phbin nht
1Tham s xc nh dc
Gi tru ra trong khong (0,1)
0- Net
0.5u im
Lin tc, v o hm lin tc
o m c a m m x c -mac biu din bng mt hmca chnh n
11Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
12/68
Hm tc ng: Hyperbolic tangent
121
),,tanh()()(
=
==+
Nete
NetNetOut++ ee
Out Cng hay c s dng
1
Tham s xcnh dc
Gi tr u ra trong khong (-1,1)
0- Net
uim
Lin tc, vo hm lin tc
-1c biu din bng mt hm cachnh n
1212Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
13/68
ANN Kin trc mng (1)
in ut
biasKin trc ca mt ANN c x/bi:
Slng cc tn hiu u vo v u ra
hidden
layer
Slng cc tng
Slng cc n-ron trong mi tng
Slng cc trng s(cc lin kt) i
output
layer
out ut
vi m i n-ron
Cch thc cc n-ron (trong mt tng,
hoc gia cc tng) lin kt vi nhau ng n-ron n o n n c c n u
iu chnh li
Mt ANN phi cV d: Mt ANN vi mt tng n
u vo: 3 tn hiu
Mt t ng u vo (input layer) Mt tng u ra (output layer)
Khng, mt, hoc nhiu tng n (hidden
u ra: 2 gi tr Tng cng, c 6 neurons
- 4 tng nayer s
- 2 tng u ra
13Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
14/68
ANN Kin trc mn 2 Mt tng (layer) cha mt nhm cc n-ron
Tng n (hidden layer) l mt tng nmgia tnguvo (input layer) v tngu ra (output layer)
c n ng n en no es ng ng c r ctip vi mi tr ng bn ngoi (ca mng n-ron)
nu miu r a t mt tng lin kt vi mi n-ron catng k tip
14Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
15/68
ANN Kin trc mn 3 Mt ANN c gi lmng lan truyn tin (feed-
nt lu v o ca mt nt khc thuc cng tng (hocthuc mt tng pha trc)
Khi ccu r a ca mt nt lin kt ngc li lm ccuvo ca mt nt thuc cng tng (hoc thuc mt tng
,network) Nu phn hi l lin ktu voi vi cc nt thuc cng tng,
Cc mng phn hi c cc vng lp kn (closed loops) c i l cc m n hi u recurrent networks
15Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
16/68
Kin trc m
ng V d
Mng lantruyn tin
Mt n-ron vihn hi n
mt t ng chnh n
quy mttng
Mng lantruyn tin
Mng hiquy nhiutng
16Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
17/68
ANN Cc quy tc hc
2 kiu hc trong cc mng n-ron nhn to Hc tham s(Parameter learning)
Mc tiu l thay i thch nghi cc trng s(weights) ca cclin kt trong mng n-ron
Mc tiu l thay i thch nghi cu trc mng, bao gm slng cc n-ron v cc kiu lin kt gia chng
2 kiu hc ny c thc thc hin ng thi hocring r
Ph n ln cc quy t c hc trong ANN thuc ki u hc thams
Trong bi hc ny, chng ta schxt vic hc tham s
17Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
18/68
u tc h
c tr
n s
tn ut
x = 1
Ti bc hc (t), mc iuchnh vec-t tr n swt l
wa neuron
x ...
w0
wj
w1
x
x1 Out
thun vi tch ca tn hiu hc r(t)
vu vox(t)
w(t) ~ r(t).x(t)
xwmxm
...Learning
signal
generator
d
w(t) = .r(t).x(t)
trong (>0) l tc hcTn hiu hc rl mt hm caw,x, v gi tr u ra mong mun d Lu : xjc thl:
m t tn hi uu vo ho c
r = g(w,x,d)Quy tc hc trng s tng qut
w(t) = . w(t) x(t) d(t) .x(t)
mt gi tru ra ca mt n-ron khc
18Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
19/68
Perceptron
=
Mt perceptron l mt kiun in nht ca ANNs
x1
x
w0w1
(chgm duy nht mt n-ron)
xm
w2
wm
Sdng hm tc nggii hn cht
( )
== =j
jjxwsignxwNetsignOut0
),(
i vi m t v d x i tr
u ra ca perceptron l1, nu Net(w,x)>0
- , n u ngc
19Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
20/68
Perce tron Minh ha
w0+w1x1+w2x2=0x1
u ra = 1
= -
x2
20Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
21/68
Perce tron Gii thu
t h
c
Vi mt tp cc v dhc D= {(x,d)}xl vectu vo
dl gi tru ra mong mun (-1 hoc 1)
Qu trnh hc ca perceptron nhm xc nh mt vecttrng-
hoc 1) cho mi v dhc
Vi mt v dhcxc erce tron hn l chnh xc, thvecttrng swkhng thay i
Nu d=1 nhng perceptron li sinh ra -1 (Out=-1), th wcn
, Nu d=-1 nhng perceptron li sinh ra 1 (Out=1), th wcn
c thay i sao cho gi trNet(w,x) gim i
21Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
22/68
Perce tron incrementalD _Initialize w (wian initial (small) random value)
do
for each training instance(x,d)
D
Compute the real output value Out
if(Outd)
w w+ (d-Out)x
en or
until all the training instances in D are correctly classified
2222Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
23/68
Perce tron batchD _Initialize w (wian initial (small) random value)
do
w
0for each training instance (x,d)D
Compute the real output value Out
if(Outd)
-
end for
w w + w
until all the training instances in D are correctly classified
return w
2323Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
24/68
Perce tron Gi
i hn
Gii thut hc cho perceptron c chngminh l hi t(converge) nu: M t erce tron khn
Cc v dhc l c thphn tch tuyn
tnh (linearly separable) Sdng mt tc hc nh
thphn lp chnh xc
i vi tp hc ny!
Gii thut hc perceptron c thkhng hitnu nhcc v dhc khng thphn
tch tuyn tnh (not linearly separable) Khi , p dng quy tc delta (delta rule)
m bo hi tvmt xp xph hpnh t ca hm m c tiu
Quy tc delta sdng chin lcgradient descenttm trong khng giangithit (cc vecttrng s) mt vecttrng s ph hp nh t vi cc v dhc
24Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
25/68
Hm
nh i li Error function
Xt mt ANN c n n-ron u ra
v m v c x, , g r c ra n ng error
gy ra bi vecttrng s(hin ti) w:
( )12
1)(
=
=n
i
ii OutdE wx
c g y ra vec rng s n w vton btp hc D:
= DD
D xx ww )()(
25Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
26/68
Gradient descent
Gradient ca E(k hiu lE) l mt vect C hng ch i ln (dc)
C di t l thun vi dc
Gradient Excnh hng gy ra vic tng nhanh nht (steepest
=Nwww
E ,...,,)(21
w
V vy, hng gy ra vic gim nhanh nht (steepest decrease) lgi tr ph nh ca gradient ca E
Niww ii ..1, == w= -.E(w);
Yu cu: Cc hm tcngc s dng trong mng phi l cc ,
26Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
27/68
Gradient descent Minh h
aKhn ian 2 chiu
E(w) E(w1,w2)
27Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
28/68
Gradient_descent_incremental (D, )
Initialize w (wian initial (small) random value)
do
for each training instance (x,d)D
Compute the network output
i
wi wi (Ex/wi)
end for
end for
until (stopping criterion satisfied)
return w
28
, , ...
28Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
29/68
ANN nhiu tng v gii thut lan truyn ngc
Mt perceptron chc thbiu din mt hm phn tch tuyn tnh(linear separation function)
Mt mng n-ron nhiu tng (multi-layer NN) c hc bi gii thutlan truyn ngc (back-propagation -BP- algorithm) c thbiu dinmt hm phn tch phi tuyn phc tp (highly non-linear separationfunction)
Gii thut hc BP c sdng hc cc trng sca mt mngn-ron nhiu tn Cu trc mng cnh (cc n-ron v cc lin kt gia chng l cnh)
i vi mi n-ron, hm tc ng phi c o hm lin tc
t u t p ng c n c gra en escen trong quy t c c pnht cc trng s cc tiu ha li (khc bit) gia cc gi tru ra thc tv cc gi tr
,
29Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
30/68
Gii thu
t h
c lan tru
n n
c 1
Gii thut hc lan truyn ngc tm kim mt vect cc trngs (weights vector) gip cc tiu ha li tng th ca h
ng v p c
Gii thut BP bao gm 2 giaion (bc) Giaio n lan tru n tin tn hi u Si nal forward . Cc tn hi u
u vo (vect cc gi tr u vo) c lan truyn tin t tngu von tngu r a (i qua cc tngn)
Giaio n lan tru n n c li Error backward
Cn c vo gi tr u ra mong mun ca vect u vo, h thng tnhton gi tr li
Btu t tn u ra, i tr li c lan tru n n c ua m n , t
tng ny qua tng khc (pha trc), chon tngu vo Vic lan truyn ngc li (error back-propagation) c thc hin
thng qua vic tnh ton (mt cch truy hi) gi tr gradient cc b ca-
30Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
31/68
Gii thut hc lan truyn ngc (2)
tn hiu: Kch hot (truyn tn hiuqua mng
Giaion lan truyn
Tnh ton liu ra Lan truyn (ngc) li
31Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
32/68
Gii thut BP Cu trc mng
x1 x xmIn utx
Xt mng n-ron 3 tng (tronghnh v) minh ha gii thut
wqj
... ...
(j=1..m)
m tn hiu u voxj(j=1..m) l n-ron tng n zq (q=1..l)
neuron zq(q=1..l)
wiq
Outq... ...
- i ..
wqj l trng sca lin kt ttn hiu u voxj ti n-ron
tn n z
... ...Outputneuron yi
=
wiq l trng sca lin kt tn-ron tng n zq ti n-ronu ra yi
i..
Outq l gi tru ra (cc b)ca n-ron tng n zq Outi l gi tru ra ca mng
- i
32Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
33/68
Gii thut BP Lan tru n tin 1i vi mi v dhcx
Vectu vox c lan tru n ttn u vo n tn u ra
Mng ssinh ra mt gi tru ra thc t(actual output) Out(lmt vectca cc gi trOuti, i=1..n)
m
i vi mt vect u vox, mt n-ron zqt ng n snhn c gi tru vo tng th(net input) bng:
=
=j
jqjq xwNet1
v sinh ra mt gi tru ra (cc b) bng:
==
=
m
j
jqjqq xwfNetfOut1
)(
. - q
33Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
34/68
Gii thut BP Lan tru n tin 2
Gi tru vo tng th(net input) ca n-ron yitng
= ==
==
l
q
m
j
jqjiq
l
q
qiqi xwfwOutwNet1 11
N-ron yi sinh ra gi tru ra (l mt gi tru ra ca
mng)
=
== = == q
m
j
jqjiq
q
qiqii xwfwfOutwfNetfOut1 11
)(
Vectcc gi tru ra Outi(i=1..n) chnh l gi tru rathc tca mng, i vi vectu vox
34Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
35/68
Gii thut BP Lan truyn ngc (1)
i vi mi v dhcx Cc tn hi u li error si nals do s khc bit ia i tr u ra
mong mun dv gi tru ra thc tOutc tnh ton
Cc tn hiu li ny c lan truyn ngc (back-propagated) ttn u r a ti cc tn ha tr c c nh t cc tr n s(weights)
xt cc tn hiu li v vic lan truyn ngc ca
( ) ( ) ( )[ ] ==n
ii
n
ii NetfdOutdwE22 11
chng, cn nh ngha mt hm nh gi li
== ii 11
=n l
qiqi Outwfd
2
1
= =q
35Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
36/68
Gii thut BP Lan truyn ngc (2)
Theo phng php gradient-descent, cc trng s ca cc linkt t tn n ti tn u ra c c nh t bi
iq
iqw
Ew
=
[ ] ( )[ ][ ] qiqiiiii
iq OutOutNetfOutdw
Net
Net
Out
Out
E
w ==
= '
ng quy c c u o m v wiq, a c
(Lu : du c kt hp vi gi tr E/Outi)
i l tn hiu li (error signal) ca n-ron yi tngu ra[ ] ( )[ ]iii
i
i
ii
i NetfOutdNetOut
OutE
NetE '=
=
=
trong Net lu vo tng th (net input) ca n-ron ytngu ra, v f'(Neti)=f(Neti)/Neti
36Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
37/68
Gii thut BP Lan truyn ngc (3)
cp nht cc tr ng s ca cc lin kt t tngu,
gradient-descent v quy tc chuio hm
=
=qj
q
q
q
qqj
qjw
e
Net
u
Outww
T cng thc tnh hm l i E(w), ta th y r ng m i thnhphn li (di-yi) (i=1..n) l mt hm ca Outq
= =
= n
i
l
q
qiqi OutwfdE1 12
1)(w
37Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
38/68
Gii thut BP Lan truyn ngc (4)
p dng quy tc chui o hm, ta c
( ) ( )[ ] ( ) jqi
iqiiiqj xNetfwNetfOutdw ''1
==
n
' jqjqi
iqi ===1
q
l tn hiu li (error signal) ca n-ron zq
tng n
( ) iqn
i
iq
q
q
qq
q wNetfNet
Out
Out
E
Net
E
=
=
=
=
1
'
trong Netq l u vo tng th(net input) ca n-ron zqtngn, v f'(Netq)=f(Netq)/Netq
38Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
39/68
Gii thut BP Lan truyn ngc (5)
Theo cc cng thc tnh cc tn hiu li i v q nu, th tnhiu li ca mt n-ron tng n khc vi tn hiu li ca mtn-ron tng u ra
Do skhc bit ny, thtc cp nht trng strong gii thutBP c i l u tc h c delta t n ut
Tn hiu li q ca n-ron zqtng n c xc nh bi
Cc tn hiu li
ica cc n-ron yitng u ra (m n-ron zq ,
Cc hschnh l cc trng swiq
c im quan trng ca gii thut BP: Quy tc cp
nht trng sc tnh cc b tnh ton thay i (cp nht) trng sca mt lin kt, h
thn chcn sd n cc i tr 2 u ca lin kt !
39Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
40/68
Gii thut BP Lan truyn ngc (6)
Qu trnh tnh ton tn hiu li (error signals) nhtrn cthc mrn khi ut ddn i vi mn n-ron c nhiu hn 1 tng n
D n tn ut ca u tc c nh t tr n stron iithut BP l:
wab = axb b v a l 2 chstng ng vi 2 u ca lin kt (ba) (tmt
n-ron (hoc tn hiu u vo) bn n-ron a)
- ,
a l tn hiu li ca n-ron a
40Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
41/68
Back_propagation_incremental(D, )Mng n-ron gmQ tng, q = 1,2,...,Q
qNetiv qOuti lu vo tng th (net input) v gi tr u r a ca n-ron itng q
Mng c m tn hiuu vo v n n-ronu raqwij l trng s ca lin kt t n-ronjtng (q-1)n n-ron itng q
Bc 0 (Khi to)
Chn ngng li Ethreshold(gi tr li c th chp nhnc)
Gn E=0
Bc 1 (Btu mt chu k hc)
p ng vec u v o c a v c v ng u v o q=qOuti = 1Outi = xi(k), i
Bc 2 (Lan truyn tin)
== qqqq OutwNetOut 1
Lan truyn tin cc tn hiuu vo qua mng, chon khi nhnc ccgi tr u r a ca mng (tngu ra) QOuti
j
41H
c My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
42/68
Bc 3 (Tnh ton liu ra)
Tnh ton liu r a ca mng v tn hiu li Q ca mi n-rontngu ra
=
+=n
i
i
Qk
i OutdEE1
2)( )(2
1
' QQ(k)Q = iiiiBc 4 (Lan truyn ngc li)
Lan truyn ngc li cp nht cc tr ng s v tnh ton cc tn hiu li q-1icho cc tng pha trc
2,...,1,allfor;
11
==
QQqw)Net'(f jq
ji
q
i
q
i
q
qwij = .(qi).(q-1Outj); qwij = qwij + qwij
jBc 5 (Kim t rakt thc mt chuk hc epoch)
Kim tra xem ton b tp hc c s dng ( xong mt chu k hc epoch)
Nu ton b tp hc c dng, chuynn Bc 6; ngc li, chuynn Bc 1
Bc 6 (Kim t ra li tng th)Nu li tng th E nh hn ngng li chp nhnc (
-
8/13/2019 L7-Mang Noron Nhan Tao
43/68
Gii thut BP Lan truyn tin (1)
f(Net1)
1x f(Net4)
f(Net2)
Out6f(Net6)
x f(Net5)
f(Net3)
43Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
44/68
Gii thut BP Lan truyn tin (2)
f(Net1)
1x
1x
21 2xw x f(Net4)
Out6
f(Net2)
f(Net6)
x f(Net5)
)( 21111 xwxwfOut xx +=f(Net3)
44Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
45/68
Gii thut BP Lan truyn tin (3)
f(Net1)
1x f(Net4)12 1
xw xOut6
f(Net2)
f(Net6)
x22 2
xw x f(Net5)
)( xwxwfOut +=f(Net3)
45Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
46/68
Gii thut BP Lan truyn tin (4)
f(Net1)
1x f(Net4)
Out6f(Net
2)
f(Net6)
2x 13 1xw xf(Net5)
23 2xw x )( 23133 xwxwfOut xx +=
f(Net3)
46Hc My IT4866
G h )
-
8/13/2019 L7-Mang Noron Nhan Tao
47/68
Gii thut BP Lan truyn tin (5)
f(Net1)
1x 242Outw
141
f(Net4)
343OutwOut6
f(Net2)
f(Net6)
2xf(Net5)
)( OutwOutwOutwOut ++=f(Net3)
47Hc My IT4866
Gii h BP L i (6)
-
8/13/2019 L7-Mang Noron Nhan Tao
48/68
Gii thut BP Lan truyn tin (6)
f(Net1)
1x 151Outw f(Net4)
Out6f(Net
2)
f(Net6)
2x252 uw
f(Net5)
353Outw
OutwOutwOutwOut ++=f(Net3)
48Hc My IT4866
Gii h BP L i (7)
-
8/13/2019 L7-Mang Noron Nhan Tao
49/68
Gii thut BP Lan truyn tin (7)
f(Net1)
1x f(Net4)
Out6
f(Net2)
f(Net6)
x565Outwf(Net5)
OutwOutwOut +=f(Net3)
49Hc My IT4866
Gii h BP T h li
-
8/13/2019 L7-Mang Noron Nhan Tao
50/68
Gii thut BP Tnh ton li
f(Net1)
1x f(Net4) Out6
f(Net2)
f(Net6)
2xf(Net5)
[ ] ( )[ ]666
6 ' NetfOutdOutEE
=
=
=f(Net3)
output value
666 etutet
50Hc My IT4866
Gii h BP L (1)
-
8/13/2019 L7-Mang Noron Nhan Tao
51/68
Gii thut BP Lan truyn ngc (1)
f(Net1)
1x4
f(Net4)
6Out6
f(Net2)
f(Net6)
x f(Net5)
'=f(Net3)
66444
51Hc My IT4866
Gii th t BP L t (2)
-
8/13/2019 L7-Mang Noron Nhan Tao
52/68
Gii thut BP Lan truyn ngc (2)
f(Net1)
1x f(Net4)
Out6
f(Net2)
f(Net6)
x65w
5
f(Net5)
))(w'(Netf 66555=f(Net3)
52Hc My IT4866
Gii th t BP L tr (3)
-
8/13/2019 L7-Mang Noron Nhan Tao
53/68
Gii thut BP Lan truyn ngc (3)
1
x41w
w
4
1
Out6f(Net )5
2
f Net
2
'=f Net 55144111
53Hc My IT4866
Gii thut BP Lan truyn ngc (4)
-
8/13/2019 L7-Mang Noron Nhan Tao
54/68
Gii thut BP Lan truyn ngc (4)
f(Net1)
1x 42w f(Net4)
w
2 Out6
f(Net2)
f(Net6)
x
5
f(Net5)
)w)(w'(Netf 55244222 +=f(Net3)
54Hc My IT4866
Gii thut BP Lan truyn ngc (5)
-
8/13/2019 L7-Mang Noron Nhan Tao
55/68
Gii thut BP Lan truyn ngc (5)
f(Net1)
1x f(Net4)
43w
Out6
f(Net2)
f(Net6)
x 53w5
f(Net5)
)w)(w'(Netf 55344333 +=3
f(Net3)
55Hc My IT4866
Gii thut BP Cp nht trng s (1)
-
8/13/2019 L7-Mang Noron Nhan Tao
56/68
Gii thut BP Cp nht trng s (1)
1f(Net1)
1x
1x
w f(Net4)2
Out6
f(Net )f(Net6)
x f(Net5)
1111 11xww xx +=
f(Net3)
2111 22 xx
56Hc My IT4866
Gii thut BP Cp nht trng s (2)
-
8/13/2019 L7-Mang Noron Nhan Tao
57/68
Gii thut BP Cp nht trng s (2)
f(Net1)
1x f(Net4)
12xw 2 Out6
f(Net2)f(Net6)
2x22x
wf(Net5)
1222 11 xww xx
=+=f(Net3)
2222 22 x
57Hc My IT4866
Gii thut BP Cp nht trng s (3)
-
8/13/2019 L7-Mang Noron Nhan Tao
58/68
Gii thut BP Cp nht trng s (3)
f(Net1)
1x f(Net4)Out6
f(Net2)f(Net6)
2x 13xw f(Net5)
23xw 1333 11
xww
xww xx
+=+=f(Net3)
22
58Hc My IT4866
Gii thut BP Cp nht trng s (4)
-
8/13/2019 L7-Mang Noron Nhan Tao
59/68
Gii thut BP Cp nht trng s (4)
f(Net1)
1x 42w41
f(Net4)
43wOut6
f(Net2)f(Net6)
2xf(Net5)
244242
144141
Outww +=
=f(Net3)
344343
59Hc My IT4866
-
8/13/2019 L7-Mang Noron Nhan Tao
60/68
Gii thut BP Cp nht trng s (6)
-
8/13/2019 L7-Mang Noron Nhan Tao
61/68
Gii thut BP Cp nht trng s (6)
f(Net1)
1x f(Net4)64
Out6f(Net2)
f(Net6)
2x65w
f(Net5)
466464 Outww +=f(Net3)566565
61Hc My IT4866
BP: Khi to gi tr ca cc trng s
-
8/13/2019 L7-Mang Noron Nhan Tao
62/68
BP: Khi to gi tr ca cc trng s
Thng thng, cc trng s c khi to vi cc gi tr nhngu nhin
Nu cc tr ng s c cc gi tr ban u ln Cc hm xch-ma (sigmoid functions) s t trng thi bo ha sm
H thng s tcmtim cc tiu cc b (local minimum) hocmt trng thi khngi (very flat plateau) gnim btu
0 - -a K hiu na l s lng cc n-roncng tng vi n-ron a
w0ab [1/na, 1/na]
0
K hiu ka l s lng cc n-ron c lin kt (tin) n n-ron a(=s lng cc lin ktu vo ca n-ron a)
, aa
62H
c My IT4866
BP: Tc hc Learnin rate
-
8/13/2019 L7-Mang Noron Nhan Tao
63/68
BP: Tc hc Learnin rate nh hng quan trng n hiu qu v kh nng hi t ca gii thut hc
BP
g r n c y n an s c a qu r n c, n ng cth lm cho h thng b qua im tiu ton cc hoc ri voim ti
u cc b
Thngc chn theo thc nghim (experimentally) i vi mi bi ton
Cc gi tr tt ca tc hclc btu (qu trnh hc) c th khng tt
m t thiim sau Nn s dng mt tc hc thch nghi (ng)
Sau khi cp nht cc tr ng s, kim tra xem vic cp nht cc tr ng s c
=a , if E < 0 consistently-b , if E > 0 (a, b > 0)
0 , otherwise.
63H
c My IT4866
BP: Momentum
-
8/13/2019 L7-Mang Noron Nhan Tao
64/68
BP: Momentum
- E t+1 +w tw(t)
-E(t+1)
w t
Phng php Gradient descentcth rt chm nunh, v c th
AA
w(t)
Bao ng mn n u qu n
gim mc dao ng, cna vo mt thnh phn
B
w(t)
-E(t+1)
momentum
w(t) = -E(t) + w(t-1)
tron 0 1 l m t tham s - w
Gradient descent i vi mt hmli bc 2 n gin.
momentum (thng ly =0.9)
Mt quy tc, da trn cc th
Qu o bn tri khng s dngmomentum.
Qu o bn phi c s dng
,
cho tc hc v momentum l:(+) > 1; trong> trnh
momentum.
64Hc My IT4866
BP: S lng cc n-ron tng n
-
8/13/2019 L7-Mang Noron Nhan Tao
65/68
g g
Kch thc (s n-ron) ca tngn l mt cu hi quan trngi vi vic p dng cc mng n-ron lan truyn tin nhiu tng gii quyt cc bi ton thc t
Trong thc t, rt kh xcnh chnh xc s lng cc n-ron cn thit t c m t chnh xc mon mun ca hthng
Kch thc ca tngn thngc xcnh qua th nghim
Gi Btu vi s l n nh cc n-rontn n =t l nh so vi s
lng cc tn hiuu vo) Nu mng khng th hi t, b sung thm cc n-ron vo tngn
, -
65Hc My IT4866
ANNs Gii hn hc
-
8/13/2019 L7-Mang Noron Nhan Tao
66/68
ANNs Gii hn hc
Cc hm nhphn (Boolean functions) Bt khm nhphn no cng c th hc c bi mt ANN s
dng 1 tng n
c m n c on nuous unc ons
Bt kmt hm lin tc bgii hn (bounded continuous function)
no cn c thh c c x x bi m t ANN sd n 1 tnn [Cybenko, 1989; Hornik et al., 1989]
Bt kmt hm mc tiu no cng c thhc c (xp x) bi,
66Hc My IT4866
ANNs u im, Nhc im
-
8/13/2019 L7-Mang Noron Nhan Tao
67/68
ANNs u im, Nhc im
Cc u im
Khnng chu nhiu/li, nhkin trc tnh ton song song
C thc thit ktthch nghi (cc trng s, cu trc mng)
Cc nhc im Khng c quy tc tng qut xc nh cu trc mng v cc
am s c u c o m p o n n n
Khng c phng php tng qut nh gi hot ng bntrong ca ANN (v vy, hthng ANN bxem nhmt hp en)
Rt kh (khng th) a ra gii thch cho ngi dng
Rt kh don hiu nng ca hthng trong tng lai (khnng khi qut ha ca hthng hc)
67Hc My IT4866
ANNs Khi no?
-
8/13/2019 L7-Mang Noron Nhan Tao
68/68
NNs Kh o?
Mi v dc biu din bi mt tp gm (rt) nhiu
Min gi tru ra ca hm mc tiu c kiu sthc, hoc,
Tp dliu c thcha nhiu/li
Khng cn thit (hoc khng quan trng) phi a ra gii
thch cho n i dn i vi cc kt u
Chp nhn thi gian (kh) lu cho qu trnh hun luyn
68Hc My IT4866