Artificial Neural Network - Back Propagation of Error

Artificial Neural Network - Back Propagation of Error

LI M UT khi ra i, my tnh nhanh chng pht trin v ng mt vai tr rt quan trng trong nghin cu khoa hc k thut cng nh trong i sng.

Nhng mt my tnh d c mnh n u chng na, cng ch c th lm vic theo mt chng trnh c hoch nh sn bi lp trnh vin. N vn khng c kh nng lin tng, kt ni s vic ny vi s vic khc, v quan trng hn ht l kh nng sng to nh con ngi.

Mng nron nhn to (Artificial neural networks ) ra i xut pht t tng m phng hot ng ca b no con ngi.

Mng noron nhn to l s ti to bng k thut nhng chc nng ca h thn kinh con ngi vi v s cc nron c lin kt truyn thng vi nhau qua mng. Ging nh con ngi, ANN c hc bi kinh nghim, lu nhng kinh nghim v s dng trong nhng tnh hung ph hp.

Mng nron trong mt vi nm tr li y c nhiu ngi quan tm v p dng thnh cng trong nhiu lnh vc khc nhau, nh ti chnh, y t, a cht v vt l, d bo, phn loi... Kt hp cht ch vi logic m, mng nron nhn to to nn cuc cch mng thc s trong vic thng minh ha v vn nng ha cc b iu khin k thut cao cho c hin nay v trong tng lai. V d nh ng dng t ng iu khin h thng li tu, h thng d bo s c,

Mng nron da trn vic m phng cp thp h thng nron sinh hc. Trong tng lai vi s pht trin m phng nron sinh hc, chng ta c th c loi my tnh thng minh tht s.

V l do nhm 11 chng em la chn ti Tm hiu k thut mng n ron nhn to. i su mng Back Propagation of Error. Ci t minh ha. y l mt ti kh ln v phc tp, song trong phm vi mt bi bo co ngn, chng em xin trnh by nhng kin thc mc c bn nht.

hon thnh bi bo co ny, chng em nhn c s hng dn nhit tnh ca ThS. Nguyn Phng Nga ging vin b mn Tnh ton mm lp KHMT2-K6 chng em. Nhng bi ging v ti liu ca c chnh l c s chng em c th hon thnh tt bo co ca mnh. Chng em xin chn thnh cm n c!

Nhm thc hinMC LCPHN 1: GII THIU CHUNG V TNH TON MM4

1. Mt s c im ca cng ngh tnh ton mm4

2. Lch s pht trin5

3. ng dng tnh ton mm5PHN 2: GII THIU V ANN6

I. LCH S PHT TRIN CA MNG NRON6

II. MNG NRON SINH HC8

III. MNG NRON NHN TO10

1. Khi nim10

2. Cc kin trc mng12

143. Perceptron

153.1 Lut hc Perceptron

153.2 Gii thut hc Perceptron

164. Mng nhiu tng truyn thng (MLP)

175. Mng Hopfied

6. Xy dng mng nron18

6.1 Chun b d liu18

216.2 Xc nh cc tham s cho mng:

236.3 Vn lng qun (catastrophic forgetting)

246.4 Vn qu khp

257. Hun luyn mng nron:

7.1 Hc c gim st25

267.2 Hc khng gim st

277.3 Hc tng cng

27IV. NG DNG CA MNG NORON

PHN 3: MNG LAN TRUYN NGC SAI S28

28I. TNG QUAN V MNG LAN TRUYN NGC LI

29II.GII THUT HC BP

1. Gii thut hun luyn mng29

312. Thut ton tng qut

323. S gii thut hc BP

324. K thut hc c gim st:

34PHN 4: BI TON MINH HA

341. Gii thiu bi ton

342. Cc bc ci t bi ton bng lnh trong Matlab

373. Ci t bi ton s dng cng c nntool trong Matlab

42KT LUN

43TI LIU THAM KHO

44 KIN NHN XT CA GING VIN HNG DN

PHN 1: GII THIU CHUNG V TNH TON MM

Trong thc t cuc sng, cc bi ton lin quan n hot ng nhn thc, tr tu ca con ngi u hm cha nhng i lng, thng tin m bn cht l khng chnh xc, khng chc chn, khng y . V d: s chng bao gi c cc thng tin, d liu cng nh cc m hnh ton y v chnh xc cho cc bi ton d bo thi tit. Nhn chung con ngi lun trong bi cnh l khng c thng tin y v chnh xc cho cc hot ng ra quyt nh ca bn thn mnh.

Trong lnh vc khoa hc k thut cng vy, cc h thng phc tp trn thc t thng khng th m t y v chnh xc bi cc phng trnh ton hc truyn thng. Kt qu l nhng cch tip cn kinh in da trn k thut phn tch v cc phng trnh ton hc nhanh chng t ra khng cn ph hp. V th, cng ngh tnh ton mm chnh l mt gii php trong lnh vc ny.

Cng ngh tnh ton mm (Soft Computing SC) l mt h thng c t chc cht ch. H thng ny l s hp nht ca cc phng php c thit k m hnh ha nhm gii quyt cc bi ton thc khng th hoc qu kh m hnh ha bng ton hc.

ch ca cng ngh tnh ton mm (SC) l pht hin v tm li gii vi bi ton kh do s m h, khng chc chn, xp x v ch ng mt phn ca d liu, nhm t c s tng ng vi kh nng tr tu ca con ngi v l gii v hc trong mi trng m h.1. Mt s c im ca cng ngh tnh ton mm:

- Tnh ton mm cn c trn cc c im, hnh vi ca con ngi v t nhin a ra cc quyt nh hp l trong iu kin khng chnh xc v khng chc chn to ra cc my thng minh (th h mi ca TTNT).

- Cc thnh phn ca tnh ton mm c s b sung, h tr ln nhau to h thng c t chc cht.

- Tnh ton mm l mt hng nghin cu m, bt k mt k thut mi no c to ra t vic bt trc tr thng minh ca con ngi u c th tr thnh mt thnh phn mi ca tnh ton mm.

- Cng ngh tnh ton mm bao gm 4 thnh phn chnh: Fuzzy Computing (FC), Evolutionary Computing (EC), Neural Computing (NC) and Probabilistic Computing (PC - cn ang pht trin).

2. Lch s pht trin:

SC

=

EC + NN+FL

Soft

Evolutionary Neural Fuzzy

Computing

Computing Netwwork Logic Zadeh

Rechenberg McCulloch Zadeh

1981

1960

1943

1965

EC =

GP + ES

+EP+ GA Evolutionary Genetic Evolution Evolutionary Genetic Computing

Programming Strategies Progamming Algorithms

Rechenberg Koza Rechenberg Fogel Holland

1960

1992

1965 1962

19703. ng dng ca tnh ton mm:Cc nghin cu gn y v tnh ton mm l bc pht trin ca cng ngh tnh ton v m ra nhiu ng dng.

Cc m hnh tnh ton mm thng da vo kinh nghim con ngi, s dng dung sai cho php ca s khng chnh xc, tnh bt nh, gn ng, xp x tm li gii hiu qu ch n gin, d hiu, d thc hin, chi ph thp.

Cc tng c bn ca tnh ton mm u tin xut hin theo cc bi bo ca Zadeh v l thuyt tp m vo 1965, sau l bi bo nm 1973 v phn tch h thng phc tp v qu trnh ra quyt nh, tip theo l bi bo nm 1981 v l thuyt kh nng v phn tch d kin mm. V sau, mng thn kinh v gii thut di truyn gp phn nng cao hiu qu ca tnh ton mm.Cc ng dng thnh cng ca tnh ton mm cho thy tnh ton mm ngy cng pht trin mnh v ng vai tr c bit trong cc lnh vc khc nhau ca Khoa hc v K thut.

Tnh ton mm biu th mt s chuyn dch, bin ha quan trng trong cc hng tnh ton. S chuyn dch ny phn nh s kin tr tu con ngi, khng nh my tnh, c kh nng ng k trong vic lu tr v x l thng tin khng chnh xc v bt nh, v y mi l nhng thng tin thc t v thng gp.

PHN 2: GII THIU V ANN

(ARTIFICIAL NEURAL NETWORK)I. LCH S PHT TRIN CA MNG NRON

Cc nghin cu v b no con ngi c tin hnh t hng nghn nm nay. Cng vi s pht trin ca khoa hc k thut c bit l nhng tin b trong ngnh in t hin i, vic con ngi bt u nghin cu cc nron nhn to l hon ton t nhin. S kin u tin nh du s ra i ca mng nron nhn to din ra vo nm 1943 khi nh thn kinh hc Warren McCulloch v nh ton hc Walter Pitts vit bi bo m t cch thc cc nron hot ng. H cng tin hnh xy dng mt mng nron n gin bng cc mch in. Cc nron ca h c xem nh l cc thit b nh phn vi ngng c nh. Kt qu ca cc m hnh ny l cc hm logic n gin chng hn nh a OR b hay a AND b.

Tip bc cc nghin cu ny, nm 1949 Donald Hebb cho xut bn cun sch Organization of Behavior. Cun sch ch ra rng cc nron nhn to s tr nn hiu qu hn sau mi ln chng c s dng.

Nhng tin b ca my tnh u nhng nm 1950 gip cho vic m hnh ha cc nguyn l ca nhng l thuyt lin quan ti cch thc con ngi suy ngh tr thnh hin thc. Nathanial Rochester sau nhiu nm lm vic ti cc phng th nghim nghin cu ca IBM c nhng n lc u tin m phng mt mng nron. Trong thi k ny tnh ton truyn thng t c nhng thnh cng rc r trong khi nhng nghin cu v nron cn giai on s khai. Mc d vy nhng ngi ng h trit l thinking machines (cc my bit suy ngh) vn tip tc bo v cho lp trng ca mnh.

Nm 1956 d n Dartmouth nghin cu v tr tu nhn to (Artificial Intelligence) m ra thi k pht trin mi c trong lnh vc tr tu nhn to ln mng nron. Tc ng tch cc ca n l thc y hn na s quan tm ca cc nh khoa hc v tr tu nhn to v qu trnh x l mc n gin ca mng nron trong b no con ngi.

Nhng nm tip theo ca d n Dartmouth, John von Neumann xut vic m phng cc nron n gin bng cch s dng rle in p hoc n chn khng. Nh sinh hc chuyn nghin cu v nron Frank Rosenblatt cng bt u nghin cu v Perceptron. Sau thi gian nghin cu ny Perceptron c ci t trong phn cng my tnh v c xem nh l mng nron lu i nht cn c s dng n ngy nay. Perceptron mt tng rt hu ch trong vic phn loi mt tp cc u vo c gi tr lin tc vo mt trong hai lp. Perceptron tnh tng c trng s cc u vo, ri tr tng ny cho mt ngng v cho ra mt trong hai gi tr mong mun c th. Tuy nhin Perceptron cn rt nhiu hn ch, nhng hn ch ny c ch ra trong cun sch v Perceptron ca Marvin Minsky v Seymour Papert vit nm 1969.

Nm 1959, Bernard Widrow v Marcian Hoff thuc trng i hc Stanford xy dng m hnh ADALINE (ADAptive LINear Elements) v MADALINE. (Multiple ADAptive LINear Elements). Cc m hnh ny s dng quy tc hc Least-Mean-Squares (LMS: Ti thiu bnh phng trung bnh). MADALINE l mng nron u tin c p dng gii quyt mt bi ton thc t. N l mt b lc thch ng c kh nng loi b tn hiu di li trn ng dy in thoi. Ngy nay mng nron ny vn c s dng trong cc ng dng thng mi.

Nm 1974 Paul Werbos pht trin v ng dng phng php hc lan truyn ngc (back-propagation). Tuy nhin phi mt mt vi nm th phng php ny mi tr nn ph bin. Cc mng lan truyn ngc c bit n nhiu nht v c p dng rng di nht cho n ngy nay.

Tht khng may, nhng thnh cng ban u ny khin cho con ngi ngh qu ln v kh nng ca cc mng nron. Chnh s cng iu qu mc c nhng tc ng khng tt n s pht trin ca khoa hc v k thut thi by gi khi ngi ta lo s rng n lc my mc c th lm mi vic ca con ngi. Nhng lo lng ny khin ngi ta bt u phn i cc nghin cu v mng nron. Thi k tm lng ny ko di n nm 1981.

Nm 1982 trong bi bo gi ti vin khoa hc quc gia, John Hopfield bng s phn tch ton hc r rng, mch lc, ng ch ra cch thc cc mng nron lm vic v nhng cng vic chng c th thc hin c. Cng hin ca Hopfield khng ch gi tr ca nhng nghin cu khoa hc m cn s thc y tr li cc nghin cu v mng nron.

Cng trong thi gian ny, mt hi ngh vi s tham gia ca Hoa K v Nht Bn bn v vic hp tc/cnh tranh trong lnh vc mng nron c t chc ti Kyoto, Nht Bn. Sau hi ngh, Nht Bn cng b nhng n lc ca h trong vic to ra my tnh th h th 5. Tip nhn iu , cc tp ch nh k ca Hoa K by t s lo lng rng nc nh c th b tt hu trong lnh vc ny. V th, ngay sau , Hoa K nhanh chng huy ng qu ti tr cho cc nghin cu v ng dng mng n ron.

Nm 1985, vin vt l Hoa K bt u t chc cc cuc hp hng nm v mng nron ng dng trong tin hc (Neural Networks for Computing).

Nm 1987, hi tho quc t u tin v mng nron ca Vin cc k s in v in t IEEE (Institute of Electrical and Electronic Engineer) thu ht hn 1800 ngi tham gia.

Ngy nay, khng ch dng li mc nghin cu l thuyt, cc nghin cu ng dng mng nron gii quyt cc bi ton thc t c din ra khp mi ni. Cc ng dng mng nron ra i ngy cng nhiu v ngy cng hon thin hn. in hnh l cc ng dng: x l ngn ng (Language Processing), nhn dng k t (Character Recognition), nhn dng ting ni (Voice Recognition), nhn dng mu (Pattern Recognition), x l tn hiu (Signal Processing), Lc d liu (Data Filtering),..II. MNG NRON SINH HC Khng c mt nh ngha tng qut v mng n ron, song cc chuyn gia trong lnh vc ny u c chung mt quan im: Mng n ron bao gm mt tp hp cc phn t x l n gin m chng ta gi l cc n ron, c lin kt vi nhau v hot ng song song vi nhau. Tnh nng ca mng ph thuc vo cu trc ca mng, trng s lin kt gia cc n ron v qu trnh x l bn trong cc n ron. Mng n ron c kh nng hc s liu v tng qut ha t cc s liu hc.

Cc mng n ron c xy dng m phng chc nng ca mt mng n ron sinh hc ni chung hay mng n ron sinh hc ca con ngi ni ring.

* M hnh v qu trnh x l trong n ron sinh hc:H thn kinh ca con ngi c khong 1010 n ron. Mi n ron bao gm thn n ron (cell body), bn trong c nhn (soma), h thng dy thn kinh vo hnh cy (dendrites) v mt u dy thn kinh ra (axon).Cc u dy thn kinh vo nhn tn hiu t cc n ron khc, nhn n ron s sinh ra tn hiu u ra ca n ron v truyn ti cc n ron khc c ni vi u ra ca chng.

im ni gia cc dy thn kinh vo v ra gi l cc khp. Tn hiu truyn trong cc dy thn kinh l tn hiu in, dng in l dng chuyn ng ca cc ion.

Ti khp thn kinh, cc tn hiu in u ra s kch thch lm gii phng cc cht ha hc, c gi l cht truyn n ron. Cc cht truyn n ron s khuch tn qua khe h khp v xuyn qua mng khp. Cc cht ha hc ny li kch thch to ra cc tn hiu in u vo ca cc n ron khc.

Khp thn kinh hot ng mt chiu, tc l ch c cc tn hiu in u ra mi to thnh c cc tn hiu in u vo ca cc n ron khc m khng c chiu ngc li. Hn na ti cc khp cn c t l bin i thng tin gia u vo v u ra, gi l khuch i lp.

Hnh 1: M hnh n ron sinh hc

Nh vy nron sinh hc hot ng theo cch thc sau: nhn tn hiu u vo, x l cc tn hiu ny v cho ra mt tn hiu output. Tn hiu output ny sau c truyn i lm tn hiu u vo cho cc nron khc.

Tt c cc c im trn u c vn dng mt cch trit trong vic xy dng mt mng n ron nhn to nhm to ra mt mng n ron ging vi mng n ron sinh hc nht.

III. MNG NRON NHN TO

1. Khi nim:Mng nron nhn to, Artificial Neural Network (ANN) gi tt l mng nron l mt m hnh x l thng tin phng theo cch thc x l thng tin ca cc h nron sinh hc. N c to ln t mt s lng ln cc phn t (gi l phn t x l hay nron) kt ni vi nhau thng qua cc lin kt (gi l trng s lin kt) lm vic nh mt th thng nht gii quyt mt vn c th no .

Mt mng nron nhn to c cu hnh cho mt ng dng c th (nhn dng mu, phn loi d liu, ...) thng qua mt qu trnh hc t tp cc mu hun luyn. V bn cht hc chnh l qu trnh hiu chnh trng s lin kt gia cc nron.

Mt nron l mt n v x l thng tin v l thnh phn c bn ca mt mng nron. Cu trc ca mt nron c m t trn hnh di.

Hnh 2: M hnh nron nhn toCc thnh phn c bn ca mt nron nhn to bao gm:

- Tp cc u vo: L cc tn hiu vo (input signals) ca nron, cc tn hiu ny thng c a vo di dng mt vector n chiu.

- Tp cc lin kt: Mi lin kt c th hin bi mt trng s (gi l trng s lin kt Synaptic weight). Trng s lin kt gia tn hiu vo th j vi nron i thng c k hiu l wij. Thng thng, cc trng s ny c khi to mt cch ngu nhin thi im khi to mng v c cp nht lin tc trong qu trnh hc mng. Trng s kt ni: W = [w1, w2, , wn]T - B tng (Summing function) (u): Thng dng tnh tng ca tch cc u vo vi trng s lin kt ca n. - Ngng (cn gi l lch - bias): K hiu: Ngng ny thng c a vo nh mt thnh phn ca hm truyn.

- Hm truyn (Transfer function) : Hm ny c dng gii hn phm vi u ra ca mi nron. N nhn u vo l kt qu ca hm tng v ngng cho. Thng thng, phm vi u ra ca mi nron c gii hn trong on [0,1] hoc [-1, 1]. Cc hm truyn rt a dng, c th l cc hm tuyn tnh hoc phi tuyn. Vic la chn hm truyn no l tu thuc vo tng bi ton v kinh nghim ca ngi thit k mng.

- u ra (Output): L tn hiu u ra ca mt nron, vi mi nron s c ti a l mt u ra.

V bn cht mt mng nron c chc nng nh l mt hm nh x F: X Y, trong X l khng gian trng thi u vo (input state space) v Y l khng gian trng thi u ra (output state space) ca mng. Cc mng ch n gin l lm nhim v nh x cc vector u vo x X sang cc vector u ra y Y thng qua b lc (filter) cc trng s. Tc l y = F(x) = f(W, x), trong W l ma trn trng s lin kt. Hot ng ca mng thng l cc tnh ton s thc trn cc ma trn.

i vi mi n ron chng ta tnh mt gi tr Net, l hm ca cc tn hiu u vo xi v trng s wi:

Net = w1x1 + w2x2 + + wnxn

Tn hiu ra ti mi n ron l mt hm s ca Net:

Y =f(Net)Hm truyn f thng s dng trong cc m hnh mng nron nh sau:Hm truyn thnh ngha

Symmetrical Hard Limit (hardlims)

Linear (purelin) y = f (u) = . u l h s gc ca hm

Saturating Linear (satlin)

Log-Sigmoid (logsig) y=f(u)

Nh vy tng t nh nron sinh hc, nron nhn to cng nhn cc tn hiu u vo, x l (nhn cc tn hiu ny vi trng s lin kt, tnh tng cc tch thu c ri gi kt qu ti hm truyn) v cho mt tn hiu u ra (l kt qu ca hm truyn).

2. Cc kin trc mng:Mc d mi nron n l c th thc hin nhng chc nng x l thng tin nht nh, sc mnh ca tnh ton nron ch yu c c nh s kt hp cc nron trong mt kin trc thng nht. Mt mng nron l mt m hnh tnh ton c xc nh qua cc tham s: kiu nron (nh l cc nt nu ta coi c mng nron l mt th), kin trc kt ni (s t chc kt ni gia cc nron) v thut ton hc (thut ton dng hc cho mng).

Cch thc kt ni cc nron trong mng xc nh kin trc (topology) ca mng. Cc nron trong mng c th kt ni y (fully connected) tc l mi nron u c kt ni vi tt c cc nron khc, hoc kt ni cc b (partially connected) chng hn ch kt ni gia cc nron trong cc tng khc nhau. Ngi ta chia ra hai loi kin trc mng chnh:

- T kt hp (autoassociative): l mng c cc nron u vo cng l cc nron u ra. Mng Hopfield l mt kiu mng t kt hp.

Hnh 3: Mng t kt hp- Kt hp khc kiu (heteroassociative): l mng c tp nron u vo v u ra ring bit. Perceptron, cc mng Perceptron nhiu tng (MLP: Multi Layer Perceptron), mng Kohonen, thuc loi ny.

Hnh 4: Mng kt hp khc kiuNgoi ra ty thuc vo mng c cc kt ni ngc (feedback connections) t cc nron u ra ti cc nron u vo hay khng, ngi ta chia ra lm 2 loi kin trc mng.

- Kin trc truyn thng (feedforward architechture): l kiu kin trc mng khng c cc kt ni ngc tr li t cc nron u ra v cc nron u vo; mng khng lu li cc gi tr output trc v cc trng thi kch hot ca nron. Cc mng nron truyn thng cho php tn hiu di chuyn theo mt ng duy nht; t u vo ti u ra, u ra ca mt tng bt k s khng nh hng ti tng . Cc mng kiu Perceptron l mng truyn thng.

Hnh 5: Mng truyn thng- Kin trc phn hi (Feedback architecture): l kiu kin trc mng c cc kt ni t nron u ra ti nron u vo. Mng lu li cc trng thi trc , v trng thi tip theo khng ch ph thuc vo cc tn hiu u vo m cn ph thuc vo cc trng thi trc ca mng. Mng Hopfield thuc loi ny.

Hnh 6: Mng phn hi3. Perceptron

Perceptron l mng nron n gin nht, n ch gm mt nron, nhn u vo l vector c cc thnh phn l cc s thc v u ra l mt trong hai gi tr +1 hoc -1.

Hnh 7: Perceptronu ra ca mng c xc nh nh sau: mng ly tng c trng s cc thnh phn ca vector u vo, kt qu ny cng ngng b c a vo hm truyn (Perceptron dng hm Hard-limit lm hm truyn) v kt qu ca hm truyn s l u ra ca mng. 3.1 Lut hc Perceptron:

Cho tp mu hc vi vecto vo X v 1 u ra l tng tng ng d(k). i vi nhim v phn lp th d(k) thng l 1 hoc -1. Lut hc:

- Bt u vi cc trng s ngu nhin.

- Ly mt vecto vo x t tp hun luyn.

- Nu u ra tnh ton yk # d(k) (tc l perceptron p ng sai) th iu chnh trng mi trng s Wi:

(wi = c(dk yk)xi.

3.2 Gii thut hc Perceptron:

- Vi mi tp cc mu hc D={(x,d)}x l vecto u vo

d l gi tr u ra mong mun (-1 hoc 1)

- Vi mt v d hc x c perceptron phn lp chnh xc, th vecto trng s w khng thay i.

- Nu d=1 nhng perceptron li sinh ra -1 (Out=-1) th w cn c thay i sao cho gi tr New(w,x) tng ln.

- Nu d=-1 nhng perceptron li sinh ra 1 (Out=1) th w cn c thay i sao cho gi tr New(w,x) gim i.

- Perceptron s iu chnh trng s trn thnh phn th I ca vecto u vo mt lng (wi = c(d f(wixi))xi.

f(wixi)) chnh l gi tr u ra ca perceptron, n c gi tr +1 hoc -1.- Hiu gia d v f(wixi)) l 0, 2 hoc -2. V vy vi mi thnh phn ca vecto u vo:

+ Nu gi tr u ra mong mun v gi tr u ra tnh ton t bng nhau th khng lm g c.

+ Nu gi tr u ra thc l -1 v 1 l gi tr mong mun th tng trng s ca ng th i ln (2cxi).

+ Nu gi tr u ra thc l 1 v -1 l gi tr mong mun th gim trng s ca ng th i (-2cxi).

- c c gi l hng s th hin tc hc v nu c ln th cc gi tr iu chnh (wi s ln, nh vy y nhanh qu trnh wi hi t v gi tr ng ca n.Perceptron cho php phn loi chnh xc trong trng hp d liu c th phn chia tuyn tnh (cc mu nm trn hai mt i din ca mt siu phng). N cng phn loi ng u ra cc hm AND, OR v cc hm c dng ng khi n trong m u vo ca n ng (n m). N khng th phn loi c u ra ca hm XOR.

4. Mng nhiu tng truyn thng (MLP)

M hnh mng nron c s dng rng ri nht l m hnh mng nhiu tng truyn thng (MLP: Multi Layer Perceptron). Mt mng MLP tng qut l mng c n (n2) tng (thng thng tng u vo khng c tnh n): trong gm mt tng u ra (tng th n) v (n-1) tng n.

Hnh 8: Mng MLP tng qutKin trc ca mt mng MLP tng qut c th m t nh sau:

- u vo l cc vector (x1, x2, ..., xn) trong khng gian n chiu, u ra l cc vector (y1, y2, ..., ym) trong khng gian m chiu. i vi cc bi ton phn loi, n chnh l kch thc ca mu u vo, m chnh l s lp cn phn loi. Mi nron thuc tng sau lin kt vi tt c cc nron thuc tng lin trc n.

- u ra ca nron tng trc l u vo ca nron thuc tng lin sau n.

Hot ng ca mng MLP nh sau: ti tng u vo cc nron nhn tn hiu vo x l (tnh tng trng s, gi ti hm truyn) ri cho ra kt qu (l kt qu ca hm truyn); kt qu ny s c truyn ti cc nron thuc tng n th nht; cc nron ti y tip nhn nh l tn hiu u vo, x l v gi kt qu n tng n th 2; qu trnh tip tc cho n khi cc nron thuc tng ra cho kt qu.

* Mt s kt qu c chng minh:

- Bt k mt hm Boolean no cng c th biu din c bi mt mng MLP 2 tng trong cc nron s dng hm truyn sigmoid.

- Tt c cc hm lin tc u c th xp x bi mt mng MLP 2 tng s dng hm truyn sigmoid cho cc nron tng n v hm truyn tuyn tnh cho cc nron tng ra vi sai s nh ty .

- Mi hm bt k u c th xp x bi mt mng MLP 3 tng s dng hm truyn sigmoid cho cc nron tng n v hm truyn tuyn tnh cho cc nron tng ra. khc phc nhng kh khn i vi nhng bi ton c mu phn chia khng tuyn tnh, mng nron nhiu lp c s dng. C rt nhiu cng trnh nghin cu vmng MLP v cho thy nhiu u im ca mng ny.

Mng MLP l mt gii php hu hiu cho vic m hnh ho, c bit vi qu trnh phc tp hoc c ch cha r rng. N khng i hi phi bit trc dng hoc tham s. Mng MLP l c s cho thut ton lan truyn ngc v kh nng xp x lin tc.

5. Mng Hopfield:

Mng Hopfield l mt dng mng nron nhn to truyn ngc do John Hopfield a ra. Mng Hopfield ng vai tr nh cc h thng b nh c th nh a ch ni dung vi cc nt ngng dng nh phn.- Bo m s hi t v mt cc tiu cc b, nhng khng m bo s hi t v mt trong cc mu c lu tr.- Gm 1 lp n cc nron kt ni y .

- Cc nron lin kt vi mi nron khc nhng khng t kt ni.

6. Xy dng mng nron:

6.1 Chun b d liu

a. Kch thc mu Khng c nguyn tc no hng dn kch thc mu phi l bao nhiu i vi mt bi ton cho trc. Hai yu t quan trng nh hng n kch thc mu:

- Dng hm ch: khi hm ch cng phc tp th kch thc mu cn tng.

- Nhiu: khi d liu b nhiu (thng tin sai hoc thiu thng tin) kch thc mu cn tng.

i vi mng truyn thng (feedforward), cho hm ch c phc tp nht nh, km mt lng nhiu nht nh th chnh xc ca m hnh lun c mt gii hn nht nh. C th cn tp mu v hn t n gii hn chnh xc. Ni cch khc chnh xc ca m hnh l hm theo kch thc tp mu. Khi kch thc mu tng, chnh xc s c ci thin - lc u nhanh, nhng chm dn khi tin n gii hn.

Dng tng qut ca mi lin h gia sai s v kch thc mu nh sau:

Hnh 12: Mi lin h gia sai s v kch thc muTrong thc hnh thng gp phi 2 vn sau :

- i vi hu ht bi ton thc t, mu b rng buc cht ch vi d liu c sn. Ta thng khng c c s lng mu mong mun.

- Kch thc mu cng c th b gii hn bi b nh hoc kh nng lu tr ca my tnh. Nu tt c cc d liu ng thi c gi trong b nh sut thi gian luyn, kch thc b nh my tnh s b chim dng nghim trng.

Nu lu tr trn a s cho php dng mu ln hn nhng thao tc c a t th h ny sang th h khc khin cho tin trnh chm i rt nhiu.

Ch : vic tng kch thc mu khng lm tng thi gian luyn. Nhng tp mu ln hn s yu cu t th h luyn hn. Nu ta tng gp i kch thc ca mu, mi th h luyn s tn thi gian khong gp i, nhng s th h cn luyn s gim i mt na. iu ny c ngha l kch thc mu (cng c ngha l chnh xc ca m hnh) khng b gii hn bi thi gian luyn.

Lut c bn l: S dng mu ln nht c th sao cho kh nng lu tr trong b nh trong (nu lu tr ng thi) hoc trn a t (nu thi gian c t a).

b. Mu con Trong xy dng m hnh cn chia tp mu thnh 2 tp con: mt xy dng m hnh gi l tp hun luyn (training set), v mt kim nghim m hnh gi l tp kim tra (test set). Thng thng dng 2/3 mu cho hun luyn v 1/3 cho kim tra. iu ny l trnh tnh trng qu khp (overfitting).

c. S phn tng mu Nu ta t chc mu sao cho mi mu trong qun th u c c hi nh nhau th tp mu c gi l tp mu i din. Tuy nhin khi ta xy dng mt mng xc nh xem mt mu thuc mt lp hay thuc mt loi no th iu ta mong mun l cc lp c cng nh hng ln mng, t c iu ny ta c th s dng mu phn tng. Xt v d sau[1]:

Gi s ta xy dng m hnh nhn dng ch ci vit tay ting Anh, v ngun d liu ca ta c 100.000 k t m mi k t c km theo mt m cho bit n l ch ci no. Ch ci xut hin thng xuyn nht l e, n xut hin 11.668 ln chim khong 12%; ch ci xut hin t nht l ch z, ch c 50 ln chim 0,05%.

Trc ht do gii hn ca b nh my tnh, gi s b nh ch c th x l c 1300 mu. Ta to hai dng tp mu: tp mu i din v tp mu phn tng. Vi tp mu i din, ch e s xut hin 152 ln (11,67% ca 1300) trong khi ch z ch xut hin mt ln (0,05% ca 1300). Ngc li ta c th to tp mu phn tng mi ch c 50 mu. Ta thy rng nu ch c th dng 1300 mu th tp mu phn tng s to ra m hnh tt hn. Vic tng s mu ca z t 1 ln 50 s ci thin rt nhiu chnh xc ca z, trong khi nu gim s mu ca e t 152 xung 50 s ch gim cht t chnh xc ca e.

By gi gi s ta dng my tnh khc c b nh x l mt lng mu gp 10 ln, nh vy s mu s tng ln 13000. R rng vic tng kch thc mu s gip cho m hnh chnh xc hn. Tuy nhin ta khng th dng tp mu phn tng nh trn na v lc ny ta s cn ti 500 mu cho ch z trong khi ta ch c 50 mu trong ngun d liu. gii quyt iu ny ta to tp mu nh sau: tp mu gm tt c cc ch him vi s ln xut hin ca n v km thm thng tin v ch c nhiu mu nht. Chng hn ta to tp mu c 50 mu ca ch z ( l tt c) v 700 mu ca ch e (ch m ta c nhiu mu nht).

Nh vy trong tp mu ca ta, ch e c nhiu hn ch z 14 ln. Nu ta mun cc ch z cng c nhiu nh hng nh cc ch e, khi hc ch z ta cho chng trng s ln hn 14 ln. lm c iu ny ta c th can thip cht t vo qu trnh lan truyn ngc trn mng. Khi mu hc l ch z, ta thm vo 14 ln o hm, nhng khi mu l ch e ta ch thm vo 1 ln o hm. cui th h, khi cp nht cc trng s, mi ch z s c nh hng hn mi ch e l 14 ln, v tt c cc ch z gp li s c nh hng bng tt c cc ch e.

d. Chn bin Khi to mu cn chn cc bin s dng trong m hnh. C 2 vn cn quan tm:

- Cn tm hiu cch bin i thng tin sao cho c li cho mng hn: thng tin trc khi a vo mng cn c bin i dng thch hp nht, mng t c hiu xut cao nht. Xt v d v bi ton d on mt ngi c mc bnh ung th hay khng. Khi ta c trng thng tin v ngi ny l ngy thng nm sinh. Mng s t c hiu qu cao hn khi ta bin i trng thng tin ny sang thnh tui. Thm ch ta c th quy tui v mt trong cc gi tr: 1 = tr em (di 18), 2 = thanh nin (t 18 n di 30), 3 = trung nin (t 30 n di 60) v 4 = gi (t 60 tr ln).

- Chn trong s cc bin c bin i, bin no s c a vo m hnh: khng phi bt k thng tin no v mu cng c li cho mng. Trong v d d on ngi c b ung th hay khng trn, nhng thuc tnh nh ngh nghip, ni sinh sng, tiu s gia nh, l nhng thng tin c ch. Tuy nhin nhng thng tin nh thu nhp, s con ci, l nhng thng tin khng cn thit.

6.2 Xc nh cc tham s cho mng:

a. Chn hm truyn Khng phi bt k hm truyn no cng cho kt qu nh mong mun. tr li cho cu hi hm truyn nh th no c coi l tt ? l iu khng h n gin. C mt s quy tc khi chn hm truyn nh sau:

- Khng dng hm truyn tuyn tnh tng n. V nu dng hm truyn tuyn tnh tng n th s lm mt vai tr ca tng n : Xt tng n th i:

Tng trng s ni = wiai-1 + bi

ai = f(ni) = wf ni +bf (hm truyn tuyn tnh)

Khi : tng trng s ti tng th (i + 1)

ni+1 = wi+1ai + bi+1

= wi+1[wf ni +bf] + bi+1

= wi+1 [wf(wiai-1 + bi) + bf] + bi+1

= Wai-1 + b

Nh vy ni+1 = Wai-1 + b, v tng i khng cn gi tr na.

- Chn cc hm truyn sao cho kin trc mng nron l i xng (tc l vi u vo ngu nhin th u ra c phn b i xng). Nu mt mng nron khng i xng th gi tr u ra s lch sang mt bn, khng phn tn ln ton b min gi tr ca output. iu ny c th lm cho mng ri vo trng thi bo ha, khng thot ra c.

Trong thc t ngi ta thng s dng cc hm truyn dng S. Mt hm s(u) c gi l hm truyn dng S nu n tha mn 3 tnh cht sau:

s(u) l hm b chn: tc l tn ti cc hng s C1 C2 sao cho: C1 s(u) C2 vi mi u.

s(u) l hm n iu tng: gi tr ca s(u) lun tng khi u tng. Do tnh cht th nht, s(u) b chn, nn s(u) s tim cn ti gi tr cn trn khi u dn ti dng v cng, v tim cn gi tr cn di khi u dn ti m v cng.

s(u) l hm kh vi: tc l s(u) lin tc v c o hm trn ton trc s.

Mt hm truyn dng - S in hnh v c p dng rng ri l hm Sigmoid.

b. Xc nh s nron tng n Cu hi chn s lng noron trong tng n ca mt mng MLP th no l kh, n ph thuc vo bi ton c th v vo kinh nghim ca nh thit k mng. Nu tp d liu hun luyn c chia thnh cc nhm vi cc c tnh tng t nhau th s lng cc nhm ny c th c s dng chn s lng nron n. Trong trng hp d liu hun luyn nm ri rc v khng cha cc c tnh chung, s lng kt ni c th gn bng vi s lng cc mu hun luyn mng c th hi t. C nhiu ngh cho vic chn s lng nron tng n h trong mt mng MLP. Chng hn h phi tha mn h>(p-1)/(n+2), trong p l s lng mu hun luyn v n l s lng u vo ca mng. Cng nhiu nt n trong mng, th cng nhiu c tnh ca d liu hun luyn s c mng nm bt, nhng thi gian hc s cng tng.

Mt kinh nghim khc cho vic chn s lng nt n l s lng nt n bng vi s ti u cc cm m (fuzzy clusters). Pht biu ny c chng minh bng thc nghim. Vic chn s tng n cng l mt nhim v kh. Rt nhiu bi ton i hi nhiu hn mt tng n c th gii quyt tt.

tm ra m hnh mng nron tt nht, Ishikawa and Moriyama (1995) s dng hc cu trc c qun (structural leanrning with forgetting), tc l trong thi gian hc ct b i cc lin kt c trng s nh. Sau khi hun luyn, ch cc noron c ng gp vo gii quyt bi ton mi c gi li, chng s to nn b xng cho m hnh mng nron.

c. Khi to trng s:Trng s thng c khi to bng phng php th sai, n mang tnh cht kinh nghim v ph thuc vo tng bi ton. Vic nh ngh th no l mt b trng stt cng khng h n gin. Mt s quy tc khi khi to trng s:- Khi to trng s sao cho mng nron thu c l cn bng (vi u vo ngu nhin th sai s lan truyn ngc cho cc ma trn trng s l xp x bng nhau):

|W1/W1| = |W2/W2| = |W3/W3|

Nu mng nron khng cn bng th qu trnh thay i trng s mt s ma trn l rt nhanh trong khi mt s ma trn khc li rt chm, thm ch khng ng k. Do cc ma trn ny t ti gi tr ti u s mt rt nhiu thi gian.

- To trng s sao cho gi tr kt xut ca cc nt c gi tr trung gian. (0.5 nu hm truyn l hm Sigmoid). R rng nu ta khng bit g v gi tr kt xut th gi tr gia l hp l. iu ny cng gip ta trnh c cc gi tr thi qu.

Th tc khi to trng s thng c p dng:

B1: Khi to cc trng s nt n v cc trng s ca cc cung lin kt trc tip gia nt nhp v nt xut, nu c gi tr ngu nhin, nh, phn b u quanh 0.

B2: Khi to mt na s trng s ca nt xut gi tr 1, v na kia gi tr -1.

6.3 Vn lng qun (catastrophic forgetting)

Catastrophic forgetting l vn mt mng qun nhng g n hc c trong cc mu trc khi ang hc cc mu mi. Nguyn nhn l do s thay i cc trng s theo cc mu mi, nu nh cc mu c trong mt thi gian khng c a vo hun luyn. trnh iu ny, ta thng thc hin vic hun luyn lun phin gia mu c v mu mi.

Hnh 13: Hun luyn lun phin trn hai tp muXt v d mng c hun luyn lun phin vi hai tp mu A v B (hnh 13). Ti mi chu k mng s hc tp mu A sau hc tp mu B. Khi bc vo chu k th hai, li lc bt u hc tp mu A cao hn l chu k th nht khi va hc xong tp A. iu ny l do gia hai ln hc tp mu A mng hc tp mu B. Tuy nhin nu xt trn c chu k th li hun luyn s gim xung. Tc l li lc bc vo chu k th ba s nh hn lc bc vo chu k th hai.

C nhiu phng php hun luyn d liu mi. Chng hn sau khi mt s mu mi c hc, mt vi mu c c chn ngu nhin trong s cc mu trc a vo hc. Vn s kh khn hn khi cc mu c khng cn na. Khi cc mu gi (pseudoexamples) c th c s dng lu gi cc trng s cng gn cc trng s trc cng tt.

6.4 Vn qu khp

a. Khi nim qu khp Vn qu khp xy ra khi mng c luyn qu khp (qu st) vi d liu hun luyn (k c nhiu), nn n s tr li chnh xc nhng g c hc, cn nhng g khng c hc th n khng quan tm. Nh vy mng s khng c c kh nng tng qut ha.

V mt ton hc, mt gi thuyt (m hnh) h c gi l qu khp nu tn ti gi thuyt h' sao cho:

1. Error train (h) < Error train (h')

2. Error test (h) > Error test (h')

b. Gii quyt qu khp Vn qu khp xy ra v mng c nng lc qu ln. C 3 cch hn ch bt nng lc ca mng:

Hn ch s nt n

Ngn khng cho mng s dng cc trng s ln

Gii hn s bc luyn

Khi mng c luyn, n chuyn t cc hm nh x tng i n gin n cc hm nh x tng i phc tp. N s t c mt cu hnh tng qut ha tt nht ti mt im no . Sau im mng s hc m hnh ha nhiu, nhng g mng hc c s tr thnh qu khp. Nu ta pht hin ra thi im mng t n trng thi tt nht ny, ta c th ngng tin trnh luyn trc khi hin tng qu khp xy ra.

Ta bit rng, ch c th nh gi mc tng qut ha ca mng bng cch kim tra mng trn cc mu n khng c hc. Ta thc hin nh sau: chia mu thnh tp mu hun luyn v tp mu kim tra. Luyn mng vi tp mu hun luyn nhng nh k dng li v nh gi sai s trn tp mu kim tra. Khi sai s trn tp mu kim tra tng ln th qu khp bt u v ta dng tin trnh luyn.

Ch rng, nu sai s kim tra khng h tng ln, tc l mng khng c s nt n qu khp. Khi mng s khng c s nt cn thit thc hin tt nht. Do vy nu hin tng qu khp khng h xy ra th ta cn bt u li nhng s dng nhiu nt n hn.7. Hun luyn mng nron:* Cc k thut hc:

Khi nim: Hc l qu trnh thay i hnh vi ca cc vt theo mt cch no lm cho chng c th thc hin tt hn trong tng lai.

Mt mng nron c huyn luyn sao cho vi mt tp cc vector u vo X, mng c kh nng to ra tp cc vector u ra mong mun Y ca n. Tp X c s dng cho hun luyn mng c gi l tp hun luyn (training set). Cc phn t x thuc X c gi l cc mu hun luyn (training example). Qu trnh hun luyn bn cht l s thay i cc trng s lin kt ca mng. Trong qu trnh ny, cc trng s ca mng s hi t dn ti cc gi tr sao cho vi mi vector u vo x t tp hun luyn, mng s cho ra vector u ra y nh mong mun.

C ba phng php hc ph bin l hc c gim st (supervised learning), hc khng gim st (unsupervised learning) v hc tng cng (Reinforcement learning):

7.1 Hc c gim st:L qu trnh hc c s tham gia gim st ca mt thy gio. Cng ging nh vic ta dy mt em nh cc ch ci. Ta a ra mt ch a v bo vi em rng y l ch a. Vic ny c thc hin trn tt c cc mu ch ci. Sau khi kim tra ta s a ra mt ch ci bt k (c th vit hi khc i) v hi em y l ch g?

Vi hc c gim st, tp mu hun luyn c cho di dng D = {(x,t) | (x,t) [IRn x Rm]}, trong : x = (x1, x2, ..., xn) l vector c trng n chiu ca mu hun luyn v t = (t1, t2, ..., tm) l vector mc tiu m chiu tng ng, nhim v ca thut ton l phi thit lp c mt cch tnh ton trn mng nh th no sao cho vi mi vector c trng u vo th sai s gia gi tr u ra thc s ca mng v gi tr mc tiu tng ng l nh nht. Chng hn mng c th hc xp x mt hm t = f(x) biu din mi quan h trn tp cc mu hun luyn (x, t).

Xs

Ytnh Ys

Hnh 9: S k thut hc c gim st

Nh vy vi hc c gim st, s lp cn phn loi c bit trc. Nhim v ca thut ton l phi xc nh c mt cch thc phn lp sao cho vi mi vector u vo s c phn loi chnh xc vo lp ca n.

7.2 Hc khng gim st: L vic hc khng cn c bt k mt s gim st no.

Xs

Out

Tn hiu ra

Hnh 10: S k thut hc khng gim st

Trong bi ton hc khng gim st, tp d liu hun luyn c cho di dng: D = {(x1, x2, ..., xn)}, vi (x1, x2, ..., xn) l vector c trng ca mu hun luyn. Nhim v ca thut ton l phi phn chia tp d liu D thnh cc nhm con, mi nhm cha cc vector u vo c c trng ging nhau.

Nh vy vi hc khng gim st, s lp phn loi cha c bit trc, v ty theo tiu chun nh gi tng t gia cc mu m ta c th c cc lp phn loi khc nhau.

7.3 Hc tng cng: i khi cn c gi l hc thng - pht (reward-penalty learning), l s t hp ca c hai m hnh trn. Phng php ny c th nh sau: vi vector u vo, quan st vector u ra do mng tnh c. Nu kt qu c xem l tt th mng s c thng theo ngha tng cc trng s kt ni ln; ngc li mng s b pht, cc trng s kt ni khng thch hp s c gim xung. Do hc tng cng l hc theo nh ph bnh (critic), ngc vi hc c gim st l hc theo thy gio (teacher).

Xs

Ytnh

Tn hiu

Y/N

Hnh 11: S k thut hc tng cngIV. NG DNG CA MNG NORONNgy nay, mng n ron ngy cng c ng dng nhiu trong thc t. c bit l cc bi ton nhn dng mu, x l, lc d liu, v iu khin. ng dng ca mng nron c chia thnh cc loi sau:

- X l ngn ng : X l ngn ng t nhin

- Nhn dng mu: Nhn dng nh, Nhn ging ni, Nhn dng ch vit

- X l tn hiu: iu khin t ng

- Lc v phn loi d liu: Chun on bnh, Tm kimPHN 3: MNG LAN TRUYN NGC SAI S

(BACK PROPAGATION OF ERROR BP)I. TNG QUAN V MNG LAN TRUYN NGC LI:

Th gii thc xy ra tnh hung l d liu b thiu hoc b nhiu. a ra c d on thch hp da trn nhng thng tin b thiu ny l rt kh (cha c mt l thuyt no c th gip ti to li d liu b mt). Mng BP c th a ra c mt s cu tr li thch hp.

Cu trc BP gm t nht 3 lp:

- Mt lp vo (Input Layer)

- t nht mt lp n gia (Hidden Layer)

- Mt lp ra (Output Layer)

Thng thng cc node u vo c kt ni y ti cc node lp n v cc node lp n c kt ni y vi cc node trong lp u ra.

u ra ca mng lan truyn ngc c xem nh mt b phn lp quyt nh.

Vi mng lan truyn ngc, qu trnh hc xy ra trong sut mt chu k hun luyn. Gm cc bc:

- Mi mu u vo trong tp mu hc c p dng cho cc node lp vo v sau c lan truyn tin.

- Mu sau khi c lan truyn n lp u ra c so snh vi mu ra (u ra l tng) tnh ton li u ra.

- Li ng vi mi mu u ra sau c lan truyn ngc t cc u ra ti cc u vo nhm iu chnh cc trng s mt cch thch hp trong mi lp ca mng.

- Sau khi mng lan truyn ngc c hun luyn phn loi chnh xc cho cc tp mu hc, n c th c kim tra trn mt tp mu cha qua hun luyn (kim tra nng lc d bo ca mng). Nu nng lc d bo tt, mng c th c dng d bo.

Hnh 12: M hnh mng nron nhiu lp lan truyn ngcII. GII THUT HC BP

1. Gii thut hun luyn mng:

Xt mng nron c 3 lp, lp input c 1 node, lp n c m node, lp output c n node. Xt gii thut lan truyn ngc vi mng nron nhn to ny:

* Kin trc c s ca gii thut:

Khi to cc trng s

Repeat

For each mu hc

Hc vi mu ny

End

Until li mc nh chp nhn c.

- Bc 1: a cc gi tr vo (input), ra (output) v dng ma trn gi tr- Bc 2: Gi nh s neural m ca lp hidden tha 1 p=[4.7 6.1 5.6 5.8 6.5;

3.2 2.8 3.0 2.7 3.2;

1.3 4.7 4.1 5.1 5.1;

0.2 1.2 1.3 1.9 2.0];

>> t=[0 1 1 0 0];

- To mng n-ron l net vi u vo l p v hm mc tiu t.

- To mng vi 2 n ron lp n v s dng hm truyn logsig cho c 2 lp:

>> net=newff(p,t,2,{'logsig' 'logsig'});Bc 2: Hun luyn mng:- Truyn mu A vo mng:>> p1=[4.7;3.2;1.3;0.2];>> t1=[0];a tp hun luyn vo mng bng lnh train():

>> net=train(net,p1,t1);

Bc 3: Tnh vecto u ra y ca mng:Sau khi hun luyn mng ta c th s dng chng bng cch gi hm sim().V d s dng hm sim() vi vect u vo p ta s thu c kt qu y l m phng ca t.>> y=sim(net,p1)y =

0.5000Bc 4: nh gi li:Hm nh gi li mc nh y l MSE (mean-square error).>> mse(t1-y)

ans =

0.2500

Bc 5: Hiu chnh trng s:

Sau khi hun luyn, mng nron cho u ra y cha ging vi vecto mc tiu t, mng tin hnh cp nht li trng s lin kt v ngng theo cng thc:wk+1 = wk kgk

vi: wk l vecto trng s v ngng hin thi

k l h s hc hin thi

gk l gi tr gradient hin thi.Bc 6: Lp i lp li cc bc t bc 2 n bc 5 cho n khi mng t trng thi hi t.- Tip tc truyn mu B vo mng:

>> p2=[6.1;2.8;4.7;1.2];>> t2=[1];

>> net=train(net,p2,t2);

>> y=sim(net,p2)

y =

1.0000

>> mse(t2-y)

ans =

1.7515e-011- Truyn mu C vo mng:

>> p3=[5.6;3.0;4.1;1.3];

>> t3=[1];>> net=train(net,p3,t3);

>> y=sim(net,p3)

y =

1.0000

>> mse(t3-y)

ans =

2.1198e-011Chng ta c th thay i cc gi tr khi to ca mng v hun luyn li mng c kt qu chnh xc nh mong i hn.C th thc hin o to mng hng lot vi tp mu p, t a vo:

>> net=train(net,p,t);

>> y=sim(net,p)

y =

0.9954 0.6744 0.9414 0.5000 0.5001

>> mse(t-y)

ans =

0.3201

>> net=init(net);

>> net=train(net,p,t);

>> y=sim(net,p)

y =

0.5000 0.5000 0.5000 0.5000 0.5000

>> mse(t-y)

ans =

0.2500..3. Ci t bi ton s dng cng c nntool trong Matlab:

- Trong ca s Command Window g lnh: nntool

Ca s Network/ Data Manager hin ra:

- Tp tp d liu u vo v tp mc tiu, chn New Data:

Tp d liu vo t Name l Input, Data Type l inputs. Nhp d liu di dng ma trn khung Value -> bm Create.

Tp mc tiu t Name l Target, Data Type l Targets. Nhp d liu di dng ma trn vo khung Value.

- To mng: Trong ca s Network/Data Manager chn New Network:

t Network Name l CayIris, Get from input chn Input, Number of layers l 2, number of neurons cho layer1 l 2, layer2 l 1 v Transfer Function cho c 2 layer l logsig ->Bm Create.

- Bm View xem kin trc mng va c to:

- o to mng: bm vo CayIris trong khung Network ca ca s Network/Data Manager -> chn Train -> Ca s Network CayIris hin ra.

- Thit lp cc trng s lin kt v ngng cho mng trong tab Weights bng Set Weight hoc s dng cc trng s do mng t ng khi to.

- Trong tab Train chn Training Data vi Inputs l Input, Targets l Target -> Bm Train Network.

- Sau khi o to mng, Output v Errors s c a ra trn ca s Network/Data Manager. xem c, bm chn vo CayIris_outputs hoc CayIris_errors -> chn View:

- Ta thy u ra output ca mng rt gn vi u ra mc tiu mong mun v li gim dn.

mng t trng thi hi t, ngha l gi tr outputs ging vi tp mc tiu nht, tin hnh o to li mng vi cc ma trn trng s v ngng c cp nht sau nhng ln o to trc .KT LUN ti ny trnh by cc vn v cng ngh tnh ton mm v mng nron, gm: tm hiu v tnh ton mm, khi nim ca mng n ron nhn to, lch s pht trin, cc m hnh mng v phng php xy dng cng nh hun luyn mng. Trong i su vo vic xy dng mt mng nron truyn thng MLP, v thut tun hun luyn Lan truyn ngc.

ti ny cng trnh by mt thc nghim cho cc l thuyt v mng MLP v thut ton Lan truyn ngc nu ra, ci t minh ha bi ton bng ngn ng Matlab.

Hng pht trin ca ti: Tm hiu v cc thut ton hun luyn mng n ron khc, c th a ra cc so snh, cng nh chn m hnh thch hp cho cc bi ton c th.

Pht trin chng trnh thc nghim thnh mt chng trnh c ngha thc t h nh phn loi mu, nhn dng ch vit tay, nhn dng nh, da trn nn tng mng xy dng.

Chng em rt mong nhn c s ng gp kin ca cc thy c gio trong khoa v cc bn sinh vin ti ca chng em c hon thin hn na trong tng lai.

Chng em xin chn thnh cm n!

TI LIU THAM KHO

[1]. Genevieve Orr, Nici Schraudolph and Fred Cummins http://www.willamette.edu/~gorr/classes/cs449/intro.html [2]. Christos Stergiou and Dimitrios Siganos. Neural Networks. http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html

[3]. Nikola K. Kasabov. Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering. Massachusetts Institute of Technology.[4]. H m, mng nron v ng dng - Bi Cng Cng Nguyn Don Phc.[5].http://www4.hcmut.edu.vn/~huynhqlinh/TinhocDC/WebLQNguyen/noron%20nhan%20tao/feedforward_1.html KIN NHN XT CA GING VIN HNG DNN1

N2

N3

N4

N1>N4

N1>N3

N3>N1

N3>N2

N3>N4

N4>N3

N4>N2

N4>N1

N2>N4

N2>N3

N1>N2

N2>N1

Mng nron N

Hiu chnh W

Sai s

Mng n ron N

Mng nron N

Hiu chnh W

Tn hiu tng cng

Bias

Hidden Layer

Input Layer

Output Layer

Lan truyn tng mu hc

Hiu chnh trng s

Ht mu hc

s vng

WHide &Wlast & RMS

Ht

Cha

Cha

3.2

1.3

0.2

T0 = 0

4.7

iNhm 11 LTKHMT2K6 1 Ging vin: Nguyn Phng Nga