Ch16 Markov Handout Pp 1-53

8/11/2019 Ch16 Markov Handout Pp 1-53

1/53


2/53

M

3.4& 5&6107.12

! ,-2'14*('

" #


3/53

R

8"#% 02 # 3%.9"#279 :'.9&22;

! ,.

40/ ;47 $& )9&G&/ 4* 4 840/2; )4894$3&S

!

J =029'&%&>74& 2%.9"#279 $'.9&229* *9;


4/53

V

! J 9.171?.?2@74&*-2'14*('


5/53

Y

,./01A %"'.?A" %"& $'.9&22

! 50 *.'1


6/53

\

3.B C"D =. C& 9#'&;

! L1&


7/53

]

EF#4$G& .H # 3%.9"#279 :'.9&22

! Z&4-1&8 90 ^43964E

" ,47 -1& '140'& 26 9- $&90A /87 -2;2882G 9* _`a 96

9- 9* /87 -2/47O $.- 2037 \`a 96 9- 9* 849090A -2/47

"

J**.;& -1&*&


8/53

_

EF#4$G& L9.171?&=M

! N40/2; )4894-&Xt-4>&* 20 -1& )43.&*+

!

L1& *-2'14*('


9/53

b

N1.%"&' EF#4$G&

! c4;$3&8d* N.90 :82$3&;+

" e2. *-48- 4 A4;& G9-1 fMS 50 &4'1 82.0/ 26 -1&

A4;&O 72. 14)& 4 02Gi


10/53

!`

8"#% 02 # ,#'-./ !"#01;

!

5&6107.1+J /9*'8&-&X(;& *-2'14*('


11/53

!!

O'#1207.1 :'.P#P0G07&2

! L1&


12/53

!M

EF#4$G&2 > 3%#7.1#'0%D

! 56 G& 4**.;& -14- !` 7&48* 682; 02GO -1&

4 $433 682; -1&

$2ES

5* k"#l 4 *-4(20487


13/53

!R

! L1& -840*9(20


14/53

!V

!

@28 &4'1 9 40/ %+

! @28 &4'1 i:

e2. 14)& -2 $& 01 2.4& 2%#%& 90 -1& 0&E- *-4A&Sn2-& -14- -19* '2.3/ 90'3./& *-4-& iH72. '2.3/*-47 G1&8& 72. 48&I

" C4'1 &0-87 90 !;.*- $& 0200&A4()&S

"

^&0'&O 433 &0-89&* 90 -1& -840*9(20


15/53

!Y

Q#4PG&'R2 K?01 :'.PG&4

! Z14- 48& -1& *-4-&*i

?2;


16/53

!\

Q'#$" .H %"& C%"&' $'.PG&4

the chance of it being dry tomorrow is 80% if it is dry today,but only 60% if it is raining today

Transition matrix:

The graph:


17/53

!]

Q#4PG&'R2 K?01 :'.PG&4

! ,47 < h `S\` H40/ -1&8&628& ! g < h `SV`I

!!!

!!!#

$$$

$$$&

=

!""""

#"$""%"$"""

"#"$""%"$""

""#"$""%"$"

""""!

%'

('

)'

!'

"'

%'(')'!'"'

!


18/53

!_

Q'#$" .H %"02 :'.9&22

States?

Transition probabilities?


19/53


20/53

M`

!>3%&$ O'#1207.1 :'.P#P0G07&2

Today

Dry

Rain

Today+ 1 day

Dry

Rain

Today+ 2 day

Dry

Rain

Today+N day

Dry

Rain

the chance of it being dry tomorrow is 80% if it is dry today,but only 60% if it is raining today

80%

40%

80%

40%

80%

40%

Pr(Rain in 2 days | it is Dry today) =

Rain

0.8x0.20+ 0.20x0.40 = 0.24

80%

40%


21/53

M!

!>3%&$ O'#1207.1 :'.P#P0G07&2

!

56 4 *-4(20487 =48>2) '1490 9* 90 4 *-4-& i4-

(;& mO G14- 9* -1& 2%&$ %'#1207.1 $'.P#P0G0%D

26 4 -840*9(20 682; *-4-& )-2 *-4-&,S

( ) ( ) ( )!"#!$!$ %"&'"&' ======+ !""

!"#$


22/53

MM

O"& 8%"&' EF#4$G&

! 56


23/53

MR

8"#% #P.?% #T&' U =#D2;

!

Z& 438&4/7 >02G 40/

! ,2 -1&


24/53

MV

8"#% 0H %"&'& C&'& U 2%#%&2;

0 1 2

( )201

p =

General case for three states:


25/53

MY

)*VU !"#$4#1>W.G4.A.'./ EX?#7.12

!

,2O 96 (h! -1&0

! J0/ 96 (h)X!O -1&0

!

@28 0hM

( ) ( ) ( ) !"###$

%

"!%&

"'%

!'&


26/53

00 01 0

10 11 1

0 1

M

M

M M MM

p p p

p p p

p p p

! "# $# $

=

# $# $# $% &

P

!

!

" " # "

!

!

!!!#

$

$$$&

!

!!!#

$

$$$&

=

!!!!

!

!

!!!!

!

!

"""

"""

"""

"""

"""

"""

!"!!

!"!!

!"

!!!!"

""!""

!"

!!!!"

""!""

#!


27/53

!!!

!!!!

!#

$$$

$$$$

$&

=

'''

'''

'''

===

===

===

!

"

"!!"

!

"

"!"

!

"

"!"

!

"

"!"

!

"

""

!

"

""

!

"

"!"

!

"

""

!

"

""

######

######

######

!!

"

!

!

!

"

!

""

!

!"

!

!

!

"!

!

!!

#

!"!!

!

L1&8&628&O 9* -1& $!'#*&3&;&0- 26 !M

H96 72. '2.0- -1& B8*- 82G 40/ '23.;0 4* `I

( )!!"#

( )

!==

!

""#$"$# %%% !

"Chapman-Kolmogorov

Equations


28/53

M_

)*VU !"#$4#1>W.G4.A.'./ EX?#7.12

!

,2O 96 (h! -1&0

! J0/ 96 (h)X!O -1&0

! Z19'1 90/9'4-&+

!

40/ 9* -1& $,'th&3&;&0- 26 !)

! 40/

( ) ( )!=

"=

!

"

#"$%"

#%$ &&&

!

"

( ) ( )

!="

=

!

""#

$

%"

$

%# &&&!

"

( ) ( )

( )

1 1n n n n

n n

! !

= " = " =

# =

P P P P P P

P P!"#$

( )n m n m+= !P P P


29/53

Mb

O"& 8%"&' EF#4$G&

!

o.&*(20+96 72. *-48- 20 4 /87 /47 H $h` IO-1&


30/53

R`

!

Z1&0 72. /2 02- >02G -1& *-4-& 26 -1& =48>2)

'1490 4- (;& #h` HG& >02G -1& 2) '1490S

!:82$4$939-7 26 $&90A 90 *-4-&'4- (;& )

! CE4;


31/53

R!

)*VZ !G#22069#7.1 .H 3%#%&2 01 # ,#'-./ !"#01

!

J $#%" 682; *-4-& i-2 *-4-&j9* 4 *&m.&0'& 26

-840*9(20* -14- $&A90* 90 i40/ &0/* 90jO *.'1 -14-

&4'1 -840*9(20 90 -1& *&m.&0'& 14* 4


32/53

! LG2 *-4-&* i40/j48& *49/ -2 9.44?109#%&96j9*

8&4'14$3& 682; iO 40/ i9* 8&4'14$3& 682;jS

!

CE4;2) '1490 9* 90 4 *&2)

'1490 9* *49/ -2 $& 0''&=?90PG&

0 1 2 3 4

0 .4 .6 0 0 0

1 .5 .5 0 0 0

2 0 0 .3 .7 0

3 0 0 .5 .4 .1

4 0 0 0 .8 .2

P

! "# $# $# $=# $# $# $% &


33/53

! J *-4-& $9* 4 %'#120&1% 2%#%&96 -1&8& &E9*-* 4 *-4-&'

-14- 9* 4''&**9$3& 682; $O $.- -1& *-4-& $9* 02-

4''&**9$3& 682; *-4-&'S" L1&8& 9* 4 '140'& -14- -1&


34/53

RV

Q#4PG&'R2 K?01

!

5/&0(67 -1& *-4-&*d -7


35/53

RY

O'#120&1% 3%#%& EF#4$G&

!

5/&0(67 -7


36/53

R\

,.'& =&6107.12+ :&'0.=090%D

!

J '&9?''&1%*-4-& $9* $&'0.=09G9-1

-2 *-4-& $14)& 4 3&0A-1 -14- 9* 4 ;.3(19*

*;433&*- 90-&A&8 G9-1 -19*


37/53

R]

E'A.=09 ,#'-./ !"#01

!

J B09-&X*-4-& 988&/.'9$3& =48>2) ?1490 G1&8&

433 *-4-&* 48& 8&'.88&0-40/ 4


38/53

R_

)*V[ \.1A K?1 :'.$&'7&2 .H ,#'-./ !"#012

!

3%=D>2%#%& $'.P#P0G07&2 48& .*&/ -2 /&*'89$&

-1& 320AX8.0 $&14)928 26 4 =48>2) '1490

! 50 *2;& 648 2p 6.-.8& (;&O G14- 9* -1&


39/53

Rb

3%=D>3%#%& :'.P#P0G07&2

!

O"&.'&4+T&- "$& -1& -840*9(20 ;4-89E 628 40

-*%*-4-& C8A2/9' '1490S L1&0 -1&8& &E9*-* 4

)&'-28

*.'1 -14- H320AX-&8; 6.-.8& *-4-& 9* 90/&


40/53

!

L19* -1&28&; -&33* .* -14- 628 407 909(43 *-4-& $O

! L1& )&'-28 "h s1213 U 1%t 9* 2W&0 '433&/ -1& 2%=D>

2%#%& =02%'0P?7.1O 28 &X?0G0P'0?4 =02%'0P?7.1O 628 -1&

=48>2) '1490S

!

C4'1 1'8&


41/53

V!

S#9- %. %"& 8%"&' EF#4$G&

3 0.8 0.2 0.76 0.24 0.752 0.248

0.6 0.4 0.72 0.28 0.744 0.256

! " ! " ! "= =# $ # $ # $

% & % & % &P

4 0.752 0.248 0.80 0.20

0.744 0.256 0.60 0.40

! " ! "= # $ # $

% & % &P

5 0.75 0.25 0.80 0.20

0.749 0.251 0.60 0.40

! " ! "= # $ # $

% & % &

P

6 0.75 0.25 0.80 0.20

0.75 0.25 0.60 0.40

! " ! "= =# $ # $

% & % &P

!#

$&

=!#

$&

!#

$&

=

!"#$%!#$

!$%'#$

$'#$

!#$"#$

$'#$

!#$"#$!!


42/53

VM

]01=01A %"& 3%=D 3%#%& :'.P#P0G07&2

!

v4'> -2 -1& ?14


43/53

VR

S#9- %. %"& 8%"&' EF#4$G&

!

"

=!=

!

"

""and

State 0 = Dry and State 1 = Rain

1

3

2

0.8 0.20.6 0.4

! "= # $% &

P

0

0,...,

M

j i ij

i

p

j M

! !

=

=

=

"


44/53

VV

3

2


45/53

VY

,.1.$.GD EF#4$G&

!

J**.;90A -14- 96 72. 340/ 90 %493O 72. 14)& -2

*-47 90 %493 628 R -.80* 28 .0(3 72. 8233 /2.$3&*O

-1& 320A -&8;


46/53

! =&/9-&8840&40 J)& `S`MR]

b ?200&'('.- J)& `S`MR]

R v43(' J)& `S`MV!

R] :48> :34'& `S`MVY

\ #89&0-43 J)& `S`MYR

_ r&8;20- J)& `S`MY_

!R ,-4-& J)& `S`MY_

RV :&00*73)4094 J)& `S`M]b

!V r98A9094 J)& `S`M__

Mb =48)90 c48/&0* `S`M_b

RM n28-1 ?4823904 J)& `S`MbV

Rb v248/G43> `S`MbY

M] r&0-028 J)& `S`Mbb

R! :4'9B' J)& `S`R``

M\ J-340(' J)& `S`R`!

!! ,- ?1483&* :34'& `S`R`V

MR 50/9404 J)& `S`R`Y

M! x&0-.'>7 J)& `S`R!`

!\ ,- y4;&* :34'& `S`R!_

!b n&G e28> J)& `S`RRV

!_ L&00&**&& J)& `S`RRY

MV 5339029* J)& `S`RYY


47/53

V]


48/53

V_

OC. 4.'& %"01A2

!

56 9 40/ % 48& 8&'.88&0- *-4-&* 682; /9p&8&0-

'34**&*+

! 56 4 *-4-&'9* -840*9&0-O -1&0

!

L1&


49/53

Vb

EF$&9%&= !.2%^K&C#'=

!

56 5 >02G -1& 320A 8.0


50/53

Y`

EF#4$G&

!

Z&4-1&8 CE4;&- 48& fY`` 20 4

84907 /47O 40/ fM``` G1&0 9- /2&*0d- 8490S

Z14-d* -1& 4)&84A& *43&* 3&)&3 &- 8.0* 628 Y` /47* 0&E- 7&48O G14- 48&

-1& -2-43 &E


51/53

Y!

EF$&9%&= !.2%^K&C#'= H.' !.4$G&F

:'.PG&42

!

Z14- 96 -1& '2*- /&


52/53

YM

,.'& 9.4$G&F #/&'#A& 9.2%2

5#+ 4 840/2; )4894$3& 90/9'4(0A


53/53

YR

S#9- %. %"& $'.PG&4

z-+

?Hz-Oe-I+ N490 D87

e-+ n2X,19< Y`` M```

,19< !Y`` V```

Ch16 Markov Handout Pp 1-53

Documents

Transcript of Ch16 Markov Handout Pp 1-53