Bayes Nets
description
Transcript of Bayes Nets
![Page 1: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/1.jpg)
Bayes Nets
10-701 recitation04-02-2013
![Page 2: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/2.jpg)
Bayes Nets
• Represent dependencies between variables• Compact representation of probability
distributionFlu Allergy
Sinus
Headache Nose
• Encodes causal relationships
![Page 3: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/3.jpg)
Conditional independence
• P(X,Y|Z) = P(X|Z) x P(Y|Z)
Flu
Sinus
Nose
Nose
Sinus
Headache
F not N⊥
F N | S⊥
N not H⊥
N H | S⊥
![Page 4: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/4.jpg)
Conditional independence
Flu
Sinus
Allergy
F A⊥
F not A | S⊥
• Explaining away:– P(F = t | S = t) is high– But P(F = t | S = t , A = t)
is lower
![Page 5: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/5.jpg)
Joint probability distribution• Chain rule of probability: P(X1, X2, …, Xn) = P( X1) P( X2|X1) … P(Xn|X1,X2…Xn-1)
![Page 6: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/6.jpg)
Joint probability distribution
Flu Allergy
Sinus
Headache Nose
• Chain rule of probability: P(F,A,S,H,N) = P(F) P(A|F)
P(S|A,F) P(H|F,A,S)
P(N|F,A,S,H)Table with 25 entries!
![Page 7: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/7.jpg)
Joint probability distribution
Flu Allergy
Sinus
Headache Nose
P(F) P(A)
P(S|F,A)
P(H|S) P(N|S)
• Local markov assumption: A variable X is independent of it’s non-descendants given it’s parents
P(F,A,S,H,N) = P(F) P(A) P(S|A,F)
P(H|S)P(N|S)
F = t, A = t F = t, A = f F = f, A = t F = f, A = f
S = t 0.9 0.8 0.7 0.1
S = f 0.1 0.2 0.3 0.9
![Page 8: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/8.jpg)
Queries, Inference
Flu Allergy
Sinus
Headache Nose=t
P(F) P(A)
P(S|F,A)
P(H|S) P(N|S)
• P(F = t | N = t) ?
• P(F=t|N=t) = P(F=t,N=t)/P(N=t)
P(F,N=t) = ΣA,S,H P(F,A,S,H,N=t) = ΣA,S,H P(F) P(A)
P(S|A,F) P(H|S)P(N=t|S)
![Page 9: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/9.jpg)
Moralizing the graph
Flu Allergy
Sinus
Nose=t
• Eliminating A will create a factor with F and S
• To assess complexity we can moralize the graph: connect parents
Headache
![Page 10: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/10.jpg)
Chose an optimal order
Flu Allergy
Sinus
Nose=t
If we start with H: P(F,N=t) = ΣA,S P(F) P(A)
P(S|A,F) P(N=t |S) ΣH P(H|S)
= ΣA,S P(F) P(A) P(S|A,F) P(N=t |S)
=1
![Page 11: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/11.jpg)
Flu Allergy
Sinus
Nose=t
Removing S P(F,N=t)= ΣA,S P(F) P(A) P(S|A,F) P(N=t |S) = ΣA P(F) P(A)
Σs P(S|A,F) P(N=t |S) = ΣA P(F) P(A) g1(F,A)
![Page 12: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/12.jpg)
Flu Allergy
Nose=t
Removing A P(F,N=t)= P(F) ΣA P(A) g1(F,A) = P(F) g2(F)
P(F=t|N=t) = P(F=t,N=t)/P(N=t)
P(N=t) = ΣF P(F,N=t)
=P(N=t|F)
![Page 13: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/13.jpg)
Independencies and active trails
Is A H? When is it not? ⊥
A is not H when given C and F or F’ or F’’ ⊥and not {B,D,E,G}
![Page 14: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/14.jpg)
Independencies and active trails
• Active trail between variables X1,X2…Xn-1 when:– Xi-1 -> Xi -> Xi+1 and Xi not observed
– Xi-1 <- Xi <- Xi+1 and Xi not observed
– Xi-1 <- Xi -> Xi+1 and Xi not observed
– Xi-1 -> Xi <- Xi+1 and Xi or one of its descendants is observed
![Page 15: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/15.jpg)
Independencies and active trails
A B ?⊥
![Page 16: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/16.jpg)
Independencies and active trails
B G |E ?⊥
![Page 17: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/17.jpg)
Independencies and active trails
I J |K ?⊥
![Page 18: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/18.jpg)
Independencies and active trails
E F |K ?⊥
![Page 19: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/19.jpg)
Independencies and active trails
F K |I ?⊥
![Page 20: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/20.jpg)
Independencies and active trails
E F |I,K ?⊥
![Page 21: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/21.jpg)
Independencies and active trails
F G |H ?⊥
![Page 22: Bayes Nets](https://reader035.fdocument.pub/reader035/viewer/2022062521/56816689550346895dda4686/html5/thumbnails/22.jpg)
Independencies and active trails
F G |H ,A ?⊥