Discrete Probability
The study of probability uses special jargon
• A sample space in a nonempty set• An experiment is a process that produces a point in a sample space
2
sample space• An event is a subset of sample space• The event space is the power set of the sample space
Experiment: Toss a coin
Sample space: U={H,T}Event Space: P(U)={Ø, {H}, {T}, {H,T}}
Discrete Probability
Remark: Often some of the more interesting events have special name
Experiment: Toss Two Coin
3
Sample space: U={HH, HT, TH, TT}Event Space: Subsets of U
Named events: Match ={HH, TT}At least one head = {HT, TH, HH}
Uniform Probability Measure
Def: The uniform probability measure on a finite sample space S assigns to each event E the probability
||
||)(
S
EEp =
4
Sample space: U={H,T}Event Ø, {H}, {T}, {H,T}Probability 0 ½ ½ 1
|| S
Uniform Probability Measure
Def: The uniform probability measure on a finite sample space S assigns to each event E the probability
||
||)(
S
EEp =
5
Experiment: Toss Two CoinSample space: U={HH, HT, TH, TT}
Event Name At least one head one of eachEvent {HT, TH, HH} {HT, TH}Probability ¾ 2/4
|| S
Uniform Probability Measure
Def: The uniform probability measure on a finite sample space S assigns to each event E the probability
||
||)(
S
EEp =
6
Experiment: Roll a dieSample space: U={1,2,3,4,5,6}Event Name odd even 2 mod 3Even {1,3,5} {2,4,6} {2,5}Probability 3/6 3/6 2/6
|| S
Uniform Probability Measure
Def: The uniform probability measure on a finite sample space S assigns to each event E the probability
||
||)(
S
EEp =
7
Experiment: Roll two dieSample space: U={11,….,16,……,61,………66}Event Name double sum is 9Even {11, 22,….,66} {36,45,54,63} Probability 6/36 4/36
|| S
Uniform Probability Measure
8
Experiment: deal one card from deckSample space: standard 52 card deckEvent Name heart sevenEvenProbability 13/52 4/52
Probability of Complementary Event
Let E be an event in finite sample space S, under the uniform probability distribution. Then
)(1)( EpEp −=
9
Pf:1. |S| = |E| +|E|
)(1)( −=
Probability of a Union Event
Let E1 and E2 be an events in a finite sample space S, under the uniform probability distribution. Then
)()()()( 212121 EEpEpEpEEp ∩−+=∪
10
Pf:1. An integer is chosen from the internal [1,2,….,100]. Find
the probability that it is divisible either by 6 or by 15.
P(6\n v 15\n) = p(6\n) +p(15\n)-p(30\n)= 16/100 + 6/100 -3/100= 19/100
212121
Non-uniform Probability Measure
Here are two non-uniform probability measure for some familiar sample spaces.
The standard loaded coin has probability p(H)=0.8 and p(T)=0.2
11
The standard loaded die has sample space U={1,2,3,4,5,6}
Then assign the singleton {j} the probability j/21
Bernoulli distribution
A Bernoulli distribution is a probability measure on a sample space with exactly two points
Flip standard loaded coinSample space = {H,T}
12
Sample space = {H,T}Event Space = {Ø {H}, {T}, {H,T}}
pr({H}) = 4/5 pr ({T}) = 1/5
Bernoulli distribution
A Bernoulli distribution is a probability measure on a sample space with exactly two points
The standard loaded die induces a Bernoulli distribution with
13
with
pr(even) = 4/7 pr ({odd}) = 3/7
Conditional probability
)()|(
YEprYEpr
∩=
Let p be a probability distribution on a sample space U and let Y be an event. Then the conditional probability of event E given that event Y has occurred is
1414
)(
)()|(
Ypr
YEprYEpr
∩=
3
14/3
4/1
)(
)(
)(
)()|(
=
=
¬=
¬¬∧=¬
TTP
HHp
TTP
TTHHpTTHHp
Conditional Independence
)()|( EprYEpr =
Let pr be a probability distribution on a sample space U.Event E is probabilistically independent of event Y if andonly if
15
)|(
)(/)(
)(/)()Pr(
)(/)()()|(
EYpr
EprEYpr
EprYEprY
YprYEprEprYEpr
=∩=
∩=∩==
Conditional Independence
Roll two fair dice. Let Y be the event that the first die is a one and the event that the sum is odd. Then
pr(E) = 1.1/2.1/2=1/2
)()|(
∩= YEprYEpr
16
2
16
36/1
36/36/1
)16,14,12(
)()|(
=
=
=
=
=
p
YprYEpr
Random Variable
A random variable is a real valued function on the domain of a probability sapce.
Flip a coin three times. Let the X(t) be the number of heads that occurs. Then
17
heads that occurs. Then
X(TTT) = 0X(TTH)=X(THT)=X(HTT) =1X(THH)=X(HHT)=X(HTH)=2X(HHH)=3
Since a random variable is a function, it is not a variable and it is not random.
Bayes Theorem
• Start with the joint probability distribution:
toothache ¬¬¬¬toothache
catch ¬catch catch ¬catch
18
cavity 0.108 0.012 0.072 0.008
¬cavity 0.016 0.064 0.144 0.576
Inference by enumeration
• Start with the joint probability distribution:
toothache ¬¬¬¬toothache
catch ¬catch catch ¬catch
19
• P(toothache) = 0.108 + 0.012 + 0.016 + 0.064 = 0.2
cavity 0.108 0.012 0.072 0.008
¬cavity 0.016 0.064 0.144 0.576
Inference by enumeration• Start with the joint probability distribution:
toothache ¬¬¬¬toothache
catch ¬catch catch ¬catch
cavity 0.108 0.012 0.072 0.008
20
• Can also compute conditional probabilities:P(¬cavity | toothache) = P(¬cavity ∧ toothache)
P(toothache)= 0.016+0.064
0.108 + 0.012 + 0.016 + 0.064= 0.4
¬cavity 0.016 0.064 0.144 0.576
Normalization
toothache ¬¬¬¬toothache
catch ¬catch catch ¬catch
cavity 0.108 0.012 0.072 0.008
¬cavity 0.016 0.064 0.144 0.576
21
• Denominator can be viewed as a normalization constant α
P(Cavity | toothache) = α, P(Cavity,toothache) = α, [P(Cavity,toothache,catch) + P(Cavity,toothache,¬ catch)]= α, [<0.108,0.016> + <0.012,0.064>] = α, <0.12,0.08> = <0.6,0.4>
Top Related