Post on 25-Aug-2018
1.SampleSpaceandProbabilityPartI
ECE302Fall2009TR3‐4:15pmPurdueUniversity,SchoolofECE
Prof.IlyaPollak
ProbabilisJcModel
• SamplespaceΩ=thesetofallpossibleoutcomesofanexperiment– E.g.,{+$1,‐$1}foraone‐monthstockmovementintheopJonexample;{$102,$100,$98}forthestockvalueaUertwomonths;{Obama,McCain,neither}foravoter’spreference,etc.
• ProbabilitylawwhichassignstoasetAofpossibleoutcomes(alsocalledanevent)anumberP(A),calledtheprobabilityofA.
Sets:basictermsandnotaJon,1
€
A countably infinite (or countable) set is a set with infinitelymany elements which can be enumerated in a list, e.g.,the set of all integers 0,−1,1,−2,2,…{ }An example of an uncountable set is the set of all real numbersbetween 0 and 1, denoted [0,1]
Sets:basictermsandnotaJon,2
€
The set of all x that have a certain property P is denoted byx | x satisfies P{ },
e.g., the interval [0,1] can alternatively be written as x | 0 ≤ x ≤1{ }.
SetoperaJons:complement
SetoperaJons:union
€
More generally,
Snn=1
∞
= S1∪ S2 ∪… = x | x ∈ Sn for some n{ }
SetoperaJons:intersecJon
€
More generally,
Snn=1
∞
= S1∩ S2 ∩… = x | x ∈ Sn for all n{ }
Disjointsets
ParJJon
CollecJvelyexhausJvesets
Example:threecointosses
H
H
H
T T
T
H
T
H
H
T
H
T
T
HHH
HHT
HTH
HTT
THH
THT
TTH
TTT
€
Sample space Ω = HHH,HHT,HTH,HTT,THH,THT,TTH,TTT{ }Event "heads on the first and second toss" is the set HHH,HHT{ }
Example:threecointosses
Example:threecointosses
Example:threecointosses
Example:threecointosses
Example:threecointosses
Example:threecointosses
Example:presidenJalelecJon% for Obama
% for McCain
100%
100%
Example:presidenJalelecJon% for Obama
% for McCain
100%
100%
Note: Even though the sample space is discrete in reality (the quantum is 1 voter out of 130M, or 0.00000077%), it is more conveniently modeled as continuous.
Example:presidenJalelecJon% for Obama
% for McCain
100%
100%
Is this sample space enough for answering these questions: (1) Will Obama win the popular vote? (2) Will Obama win the election?
Example:presidenJalelecJon% for Obama
% for McCain
100%
100%
Is this sample space enough for answering these questions: (1) Will Obama win the popular vote? (2) Will Obama win the election? (1) No: need to include 3rd-party candidates (2) No: need both 3rd-party candidates and electoral
(not popular) vote
Example:presidenJalelecJon% for Obama
% for McCain
100%
100%
Example:presidenJalelecJon% for Obama
% for McCain
100%
100%
ProbabilisJcModel
• SamplespaceΩ• Probabilitylaw:assignstoaneventAanumberP(A)saJsfyingthefollowingprobabilityaxioms:
€
1. P(A) ≥ 02. If A∩ B =∅, then P(A∪ B) = P(A) + P(B)
If A1,A2,… are disjoint, then P Ann=1
∞
= P(An )
n=1
∞
∑
3. P(Ω) =1
RelaJonshipbetweenprobabilityandrelaJvefrequencyofoccurrence
Example:threecointosses
€
Suppose all outcomes are equally likelyP(Ω) =1 by axiom 3P HHH{ }( ) + P HHT{ }( ) +…+ P TTT{ }( ) = P(Ω) by axiom 2
P HHH{ }( ) = P HHT{ }( ) =… = P TTT{ }( ) =1/8
P(S4 ) = P("two tails in a row") = P HTT,TTH,TTT{ }( ) = 3/8 by axiom 2
€
Discrete uniform probability law : if Ω consists of N equally likelyoutcomes, then, for any event A,
P(A) =number of elements in A
N
Example:presidenJalelecJon% for Obama
% for McCain
100%
100%
Example:presidenJalelecJon% for Obama
% for McCain
100%
100%
Example:someproperJesofprobabilitylaws
Example:someproperJesofprobabilitylaws
Example:someproperJesofprobabilitylaws
Example:someproperJesofprobabilitylaws
Example:someproperJesofprobabilitylaws
Example:someproperJesofprobabilitylaws
€
(a) If A ⊂ B then P(A) ≤ P(B)Proof : Let C = Ac ∩ BThen B = A∪Cand C∩ A =∅ (since C ⊂ Ac ).Therefore,
P(B) = P(A∪C) =Axiom 2
P(A) + P(C) ≥Axiom 1 applied to P (C )
P(A)
Example:someproperJesofprobabilitylaws
Example:someproperJesofprobabilitylaws
Example:someproperJesofprobabilitylaws
Example:someproperJesofprobabilitylaws
Example:someproperJesofprobabilitylaws
Example:someproperJesofprobabilitylaws
Example:someproperJesofprobabilitylaws
€
(c) P(A∪ B) ≤ P(A) + P(B)Proof :Use property (b) : P(A∪ B) = P(A) + P(B) − P(A∩ B)
≥0, by Axiom 1
≤ P(A) + P(B)
Example:someproperJesofprobabilitylaws
€
(d) P(A∪ B∪C) = P(A) + P(Ac ∩ B) + P(Ac ∩ Bc ∩C)Proof :Since A, Ac ∩ B, Ac ∩ Bc ∩C form a partition for A∪ B∪C,the statement follows from Axiom 2.
CondiJonalProbability• NotaJon:P(A|B)• ProbabilityofeventA,giventhateventBoccurred• DefiniJon:assumingP(B)≠0,
• CondiJonalprobabiliJesspecifyaprobabilitylawonthenewuniverseB(exercise)
Example:ProblemofPoints• Besttwooutofthreefaircoinflips• HelenbetsonH,TombetsonT
(a) What’stheprobabilitythatHelenwins1stround?
(b) What’stheprobabilitythatHelenwinsoverall?(c) ThegameisinterruptedaUerHelenwins1stround.
What’sthecondiJonalprobabilitythatshewouldhavewonoverall?
• SoluJon(a) ½‐‐followsdirectlyfromthefactthatthecoinisfair.
ProblemofPoints,SoluJon
H
H
H
T T
T
H
T
H
H
T
H
T
T
HHH
HHT
HTH
HTT
THH
THT
TTH
TTT
H wins 1st round
H wins overall
P(H wins 1st round) = 1/2 P(H wins 1st round and H wins overall) = 3/8
P(H wins overall | H wins 1st round) = P(H wins 1st round and H wins overall)/ P(H wins 1st round) = (3/8)/(1/2) = 3/4
AlternaJvely,…
HHH
HHT
HTH
HTT
H wins 1st round
H wins overall View the blue set as the new sample space. Three out of four equally likely outcomes result in H’s overall win. Therefore, P(H wins overall | H wins 1st round) = 3/4.
Example1.9:IntrusionDetecJonEvent A: intrusion Event B: alarm
Suppose that, from past experiences, we know that P(A) = 0.02, P(B|Ac) = 0.05, P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm probability P(B and Ac)
Example:IntrusionDetecJon
A Bc∩A
Event A: intrusion Event B: alarm
Suppose that, from past experiences, we know that P(A) = 0.02, P(B|Ac) = 0.05, P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm probability P(B and Ac)
Ac
B∩A
B∩Ac
Bc∩Ac
Example:IntrusionDetecJon
P(A) = 0.02 A
Bc∩A = missed detection
P(B|Ac)=0.05
Event A: intrusion Event B: alarm
Suppose that, from past experiences, we know that P(A) = 0.02, P(B|Ac) = 0.05, P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm probability P(B and Ac)
Ac
B∩A
B∩Ac = false alarm
Bc∩Ac
P(Bc|A)=0.01
Example:IntrusionDetecJon
P(A) = 0.02 A
Bc∩A = missed detection
P(B|Ac)=0.05
Event A: intrusion Event B: alarm
Suppose that, from past experiences, we know that P(A) = 0.02, P(B|Ac) = 0.05, P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm probability P(B and Ac)
Ac
B∩A
B∩Ac = false alarm
Bc∩Ac
P(Bc|A)=0.01
P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
Example:IntrusionDetecJon
P(A) = 0.02 A
Bc∩A = missed detection
P(B|Ac)=0.05
Event A: intrusion Event B: alarm
Suppose that, from past experiences, we know that P(A) = 0.02, P(B|Ac) = 0.05, P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm probability P(B and Ac)
P(Ac) = 0.98 Ac
B∩A
B∩Ac = false alarm
Bc∩Ac
P(Bc|A)=0.01
P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
Example:IntrusionDetecJon
P(A) = 0.02 A
Bc∩A = missed detection
P(B|Ac)=0.05
Event A: intrusion Event B: alarm
Suppose that, from past experiences, we know that P(A) = 0.02, P(B|Ac) = 0.05, P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm probability P(B and Ac)
P(Ac) = 0.98 Ac
B∩A
B∩Ac = false alarm
Bc∩Ac P(Bc|Ac)=0.95
P(Bc|A)=0.01
P(B|A)=0.99
P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
Example:IntrusionDetecJon
P(A) = 0.02 A
Bc∩A = missed detection
P(B|Ac)=0.05
Event A: intrusion Event B: alarm
Suppose that, from past experiences, we know that P(A) = 0.02, P(B|Ac) = 0.05, P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm probability P(B and Ac)
P(Ac) = 0.98 Ac
B∩A
B∩Ac = false alarm
Bc∩Ac P(Bc|Ac)=0.95
P(Bc|A)=0.01
P(B|A)=0.99
P(B∩A) = P(A)P(B|A) = 0.02·0.99 = 0.0198 P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
Example:IntrusionDetecJon
P(A) = 0.02 A
Bc∩A = missed detection
P(B|Ac)=0.05
Event A: intrusion Event B: alarm
Suppose that, from past experiences, we know that P(A) = 0.02, P(B|Ac) = 0.05, P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm probability P(B and Ac)
P(Ac) = 0.98 Ac
B∩A
B∩Ac = false alarm
Bc∩Ac P(Bc|Ac)=0.95
P(Bc|A)=0.01
P(B|A)=0.99
P(B∩Ac) = P(Ac)P(B|Ac) = 0.98·0.05 = 0.049 P(B∩A) = P(A)P(B|A) = 0.02·0.99 = 0.0198 P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
Example:IntrusionDetecJon
P(A) = 0.02 A
Bc∩A = missed detection
P(B|Ac)=0.05
Event A: intrusion Event B: alarm
Suppose that, from past experiences, we know that P(A) = 0.02, P(B|Ac) = 0.05, P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm probability P(B and Ac)
P(Ac) = 0.98 Ac
B∩A
B∩Ac = false alarm
Bc∩Ac P(Bc|Ac)=0.95
P(Bc|A)=0.01
P(B|A)=0.99 Event B
P(B∩Ac) = P(Ac)P(B|Ac) = 0.98·0.05 = 0.049 P(B∩A) = P(A)P(B|A) = 0.02·0.99 = 0.0198 P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
Example:IntrusionDetecJon
P(A) = 0.02 A
Bc∩A = missed detection
P(B|Ac)=0.05
Event A: intrusion Event B: alarm
Suppose that, from past experiences, we know that P(A) = 0.02, P(B|Ac) = 0.05, P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm probability P(B and Ac)
P(Ac) = 0.98 Ac
B∩A
B∩Ac = false alarm
Bc∩Ac P(Bc|Ac)=0.95
P(Bc|A)=0.01
P(B|A)=0.99 Event B
P(B∩Ac) = P(Ac)P(B|Ac) = 0.98·0.05 = 0.049 P(B∩A) = P(A)P(B|A) = 0.02·0.99 = 0.0198
P(B) = P(B∩A) + P(B∩Ac) = 0.0198 + 0.049 = 0.0688
P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
Example:IntrusionDetecJon
P(A) = 0.02 A
Bc∩A = missed detection
P(B|Ac)=0.05
Event A: intrusion Event B: alarm
Suppose that, from past experiences, we know that P(A) = 0.02, P(B|Ac) = 0.05, P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm probability P(B and Ac)
P(Ac) = 0.98 Ac
B∩A
B∩Ac = false alarm
Bc∩Ac P(Bc|Ac)=0.95
P(Bc|A)=0.01
P(B|A)=0.99 Event B
P(B∩Ac) = P(Ac)P(B|Ac) = 0.98·0.05 = 0.049
P(A|B) = P(A∩B)/P(B) =0.0198/0.0688 ≈ 0.288
P(B∩A) = P(A)P(B|A) = 0.02·0.99 = 0.0198
P(B) = P(B∩A) + P(B∩Ac) = 0.0198 + 0.049 = 0.0688
P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
Example1.18:False‐PosiJvePuzzle
• Acrimecommikedonanisland,populaJon5000
• Apriori,allareequallylikelytohavecommikedit• Basedonaforensictest,asuspectisarrested• Apartfromthetest,nootherevidence
• Accuracyofthetestis99.9%,i.e.,P(testisposiJve|suspectisguilty)=0.999andP(testisnegaJve|suspectisinnocent)=0.999
• Youareonthejury.Doyouhave“reasonabledoubt”?
False‐PosiJvePuzzle:SoluJon
• Proceedsimilartotheintruderexample
• P(+)=P(+andguilty)+P(+andinnocent)=P(+|guilty)P(guilty)+P(+|innocent)P(innocent)=0.999∙0.0002+0.001∙0.9998=0.0011996
• P(guilty|+)=P(+andguilty)/P(+)=0.999∙0.0002/0.0011996≈0.167!!!
• Theseeminglyreliabletestisnotveryreliableatall!
• Supposethisisrepeatedon1000islands• Supposewetestall5,000,000peopleon1000islands
False‐PosiJvePuzzle:SomeIntuiJon
• Supposethisisrepeatedon1000islands• Supposewetestall5,000,000peopleon1000islands• Onaverage,weexpectroughlythefollowingtestresults:
False‐PosiJvePuzzle:SomeIntuiJon
5,000,000 people
1000 guilty
4,999,000 innocent
~1 tests -
~999 test +
• Supposethisisrepeatedon1000islands• Supposewetestall5,000,000peopleon1000islands• Onaverage,weexpectroughlythefollowingtestresults:
False‐PosiJvePuzzle:SomeIntuiJon
5,000,000 people
1000 guilty
4,999,000 innocent
~1 tests -
~999 test +
~4999 test +
~4,994,001 test -
• Supposethisisrepeatedon1000islands• Supposewetestall5,000,000peopleon1000islands• Onaverage,weexpectroughlythefollowingtestresults:
False‐PosiJvePuzzle:SomeIntuiJon
5,000,000 people
1000 guilty
4,999,000 innocent
~1 tests -
~999 test +
~4999 test +
~4,994,001 test -
~999 criminals out of ~5998 who tested + i.e., roughly 1/6
• Supposethisisrepeatedon1000islands• Supposewetestall5,000,000peopleon1000islands• Onaverage,weexpectroughlythefollowingtestresults:
• I.e.,therearesofewcriminalsthatthebulkofpeoplewhotestposiJveareinnocent!
False‐PosiJvePuzzle:SomeIntuiJon
5,000,000 people
1000 guilty
4,999,000 innocent
~1 tests -
~999 test +
~4999 test +
~4,994,001 test -
~999 criminals out of ~5998 who tested + i.e., roughly 1/6
TotalProbabilityTheorem
A1
A2 A3B
TotalProbabilityTheorem
• OnewayofcompuJngP(B):P(B)=P(B∩A1)+P(B∩A2)+P(B∩A3)
=P(B|A1)P(A1)+P(B|A2)P(A2)+P(B|A3)P(A3)
A1
A2 A3B
TotalProbabilityTheorem
• OnewayofcompuJngP(B):P(B)=P(B∩A1)+P(B∩A2)+P(B∩A3)
=P(B|A1)P(A1)+P(B|A2)P(A2)+P(B|A3)P(A3)
• Moregenerally,
P(B)=ΣiP(B|Ai)P(Ai)ifAi’saremutuallyexclusiveandBisasubsetoftheunionofAi’s
A1
A2 A3B
Bayes’Rule• Priormodel:probabiliJesP(Ai)
Bayes’Rule• Priormodel:probabiliJesP(Ai)• Measurementmodel:P(B|Ai)
– condiJonalprobabilitytoobservedataBgiventhatthetruthisAi
Bayes’Rule• Priormodel:probabiliJesP(Ai)• Measurementmodel:P(B|Ai)
– condiJonalprobabilitytoobservedataBgiventhatthetruthisAi
• WanttocomputeposteriorprobabiliBesP(Ai|B)– CondiJonalprobabilitythatthetruthisAigiventhatweobserveddataB
Bayes’Rule• Priormodel:probabiliJesP(Ai)• Measurementmodel:P(B|Ai)
– condiJonalprobabilitytoobservedataBgiventhatthetruthisAi
• WanttocomputeposteriorprobabiliBesP(Ai|B)– CondiJonalprobabilitythatthetruthisAigiventhatweobserveddataB
€
P(Ai |B) =P(Ai∩ B)P(B)
=P(B | Ai)P(Ai)
P(B)
=P(B | Ai)P(Ai)P(B | A j )P(A j )
j∑
(by total probability thm)
MulJplicaJonRule
€
P Aii=1
n
= P(A1)P(A2 | A1)P(A3 | A1∩ A2) ⋅… ⋅ P An Ai
i=1
n−1
,
provided all the conditioning events have nonzero probability
Example1.10• Threecardsaredrawnfromadeckof52cards,without
replacement.Ateachstep,eachoneoftheremainingcardsisequallylikelytobepicked.What’stheprobabilitythatnoneofthethreecardsisaheart?
• LetAi={i‐thcardisnotaheart},i=1,2,3
Example• Threecardsaredrawnfromadeckof52cards,without
replacement.Ateachstep,eachoneoftheremainingcardsisequallylikelytobepicked.What’stheprobabilitythatnoneofthethreecardsisaheart?
• LetAi={i‐thcardisnotaheart},i=1,2,3
€
P(A1) =3952
€
P(A1c ) =
1352
€
A1
€
A1c
Example• Threecardsaredrawnfromadeckof52cards,without
replacement.Ateachstep,eachoneoftheremainingcardsisequallylikelytobepicked.What’stheprobabilitythatnoneofthethreecardsisaheart?
• LetAi={i‐thcardisnotaheart},i=1,2,3
€
P(A1) =3952
€
P(A1c ) =
1352
€
A1
€
A1c
€
A1∩ A2
€
A1∩ A2c€
P(A2 | A1) =3851
€
1351
Example• Threecardsaredrawnfromadeckof52cards,without
replacement.Ateachstep,eachoneoftheremainingcardsisequallylikelytobepicked.What’stheprobabilitythatnoneofthethreecardsisaheart?
• LetAi={i‐thcardisnotaheart},i=1,2,3
€
P(A1) =3952
€
P(A1c ) =
1352
€
A1
€
A1c
€
A1∩ A2
€
A1∩ A2c
€
A1∩ A2∩ A3
€
A1∩ A2∩ A3c
€
P(A2 | A1) =3851
€
1351
€
1350
€
P(A3 | A1∩ A2) =3750
Example• Threecardsaredrawnfromadeckof52cards,without
replacement.Ateachstep,eachoneoftheremainingcardsisequallylikelytobepicked.What’stheprobabilitythatnoneofthethreecardsisaheart?
• LetAi={i‐thcardisnotaheart},i=1,2,3
€
P(A1) =3952
€
P(A1c ) =
1352
€
A1
€
A1c
€
A1∩ A2
€
A1∩ A2c
€
A1∩ A2∩ A3
€
A1∩ A2∩ A3c
€
P(A2 | A1) =3851
€
1351
€
1350
€
P(A3 | A1∩ A2) =3750
€
P(A1∩ A2∩ A3) = P(A1)P(A2 | A1)P(A3 | A1∩ A2) =3952
⋅3851⋅3750
≈ 0.41
Example:Prisoner’sDilemma(p.58)• Threeprisoners:A,B,andC.• Onewillbeexecutednextday,tworeleased.• PrisonerAaskstheguardtotellhimthenameofoneoftheothertwowhowillbereleased.
• GuardsaysthatBwillbereleased.(Assumethattheguardistellingthetruth.)
• Aargues:beforemychancestobeexecutedwere1/3,nowtheyare1/2sinceIknowit’seithermeorC.What’swrongwithhisreasoning?
Prisoner’sDilemma:Afewremarks
• Theprisonersonlyknowthatoneofthemwillbeexecuted,butdonotknowwhichone.Thus,fromtheirpointofview,areasonablemodelisadiscreteuniformprobabilitylawthatassignseachofthemprobability1/3tobeexecuted.
• Theguardknowswhichoneofthemwillbeexecuted.• SinceAalreadyknowsthateitherBorCwillbereleased,itdoesnotseemlikeknowingthatBwillbereleasedshouldinfluenceA’schancesinanyway.Infact,A’sprobabilitytobeexecutedissJll1/3(fromA’spointofview),asshowninthenextfewslides.
Prisoner’sDilemma:SoluJon• Theguard’sresponseneedstobeincludedintheprobabilisJc
model.
• LetEi={prisoneriwillbeexecuted}fori=A,B,C• LetGj={guardnamesprisonerj}forj=B,C
Prisoner’sDilemma:SoluJon• Theguard’sresponseneedstobeincludedintheprobabilisJc
model.
• LetEi={prisoneriwillbeexecuted}fori=A,B,C• LetGj={guardnamesprisonerj}forj=B,C
EA
EB
EC
1/3
1/3
1/3
Prisoner’sDilemma:SoluJon• Theguard’sresponseneedstobeincludedintheprobabilisJc
model.
• LetEi={prisoneriwillbeexecuted}fori=A,B,C• LetGj={guardnamesprisonerj}forj=B,C
EA
EB
EC
1/3
1/3
1/3
1/2
1/2
EA∩GB
EA∩GC
Prisoner’sDilemma:SoluJon• Theguard’sresponseneedstobeincludedintheprobabilisJc
model.
• LetEi={prisoneriwillbeexecuted}fori=A,B,C• LetGj={guardnamesprisonerj}forj=B,C
EA
EB
EC
1/3
1/3
1/3
1/2
1/2
0
1
EA∩GB
EA∩GC EB∩GB
EB∩GC
Prisoner’sDilemma:SoluJon• Theguard’sresponseneedstobeincludedintheprobabilisJc
model.
• LetEi={prisoneriwillbeexecuted}fori=A,B,C• LetGj={guardnamesprisonerj}forj=B,C
EA
EB
EC
1/3
1/3
1/3
1/2
1/2
0
1
10
EA∩GB
EA∩GC EB∩GB
EB∩GC
EC∩GB
EC∩GC
Prisoner’sDilemma:SoluJon• Theguard’sresponseneedstobeincludedintheprobabilisJc
model.
• LetEi={prisoneriwillbeexecuted}fori=A,B,C• LetGj={guardnamesprisonerj}forj=B,C
EA
EB
EC
1/3
1/3
1/3
1/2
1/2
0
1
10
EA∩GB
EA∩GC EB∩GB
EB∩GC
EC∩GB
EC∩GC
1/6
1/6
0
1/3
1/3
0
Prisoner’sDilemma:SoluJon• Theguard’sresponseneedstobeincludedintheprobabilisJc
model.
• LetEi={prisoneriwillbeexecuted}fori=A,B,C• LetGj={guardnamesprisonerj}forj=B,C
EA
EB
EC
1/3
1/3
1/3
1/2
1/2
0
1
10
EA∩GB
EA∩GC EB∩GB
EB∩GC
EC∩GB
EC∩GC
1/6
1/6
0
1/3
1/3
0
P(EA|GB) = P(EA∩GB)/P(GB) = (1/6)/(1/6 + 0 + 1/3) = 1/3
TheMontyHallPuzzle(Ex.1.12)
• Prizebehindoneofthreedoors.• Contestantpicksadoor.• Host(whoknowswheretheprizeis)opensoneoftheremainingtwodoorswhichdoesnothavetheprize.
• Contestantisofferedanopportunitytostaywithhisdoor,ortoswitchtoanotherdoor.
• Stayorswitch?
TheMontyHallPuzzle:SoluJon
• Ifstay,P(win)=1/3
TheMontyHallPuzzle:SoluJon
• Ifstay,P(win)=1/3• Ifswitch,theonlywaytoloseisifiniJallypointedtothedoorwithprize:P(lose)=1/3andsoP(win)=2/3
TheMontyHallPuzzle:SoluJon
• Ifstay,P(win)=1/3• Ifswitch,theonlywaytoloseisifiniJallypointedtothedoorwithprize:P(lose)=1/3andsoP(win)=2/3
• Conclusion:mustswitch!
• Switchingisadvantageousbecausethehost’sacJontellsyousomething.IfyouiniJallypickedadoorwithnoprize,heisforcedtoopentheotherdoorwithnoprize.
TheMontyHallPuzzle:Discussion
• Crucialpartsoftheproblemstatement:– thehostknowswheretheprizeis– hemustopenthedoorwithnoprize
TheMontyHallPuzzle:Discussion
• Crucialpartsoftheproblemstatement:– thehostknowswheretheprizeis– hemustopenthedoorwithnoprize
• Thus,ifyouhavechosenadoorwithnoprize,youareforcinghimtoopentheonlyotherdoorwithnoprizeandthusshowyouwheretheprizeis.
TheMontyHallPuzzle:AnotherSoluJon
• UsethetotalprobabilitytheoremtoevaluatetheprobabiliJesofwinningunderthetwostrategies.
TheMontyHallPuzzle:AnotherSoluJon
• UsethetotalprobabilitytheoremtoevaluatetheprobabiliJesofwinningunderthetwostrategies.
• LetW=“win”andN=“originallypointtoadoorwithnoprize”.
TheMontyHallPuzzle:AnotherSoluJon
• UsethetotalprobabilitytheoremtoevaluatetheprobabiliJesofwinningunderthetwostrategies.
• LetW=“win”andN=“originallypointtoadoorwithnoprize”.• P(N)=2/3;P(Nc)=1/3.
TheMontyHallPuzzle:AnotherSoluJon
• UsethetotalprobabilitytheoremtoevaluatetheprobabiliJesofwinningunderthetwostrategies.
• LetW=“win”andN=“originallypointtoadoorwithnoprize”.• P(N)=2/3;P(Nc)=1/3.
• Ifyoudonotswitch,P(W|N)=0andP(W|Nc)=1.
TheMontyHallPuzzle:AnotherSoluJon
• UsethetotalprobabilitytheoremtoevaluatetheprobabiliJesofwinningunderthetwostrategies.
• LetW=“win”andN=“originallypointtoadoorwithnoprize”.• P(N)=2/3;P(Nc)=1/3.
• Ifyoudonotswitch,P(W|N)=0andP(W|Nc)=1.• So,ifyoudonotswitch,P(W)=P(W|N)P(N)+P(W|Nc)P(Nc)=1/3.
TheMontyHallPuzzle:AnotherSoluJon
• UsethetotalprobabilitytheoremtoevaluatetheprobabiliJesofwinningunderthetwostrategies.
• LetW=“win”andN=“originallypointtoadoorwithnoprize”.• P(N)=2/3;P(Nc)=1/3.
• Ifyoudonotswitch,P(W|N)=0andP(W|Nc)=1.• So,ifyoudonotswitch,P(W)=P(W|N)P(N)+P(W|Nc)P(Nc)=1/3.
• Ifyouswitch,P(W|N)=1becauseifyouoriginallychooseadoorwithnoprizethehostisforcedtoopentheonlyotherdoorwithnoprize.
TheMontyHallPuzzle:AnotherSoluJon
• UsethetotalprobabilitytheoremtoevaluatetheprobabiliJesofwinningunderthetwostrategies.
• LetW=“win”andN=“originallypointtoadoorwithnoprize”.• P(N)=2/3;P(Nc)=1/3.
• Ifyoudonotswitch,P(W|N)=0andP(W|Nc)=1.• So,ifyoudonotswitch,P(W)=P(W|N)P(N)+P(W|Nc)P(Nc)=1/3.
• Ifyouswitch,P(W|N)=1becauseifyouoriginallychooseadoorwithnoprizethehostisforcedtoopentheonlyotherdoorwithnoprize.
• Ifyouswitch,P(W|Nc)=0becauseifyouoriginallychoosethedoorwiththeprize,youwillswitchoutofitandlose.
TheMontyHallPuzzle:AnotherSoluJon
• UsethetotalprobabilitytheoremtoevaluatetheprobabiliJesofwinningunderthetwostrategies.
• LetW=“win”andN=“originallypointtoadoorwithnoprize”.• P(N)=2/3;P(Nc)=1/3.
• Ifyoudonotswitch,P(W|N)=0andP(W|Nc)=1.• So,ifyoudonotswitch,P(W)=P(W|N)P(N)+P(W|Nc)P(Nc)=1/3.
• Ifyouswitch,P(W|N)=1becauseifyouoriginallychooseadoorwithnoprizethehostisforcedtoopentheonlyotherdoorwithnoprize.
• Ifyouswitch,P(W|Nc)=0becauseifyouoriginallychoosethedoorwiththeprize,youwillswitchoutofitandlose.
• So,ifyouswitch,P(W)=P(W|N)P(N)+P(W|Nc)P(Nc)=2/3.
Two‐EnvelopesPuzzle(p.58)
• Youarehandedtwoenvelopes,eachcontaininganintegernumberofdollars,unknowntoyou.
• Thetwoamountsaredifferent.
• Youselectatrandomoneenvelopeandlookinside.
• YoucaneithersJckwiththisenvelopeortaketheotherenvelope.YourobjecJveistogetthelargeramount.
• Doesitmakerwhatyoudo?
Two‐EnvelopesPuzzle:SoluJon• MakeindependentflipsofafaircoinunJlheadscomeupforthe
firstJme.LetX=1/2+numberoftossestogetfirstH.
• Strategy:– Iftheamountinyourenvelope>X,stay
– Iftheamountinyourenvelope<X,switch
Two‐EnvelopesPuzzle:SoluJon• MakeindependentflipsofafaircoinunJlheadscomeupforthe
firstJme.LetX=1/2+numberoftossestogetfirstH.
• Strategy:– Iftheamountinyourenvelope>X,stay
– Iftheamountinyourenvelope<X,switch
• DenotethetwoamountsdandD,d<D
Two‐EnvelopesPuzzle:SoluJon• MakeindependentflipsofafaircoinunJlheadscomeupforthe
firstJme.LetX=1/2+numberoftossestogetfirstH.
• Strategy:– Iftheamountinyourenvelope>X,stay
– Iftheamountinyourenvelope<X,switch
• DenotethetwoamountsdandD,d<D• Case1:X<d‐‐‐alwaysstay‐‐‐winwithprobability1/2
Two‐EnvelopesPuzzle:SoluJon• MakeindependentflipsofafaircoinunJlheadscomeupforthe
firstJme.LetX=1/2+numberoftossestogetfirstH.
• Strategy:– Iftheamountinyourenvelope>X,stay
– Iftheamountinyourenvelope<X,switch
• DenotethetwoamountsdandD,d<D• Case1:X<d‐‐‐alwaysstay‐‐‐winwithprobability1/2• Case2:X>D‐‐‐alwaysswitch‐‐‐winwithprobability1/2
Two‐EnvelopesPuzzle:SoluJon• MakeindependentflipsofafaircoinunJlheadscomeupforthe
firstJme.LetX=1/2+numberoftossestogetfirstH.
• Strategy:– Iftheamountinyourenvelope>X,stay
– Iftheamountinyourenvelope<X,switch
• DenotethetwoamountsdandD,d<D• Case1:X<d‐‐‐alwaysstay‐‐‐winwithprobability1/2• Case2:X>D‐‐‐alwaysswitch‐‐‐winwithprobability1/2• Case3:d<X<D‐‐‐stayifpickD,switchifpickd‐‐‐win!
Two‐EnvelopesPuzzle:SoluJon• MakeindependentflipsofafaircoinunJlheadscomeupforthe
firstJme.LetX=1/2+numberoftossestogetfirstH.
• Strategy:– Iftheamountinyourenvelope>X,stay
– Iftheamountinyourenvelope<X,switch
• DenotethetwoamountsdandD,d<D• Case1:X<d‐‐‐alwaysstay‐‐‐winwithprobability1/2• Case2:X>D‐‐‐alwaysswitch‐‐‐winwithprobability1/2• Case3:d<X<D‐‐‐stayifpickD,switchifpickd‐‐‐win!• P(win)=P(win|X<d)P(X<d)+P(win|X>D)P(X>D)+P(win|d<X<D)P(d<X<D)(bytotalprobabilitytheorem)
Two‐EnvelopesPuzzle:SoluJon• MakeindependentflipsofafaircoinunJlheadscomeupforthe
firstJme.LetX=1/2+numberoftossestogetfirstH.
• Strategy:– Iftheamountinyourenvelope>X,stay
– Iftheamountinyourenvelope<X,switch
• DenotethetwoamountsdandD,d<D• Case1:X<d‐‐‐alwaysstay‐‐‐winwithprobability1/2• Case2:X>D‐‐‐alwaysswitch‐‐‐winwithprobability1/2• Case3:d<X<D‐‐‐stayifpickD,switchifpickd‐‐‐win!• P(win)=P(win|X<d)P(X<d)+P(win|X>D)P(X>D)+P(win|d<X<D)P(d<X<D)=1/2P(X<d)+1/2P(X>D)+P(d<X<D)
Two‐EnvelopesPuzzle:SoluJon• MakeindependentflipsofafaircoinunJlheadscomeupforthe
firstJme.LetX=1/2+numberoftossestogetfirstH.
• Strategy:– Iftheamountinyourenvelope>X,stay
– Iftheamountinyourenvelope<X,switch
• DenotethetwoamountsdandD,d<D• Case1:X<d‐‐‐alwaysstay‐‐‐winwithprobability1/2• Case2:X>D‐‐‐alwaysswitch‐‐‐winwithprobability1/2• Case3:d<X<D‐‐‐stayifpickD,switchifpickd‐‐‐win!• P(win)=P(win|X<d)P(X<d)+P(win|X>D)P(X>D)+P(win|d<X<D)P(d<X<D)=1/2P(X<d)+1/2P(X>D)+P(d<X<D)
=1/2(P(X<d)+P(X>D)+P(d<X<D))+1/2P(d<X<D) =1
Two‐EnvelopesPuzzle:SoluJon• MakeindependentflipsofafaircoinunJlheadscomeupforthe
firstJme.LetX=1/2+numberoftossestogetfirstH.
• Strategy:– Iftheamountinyourenvelope>X,stay
– Iftheamountinyourenvelope<X,switch
• DenotethetwoamountsdandD,d<D• Case1:X<d‐‐‐alwaysstay‐‐‐winwithprobability1/2• Case2:X>D‐‐‐alwaysswitch‐‐‐winwithprobability1/2• Case3:d<X<D‐‐‐stayifpickD,switchifpickd‐‐‐win!• P(win)=P(win|X<d)P(X<d)+P(win|X>D)P(X>D)+P(win|d<X<D)P(d<X<D)=1/2P(X<d)+1/2P(X>D)+P(d<X<D)
=1/2(P(X<d)+P(X>D)+P(d<X<D))+1/2P(d<X<D)=1/2(1+P(d<X<D))>1/2
Two‐EnvelopesPuzzle:Comment
• NotethatXcanbeanyrandomvariablehavingnon‐zerovaluesat3/2,5/2,7/2(or,infact,atanypoint(s)on]1,2[,atanypoint(s)on]2,3[,etc.)