Probability ProbDist

7/26/2019 Probability ProbDist

1/61

Probability & ProbabilityDistributions

Carolyn J. Anderson

EdPsych 580

Fall 2005

Probability & Probability Distributions p. 1/61


2/61

Probability & Probability Distributions

Elementary Probability Theory Definitions

Rules

Bayes Theorem

Probability Distributions

Discrete & continuous variables. Characteristics of distributions.

Expectations

http://-/?-http://-/?-


3/61

Elementary Probability Theory

or

How Likely are the results?

Probabilities arise when sampling individualsfrom a population and in experimental situations,because different trials or replications of the

same experiment usually result in differentoutcomes.



4/61

Statistical ExperimentA (simple) statistical experiment is some well

defined act or process (including sampling) thatleads to one well defined outcome.

Its repeatable (in principle).

There is uncertainty about the results. Uncertainty is modeled by assigning

probabilities to the outcomes.

Examples. . .



5/61

Examples of Statistical ExperimentsWell defined, repeatable, uncertainty, model byprobabilities?

Flip a coin 5 times & record number of heads.

Count the number of blue M& Ms in a 9 oz.

package. Roll two dice & record the total number of

spots.

Ask people who they intend to vote for in thenext presidential election.

Recorded number of correct responses on atest.



6/61

Statistical Experiments

A statistical experiment maybe

Real (it can actually be done).

Conceptualize (completely idealized).



7/61

Definition: Probability

The probability of an eventis the proportion of

times that the event occurs in a large number oftrials of the experiment.

It is the long-run relative frequency of the event.



8/61

Example Experiment: Draw a card from a standard

deck of 52.

Sample space: The set of all possible distinct

outcomes,S(e.g., 52 cards).

Elemenatary event or sample point: amember of the sample space. (e.g., the aceof hearts).

Event(or event class): any set of elementaryevents. e.g., Suit (Hearts), Color (Red), orNumber (Ace).



9/61

Example (continued)

Probability of an Ace =

number of aces

number of cards

= 4

52=.0769

Notes:

Elementary events are equally likely

Denote events by roman letters (e.g.,A,B,etc)

Denote probability of an event asP(A).Probability & Probability Distributions p. 9/61


10/61

More Definitions Joint Eventis when you consider two (or

more events) at a time. e.g.,A=heads on

penny,B =heads on quarter, and joint eventis heads on both coins.

Intersection: (A B) =AandB occur at thesame time.

Union: (A B) =AorB occur

OnlyAoccurs. OnlyB occurs. AandB occur.



11/61

More Definitions Complementof an event is that the event did

not occur. AnotA. e.g., ifA=red card,then Ais a black card (not a red card).

Mutually exclusive eventsare events that

cannot occur at the same time. Events haveno elementary events in common. e.g.,A=heart andB =club.

Mutually exclusive and exhaustiveevents area complete partition of the sample space. e.g.,

Suits (hearts, diamonds, clubs, spades)

Numbers (A, 2, 3, 4, 5, 6, 7, 8, 9, J, Q, K)Probability & Probability Distributions p. 11/61


12/61

Formal Defintion of ProbabilityProbabilityis a number assigned to each and

every member in the sample space. Denote by

P().

A probability function is a rule of correspondence

that associates with each eventAin the samplespaceSa numberP(A)such that

0P(A)1, for any eventA.

The sum of probabilities for all distinct events is 1.

IfAandB are mutually exclusive events, then

P(AorB) =P(A B) =P(A) + P(B)



13/61

ExampleLetA=number card (i.e., 210),B =face card(i.e., J, Q, K), andC=Ace.

Probabilities of events:

P(A) = 9(4)/52 = 36/52 =.6923

P(B) = 3(4)/52 = 12/52 =.2308P(C) = 1(4)/52 = 4/52 =.0769

P(A) + P(B) + P(C) = 1 P(A B) =P(A) + P(B) =.6923 + .2308 =

.9231 = 48/52.



14/61

Another Example Experiment: Randomly select a third grade

student from a Unit 4 public school in

Champaign county. Sample Space: All 3rd grade students at Unit

4 public schools.

Elementary Event: A characteristic of thechild. e.g., brown hair, age (in months),

weight, gender, the response very much toquestion How much do you like school?



15/61

Venn Diagram

A

B

S

C

A B

A C

Addition rules. . .



16/61

Addition Rules Rule 1: If 2 events,B &C, are

mutually exclusive (i.e., no overlap) then theprobability that one or both occur is

P(B orC) =P(B C) =P(B) + P(C) Rule 2: For any 2 events,A&B, the

probability that one or both occur is

P(AorB) =P(AB) =P(A)+P(B)P(AB)



17/61

Example: Teachers by Region

The population consists of all elementary and

secondary teachers in US in 1969.Level

Region Elementary Secondary

Northeast 273,687 224,013 517,700

North Central 314,614 265,848 580,462

South 240,028 183,180 423,208West 279,445 213,021 492,466

1,107,774 906,062 2,013,836



18/61

Example: Teachers by Region Elementary event (or sample point) is a

teacher. Event is any set of teachers. (e.g., region,

level, or combination).

Simple Experiment: Select 1 teacher atrandom,

P(elementary) =1, 107, 7742, 0138, 836

=.55

P(not elementary) =P(secondary) = 1 .55 =.45



19/61

Example: Addition RulesRule 1:Events are an elementary teacher from

the South & an elementary teacher from theWest,

P(elementary in S or W

) ==P(elementary, South) + P(elementary, West)

= 240, 028

2, 013, 836 +

279, 445

2, 013, 836=.26



20/61

Example: Addition Rules (continued)

Rule 2:Events are an elementary teacher and a

teacher from the South

P(elementary or from South) =

=P(elementary) + P(South)P(elementary and South)

=

1, 107, 774

2, 013, 836+

423, 208

2, 013, 836

240, 028

2, 013, 836=.64



21/61

Conditional Probability

Conditional Probabilityequals the probability

of an eventAgiven that we know that eventB

has occurred.

P(A|B) =P(A B)

P(B)

=P(A, B)

P(B)

Example: What is the probability that a

teacher is from the South given that he/she isan elementary school teacher?



22/61

Example: Answer

P(South|elementary) = P(elementary and South)P(elementary)

=

240, 028/2, 013, 836

1, 107, 774/2, 013, 836

= 240, 028

1, 107, 774= .217



23/61

Example (continued)

Note that

P(South) = 423, 2082, 013, 396

=.210

Knowing that a teacher is an elementaryschool teacher changes the chance that theteacher is also from the south,

P(South|elementary) = P(South)

.217 = .210



24/61

Bayes Theorem

P(A B) =P(A, B) =P(A|B)P(B)

P(A B) =P(A, B) =P(B|A)P(A)

Bayes Theorem:

P(A|B) =P(B|A)P(A)

P(B)

Example: Monty hall problem.

Door A Door B Door C



25/61

Monty Hall Problem

Start of Game: Probability of getting the big

prise (e.g, car)

P(A) =1

3

P(B) =1

3

P(C) =1

3 You pick doorA.

Monty opens doorB and gives you thechance to switch from doorAto doorC. Whatshould do you do?

Montys trying to not let you win. . .Probability & Probability Distributions p. 25/61


26/61

Monty Hall Problem (continued)

Choose the door for which has the larger

conditional probability, i.e., P(A|Monty openedB)

orP(C|Monty openedB).

Use Bayes Theorem. . . so we need

Conditional probabilities that Monty opens doorB

given the car is behindA, behindB and behindC.

Joint probabilities that Monty chooses doorB and

the car is behind doorA, doorB and doorC.

Unconditional probability that Monty chooses door

B.



28/61


(Unconditional) Probability that Monty opensdoorB:

P(BMonty) = P(BMonty, A) + P(BMonty, B) + P(BMonty, C)

= 1

6+ 0 +

1

3=

1

2

Apply Bayes Theorem. . .

P(A|BMonty) = P(A)

P(BMonty)

P(BMonty|A) =1/3

1/2

1

2

=1

3

P(C|BMonty) = P(C)

P(BMonty)P(BMonty|C) =

1/3

1/2 1 =

2

3



29/61


I got this example from: Gill, J. (2002).

Bayesian Methods for the Social and BehavioralSciences.Chapman & Hall.

Other sources on The Monty Hall Problem.

History.

Use today.



30/61

Independence

If the conditional and unconditional

probabilities are identical, then the two eventsareIndependent.

For Independent events,

P(A|B) =P(A) P(B|A) =P(B)

P(AandB) =P(A B) =P(A)P(B) =the multiplicative rule.



31/61

Conditional Independence (continued)

Conditional probabilities and Conditional

Independence: two very important concepts. Conditional probability and regression.

Conditional Independence: explainingdependency (e.g., classic example: Calgraduate admissions)

Demonstration: Toss penny and quarter andgive information about what happens.



32/61

Are Events Conditionally Independence

Physical considerations physically

unrelated events.. Deduced from observations.



33/61

Independent: Physical Considerations

Examples:

Toss a penny & a quarter:

P(penny=head & quarter=head) =

P(penny=head)P(quarter=head) = (.5)(.5)

= .25

Role two dice:

P(die1 = 5& die2 = 6) =

P(die1 = 5)P(die2 = 6) = (1/6)(1/6)

= 1/36 =.0278Probability & Probability Distributions p. 33/61


34/61

Independent: Physical Considerations

Examples:

Administer an test that measures attitude

toward gun control to 2 randomly drawnadults in the US population.

P(Score1= 50and Score2= 55)=P(Score1= 50)P(Score2= 55)



35/61

Independence: Deduction

Whether events are independent can sometimesbe deduced from observations, e.g., Mendals

experiments. Mendal postulated that existence of genes

that are recessive and dominant.

Experiment: Bred pure strains of yellow peas& green peas.

1st generation: Cross the yellow and greenpeas.

2nd generation: Cross plants from 1st

generation with each other and found. . .Probability & Probability Distributions p. 35/61


36/61

Mendals Experiments (continued)

Results: About75%yellow and About25%

green. Results were very regular and replicable (with

other traits and plants).

Part of explanation involves assumption ofindependence.



37/61

Mendals Experiments: Explanation

There exist genes which when paired up

control seed color according to rules:y/gyellow g/yyellowy/yyellow g/ggreen

1st generation: Pure yellow strain (y/y) couldonly give ay gene and pure green strain (g/g)could only give ag gene.

2nd generation: About 1/2 of parent plantscontribute ay, about 1/2 contribute ag, andpairing in random (independent).



38/61

Mendals Experiments: ExplanationMaternal

y g

y y/y y/g

Paternal (.25) (.25) (.50)

g g/y g/g

(.25) (.25) (.50)

(.50) (.50) (1.00)

Probability of each cell= (.50)(.50) =.25. . . this is the

independence part of the theory.

Probability of phenotype:

P(yellow pea) = P(y/y) + P(y/g) +P(g/y) =.75

P(green pea) = P(g/g) =.25Probability & Probability Distributions p. 38/61


39/61

Mendals Experiments: Explanation

Mendals theory is an example where anabstract probability theory is applied to

observed data. The postulated probability distribution of seed

color for 2nd generation



40/61

Basic Logic

Assumed some things to be true (e.g.,

Mendals theory). Make deductions about what should be true

in the long-run (e.g., 2nd generation: 75%

yellow and25%green). Its physically impossible to do all possible

experiments, so we do some (sample).

By chance the results will differ from whatshould be true; however, in the long-run, it

would be exactly equal/true.Probability & Probability Distributions p. 40/61


41/61

Probability Distributions

From Hayes:

Any statement of a function associating eachof a set of mutually exclusive and exhaustiveevents with its probability is a probability

distribution LetXrepresent a function that associates a

Real number with each and every elementaryevent in some sample spaceS. ThenX iscalled a random variable on the samplespaceS.



42/61

Random Variables

If random variable can only equal a finite

number of values, it is a discrete randomvariable.Probability distribution is known as a

probability mass function. If a random variable can equal an infinite (or

really really large) number of values, then it is

a continuous random variable.Probability distribution is know as aprobability density function.



43/61

Discrete Random Variables

From Mendals theory, assign event to realnumber (arbitary):

Y =

1 if yellow

0 if green

Probability Mass Function:



44/61

Lottery Spinner

Color Y P(Y)

Yellow 100 .10Blue 5 .20

Red 0 .50

Green 10 .10Tan 100 .10



45/61

Lottery Spinner



46/61

Lottery Spinner



47/61

Continuous Random Variables

When a numerical variable is continuous, itsprobability distribution is represented by a

curve known as a probability densityfunction or just p.d.f.

Denote a p.d.f byf(y).

P(x1Y x2) =area under curve.Probability=area under curve.

P(Y =y) = 0



48/61

Continuous Random Variable

The event is how many miles a randomlyselected graduate student attending UIUC is

from home.

What would c.d.f look like?Probability & Probability Distributions p. 48/61

C i i


49/61

Continuous Random Variable

Probability that a graduate student attendingUIUC is 2,000 or more miles from home

corresponds to the shaded area.


C i R d V i bl


50/61

Continuous Random Variables

The event is temperature outside the educationbuilding on January 27th.

P(22.0o) = 0

P(19.5o temperature21o) =black area.Probability & Probability Distributions p. 50/61

E l f d f


51/61

Examples of p.d.f.s

Wheres the mean, median and mode?


E l f d f


52/61

Examples of p.d.f.s

Wheres the mean, median and mode?


Ch t i ti f Di t ib ti


53/61

Characteristics of Distributions

Discrete or continuous

Shape Central tendency

Dispersion (variability)


E t d V l


54/61

Expected Value

If you played this game what would you expect towin or lose?

Color Y P(Y)Yellow 100 .10

Blue 5 .20

Red 0 .50Green 10 .10

Tan 100 .10

Y = E(Y)

= .1(100) + .2(5) + .5(0) + .1(10) + .1(100) = 0Probability & Probability Distributions p. 54/61

E t ti M


55/61

Expectations are Means For discrete random variable,

E[Y] =y n

i=1

yiP(yi)

For continuous variables,

E[Y] =y

yf(y)d(y)

Variance is the mean squared deviation,

2y =E[(y y)2] = E[y2 2yy+

2

y]

= E[y2] 2yE[y] +2

y]

= E[y2

] 2

y


Expectations are Means


56/61

Expectations are Means (continued)

Example: The variance of lottery spinner:

2 = E[(y y)2]

=5

i=1

(yi )2P(Y

i)

= .1(100 0)2+.2(5 0)2 + .5(0 0)2

.1(10 0)2

+ .1(100 0)2

= 2, 015

. . . So how much would you pay to take a spin?Probability & Probability Distributions p. 56/61

The Algebra of Expectations


57/61


Why? We dont have to deal with calculus & its

used alot in statistics. From Hayes Appendix B, Rule 1: Ifais a constant, then

E(a) =a

Rule 2: Ifais a constant real number andY isa random variable with expectationE(Y),then

E(aY) =aE(Y)




58/61


Rule 3: Ifais a constant real number andY isa random variable with expectationE(Y),

then

E(Y + a) =E(Y) + a

Rule 4: IfXandYare random variables withexpectationsE(X)andE(Y), respectively,then

E(X+ Y) =E(X) + E(Y)




59/61


Rule 5: Given a finite number of random

variables, the expectation of the sum of thosevariables is the sum of their individualexpectations, e.g.

E(X+ Y + Z) =E(X) + E(Y) + E(Z)

Variances:

Rule 6: Ifais a constant and ifY is a randomvariable with variance2y, then the random

variable(Y + a)has variance

2

y.Probability & Probability Distributions p. 59/61



60/61


Rule 7: Ifais a constant and ifY is a random

variable with variance2

y, then the randomvariable(aY)has variancea22y.

Rule 8: IfXandYare independent randomvariables with variances2x and

2y, then the

variance ofX+ Y is

2(x+y)=2x+

2y

What about variance of(X y)?2 2 2 Probability & Probability Distributions p. 60/61



61/61


Independence

Rule 9a: Given random variablesXandY

with expectationsE(X)andE(Y),respectively, thenXandY areindependent

if

E(XY) =E(X)E(Y)

Rule 9b: IfE(XY)=E(X)E(Y), thevariablesXandYare not independent.

. . . thats enough for now.o Probability & Probability Distributions p. 61/61

Probability ProbDist

Documents

Transcript of Probability ProbDist