21 Ml

Hadley Wickham

Stat310Maximum likelihood

Sunday, 11 April 2010

1. Assessment

2. Feedback

3. Joint pdf

4. Maximum likelihood


Assessment

• All grading now 100% up to date (as far as I know)

• Overall grade to date in owlspace (but doesn’t account for dropping lowest homework)

• Quizzes were going to be worth 10%, change to 5%?


So far

• 2 / 2 tests * 10% = 20%

• 7 / 10 homeworks * 40% = 28%

• 3 / 5 quizzes * 5% = 3%

• Total: 51% of grade


To come

• 1 final * 30% = 30%

• 3 / 10 homeworks * 40% = 12%

• 2 / 5 quizzes * 5% = 2%

• 5% TBA

• Total: 49% of grade


Test

• Bad news: It was harder

• Good news: I’ve figured out why, so it won’t happen on the final


T2

count

0

2

4

6

8

10

12

14

0.0 0.2 0.4 0.6 0.8 1.0


Test 1

Test

2

0.0

0.2

0.4

0.6

0.8

1.0

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

0.0 0.2 0.4 0.6 0.8 1.0

Worse

Better


Overall

count

0

5

10

15

10 20 30 40 50


Overall

count

0

5

10

15

10 20 30 40 50

ABCF

These are minimums described in the syllabus


Options

• Do nothing.

• Add 3 points on to test. Distribute 5% evenly across all assessment.

• 1 hour take home exam worth 5%. 2-3 problems from the book.

• 1 extra homework worth 5%. 4-5 problems from the book.


Homeworks• Due Thursday in class

• Out of the goodness of my heart I have been accepting late homeworks

• But it is getting excessive - I shouldn’t have to deal with 15 late homeworks a week

• Please turn in on time or I will start enforcing the late homework penalty.


Feedback


Feedback about me

Doing well: Lectures/teaching (13), engaging/interesting lectures (11), website (10), examples (10), homeworks (8), help sessions (6), pace (4), funny (3), being awesome (2)

Needs improvement: test too hard (too many to count), hard to study from ppt (7), more activities (5), less mistakes (5), too fast (4), homework session should be a tutorial (3)


ChangesMy notes are scattered between slides, the board and my voice. Your notes should not be!

Will continue to try and find interesting examples and activities.

For final review session, will have voting system and I’ll re-cover popular topics on the board.


You

Doing well

Needs improvement


You

Doing well

Needs improvement

Marijuana?


You

Doing well

Needs improvement

Probably read ahead, but

who does that anyways


You

Doing well

Needs improvement

I’m enjoying the weather


You

Doing well

Needs improvement my grade


Why do we care about random

variables?


Experiments

If we capture all the relevant information about an experiment, we can repeat virtually (either mathematically or computationally). This is usually easier and cheaper than doing the real experiment!

The mathematical abstraction we use to do this is the random variable.


So

The purpose of a random variable is to describe (or at least approximate) the behaviour of an experiment. So:

X ~ SomeDist(some params)

means we have a single experiment whose behaviour is defined.


Replications

X1 ~ SomeDist(some params)

X2 ~ SomeDist(some params)

Means we repeat the experiment twice - it’s the same distribution, which implies that the experiment is repeated under identical conditions.

f(x1, x2) is the bivariate pdf which allows us to figure out the probability of any event involving the two replicates


Replicates

Xi ~ SomeDist(some params)

i = 1, 2, ..., n

Means we repeat the experiment n times.

f(x1, x2, ..., xn) is the joint pdf which allows us to figure out the probability of any event involving the n replicates


Maximum likelihood


Your turn

On Tuesday I was dismayed to find that if Xi ~ Binomial(n, p) then an estimator for p is

In fact, this estimator is basically correct, but there is a problem with my notation.

Can you spot where I went wrong? (everything you need is on this slide)

�ni Xi/n2


Formal definition

The maximum likelihood estimator is a value of the parameter that maximises the likelihood function with respect to the parameter.

θ̂ML = maxθ∈Θ

l(θ;x1, x2, . . . , xn)


StepsWrite out likelihood (=joint pdf)

Write out log-likelihood

(Discard constants)

Find maximum:

Differentiate and set to 0

(Check second derivative is negatice)

(Check end points)


Maximum

• Derivative zero

• Derivative undefined

• At boundary points


Your turn

Xi ~ Poisson(λ) i = 1,..., n

Use maximum likelihood to find an estimator for λ


Invariance principle

One neat property of maximum likelihood estimators is invariance


What else?

MLEs are:

Unbiased

Minimum variance

Have asymptotically normal distribution!

V ar(θ̂ML) =−1

E δ2

δθ2 l(X|θ)


But

That math is too hard for this course :(

So we need some other ways to work out how much error our estimators have.


Your turn

What is the variance of ?λ̂ML


Your turn

I repeated an experiment defined by Poisson(λ) 10 times, and recorded the following results:

6 11 10 6 12 7 8 5 7 10

What is the MLE of λ?

What is the standard deviation of our estimate?


Answer

Mean = 8.2

SD = 0.90

Can you create an interval around the estimate that ensures that the true value will be inside it 95% of the time?

(Use clt)


Reading

6.1, 6.1.1


21 Ml

Documents

Transcript of 21 Ml