Post on 13-May-2015
Hadley Wickham
Stat310Maximum likelihood
Sunday, 11 April 2010
1. Assessment
2. Feedback
3. Joint pdf
4. Maximum likelihood
Sunday, 11 April 2010
Assessment
• All grading now 100% up to date (as far as I know)
• Overall grade to date in owlspace (but doesn’t account for dropping lowest homework)
• Quizzes were going to be worth 10%, change to 5%?
Sunday, 11 April 2010
So far
• 2 / 2 tests * 10% = 20%
• 7 / 10 homeworks * 40% = 28%
• 3 / 5 quizzes * 5% = 3%
• Total: 51% of grade
Sunday, 11 April 2010
To come
• 1 final * 30% = 30%
• 3 / 10 homeworks * 40% = 12%
• 2 / 5 quizzes * 5% = 2%
• 5% TBA
• Total: 49% of grade
Sunday, 11 April 2010
Test
• Bad news: It was harder
• Good news: I’ve figured out why, so it won’t happen on the final
Sunday, 11 April 2010
T2
count
0
2
4
6
8
10
12
14
0.0 0.2 0.4 0.6 0.8 1.0
Sunday, 11 April 2010
Test 1
Test
2
0.0
0.2
0.4
0.6
0.8
1.0
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0.0 0.2 0.4 0.6 0.8 1.0
Worse
Better
Sunday, 11 April 2010
Overall
count
0
5
10
15
10 20 30 40 50
Sunday, 11 April 2010
Overall
count
0
5
10
15
10 20 30 40 50
ABCF
These are minimums described in the syllabus
Sunday, 11 April 2010
Options
• Do nothing.
• Add 3 points on to test. Distribute 5% evenly across all assessment.
• 1 hour take home exam worth 5%. 2-3 problems from the book.
• 1 extra homework worth 5%. 4-5 problems from the book.
Sunday, 11 April 2010
Homeworks• Due Thursday in class
• Out of the goodness of my heart I have been accepting late homeworks
• But it is getting excessive - I shouldn’t have to deal with 15 late homeworks a week
• Please turn in on time or I will start enforcing the late homework penalty.
Sunday, 11 April 2010
Feedback
Sunday, 11 April 2010
Feedback about me
Doing well: Lectures/teaching (13), engaging/interesting lectures (11), website (10), examples (10), homeworks (8), help sessions (6), pace (4), funny (3), being awesome (2)
Needs improvement: test too hard (too many to count), hard to study from ppt (7), more activities (5), less mistakes (5), too fast (4), homework session should be a tutorial (3)
Sunday, 11 April 2010
ChangesMy notes are scattered between slides, the board and my voice. Your notes should not be!
Will continue to try and find interesting examples and activities.
For final review session, will have voting system and I’ll re-cover popular topics on the board.
Sunday, 11 April 2010
You
Doing well
Needs improvement
Sunday, 11 April 2010
You
Doing well
Needs improvement
Marijuana?
Sunday, 11 April 2010
You
Doing well
Needs improvement
Probably read ahead, but
who does that anyways
Sunday, 11 April 2010
You
Doing well
Needs improvement
I’m enjoying the weather
Sunday, 11 April 2010
You
Doing well
Needs improvement my grade
Sunday, 11 April 2010
Why do we care about random
variables?
Sunday, 11 April 2010
Experiments
If we capture all the relevant information about an experiment, we can repeat virtually (either mathematically or computationally). This is usually easier and cheaper than doing the real experiment!
The mathematical abstraction we use to do this is the random variable.
Sunday, 11 April 2010
So
The purpose of a random variable is to describe (or at least approximate) the behaviour of an experiment. So:
X ~ SomeDist(some params)
means we have a single experiment whose behaviour is defined.
Sunday, 11 April 2010
Replications
X1 ~ SomeDist(some params)
X2 ~ SomeDist(some params)
Means we repeat the experiment twice - it’s the same distribution, which implies that the experiment is repeated under identical conditions.
f(x1, x2) is the bivariate pdf which allows us to figure out the probability of any event involving the two replicates
Sunday, 11 April 2010
Replicates
Xi ~ SomeDist(some params)
i = 1, 2, ..., n
Means we repeat the experiment n times.
f(x1, x2, ..., xn) is the joint pdf which allows us to figure out the probability of any event involving the n replicates
Sunday, 11 April 2010
Maximum likelihood
Sunday, 11 April 2010
Your turn
On Tuesday I was dismayed to find that if Xi ~ Binomial(n, p) then an estimator for p is
In fact, this estimator is basically correct, but there is a problem with my notation.
Can you spot where I went wrong? (everything you need is on this slide)
�ni Xi/n2
Sunday, 11 April 2010
Formal definition
The maximum likelihood estimator is a value of the parameter that maximises the likelihood function with respect to the parameter.
θ̂ML = maxθ∈Θ
l(θ;x1, x2, . . . , xn)
Sunday, 11 April 2010
StepsWrite out likelihood (=joint pdf)
Write out log-likelihood
(Discard constants)
Find maximum:
Differentiate and set to 0
(Check second derivative is negatice)
(Check end points)
Sunday, 11 April 2010
Maximum
• Derivative zero
• Derivative undefined
• At boundary points
Sunday, 11 April 2010
Your turn
Xi ~ Poisson(λ) i = 1,..., n
Use maximum likelihood to find an estimator for λ
Sunday, 11 April 2010
Invariance principle
One neat property of maximum likelihood estimators is invariance
Sunday, 11 April 2010
What else?
MLEs are:
Unbiased
Minimum variance
Have asymptotically normal distribution!
V ar(θ̂ML) =−1
E δ2
δθ2 l(X|θ)
Sunday, 11 April 2010
But
That math is too hard for this course :(
So we need some other ways to work out how much error our estimators have.
Sunday, 11 April 2010
Your turn
What is the variance of ?λ̂ML
Sunday, 11 April 2010
Your turn
I repeated an experiment defined by Poisson(λ) 10 times, and recorded the following results:
6 11 10 6 12 7 8 5 7 10
What is the MLE of λ?
What is the standard deviation of our estimate?
Sunday, 11 April 2010
Answer
Mean = 8.2
SD = 0.90
Can you create an interval around the estimate that ensures that the true value will be inside it 95% of the time?
(Use clt)
Sunday, 11 April 2010
Reading
6.1, 6.1.1
Sunday, 11 April 2010