EE 505 Sums of RV’s and Long-Term Averages The Law of Large Numbers
19
ECE 5345 Sums of RV’s and Long-Term Averages The Law of Large Numbers
description
EE 505 Sums of RV’s and Long-Term Averages The Law of Large Numbers. Sum of RV’s. Sum Mean of the sum If i.i.d. Sum of RV’s. Variance. Sum of RV’s. Variance: Special case: independent RV’s The Kronecker delta is Thus…. Sum of RV’s. For i.i.d. RV’s, if The mean is… The variance is…. - PowerPoint PPT Presentation
Transcript of EE 505 Sums of RV’s and Long-Term Averages The Law of Large Numbers
ECE 5345Sums of RV’s and Long-Term Averages The Law of Large Numbers
Sum of RV’s
Sum
Mean of the sum
If i.i.d.
n
kkXS
1
n
kkXS
1
XnS
Sum of RV’s
Variance
n
kkXS
1
XX
XXXXE
XXXXE
XXE
SSES
k
n
k
n
kk
n
k
n
n
kkk
n
n
k
n
kkk
,cov
]var[
11
11
11
2
1 1
2
Sum of RV’s
Variance: Special case: independent RV’s
The Kronecker delta is
Thus…
n
kkXS
1
2
1k
n
k]Svar[
]k[)X,Xcov( kk 2
00
01
k;
k;]k[
Sum of RV’s
For i.i.d. RV’s, if
The mean is…
The variance is…
n
kkXS
1
2Xn]Svar[
XnS
Sum of RV’s: The Characteristic Function
For i.i.d. RV’s, if
Then…
And…
Note: can find the same result for mean and variance by differentiation and substitution.
n
kkXS
1
)()( nXS
)(n)( XS
Recall: for i.i.d. RV’s
Then…
• S is Gaussian if the Xk’s are Gaussian.
• S is Poisson if the Xk’s are Poisson.
• S is Binomial if the Xk’s are Binomial.
• S is Gamma if the Xk’s are Gamma.
• S is Cauchy if the Xk’s are Cauchy.
• S is Negative Binomial if the Xk’s are Negative Binomial
The Average (Sample Mean)
Sn
Xn
MAn
kkn
11
1
n
]e[E
]e[E)(
S
n/Sj
AjA
Then…
The Average
n
n)(
nX
SA
•If samples are i.i.d…
•Thus…
)()( nXS
The Average of i.i.d. RV’s
n)( n
XA
•The average…
•If samples are i.i.d…
•And…
nn)( XA
n
kkX
nA
1
1
Mean and Variance of the Average of i.i.d. RV’s
•Thus…
•Or…
•As it should be!
nnd
dn)(
d
d 'XXA
Xj)(Aj 'X
'A 00
XA
Mean and Variance of the Average of i.i.d. RV’s
•Thus…
•Or…
•Makes Sense!
nnnd
dn)(
d
d "XXA
1
2
2
2
2
)Xvar(nn
)()Avar( "X
"A
10
10
)Xvar(n
)Avar(1
Mean and Variance of the Average of i.i.d. RV’s
pdf’s…
X
A A for bigger n
xX
Apply Chebyshev to Average of i.i.d. RV’s
Chebyshev…
Since…
And…
Thus…
2
1
)Avar(|AA|Pr
XA
)Xvar(n
)Avar(1
2
1
n
)Xvar(|XA|Pr
The Weak Law of Large Numbers
This gives, in the limit, the “weak law of large numbers”
The average, in this sense, approaches the mean.
1
|XA|Prlimn
2
1
n
)Xvar(|XA|Pr
The Strong Law of Large Numbers
This is a stronger statement:
1
XAlimPr
n
Sloppy Confidence Intervals on Bernoulli
Trials
Unknown p parameter in Bernoulli trial.
E(X)=E(A)=p, var(X)=p(1-p) ¼
The weak law of large numbers gives:
22 4
111
nn
)Xvar(|pA|Pr
Sloppy Confidence Intervals on Bernoulli
Trials (cont)
24
11
n|pA|Pr
Assume we want to be 95% or more sure that the result is within 1% of being correct. How many samples do we need to take to assure this?
Set
9504
11
2.
n
010. Solving gives n=50,000.
Chebyshev gives loose bounds. There are
better ways to do this.
Sample Variance
The average (or sample mean) is used to estimate the variance.
n
kkX
nAX
1
1
The sample variance is used to estimate the variance.
2
11
1
n
kk AX
nV)Xvar(
Note: A and V can be shown to be independent RV’s.
