Post on 06-Apr-2022
EC 831: Empirical Methods in Macroeconomics
Aeimit Lakdawala
Michigan State University
Linear State Space Models
Observation Equation
yt = Axt +Hξt + wt
yt ,wt are n x 1,ξt is r x 1 andxt is k x 1
State Equationξt+1 = F ξt + vt+1(
vtwt
)∼ iiN
([00
],
[Q 00 R
])Observed variables: yt ,xtUnobserved variables: ξt ,vt and wt
Consider an AR(1)yt = ρyt−1 + εt
Define ξt = yt , vt = εt and yt = yt , wt = 0, xt = 0 andF = ρ, H = 1 and A = 0
Observation Equation:
yt = Axt +Hξt + wt
yt = yt
State Equation
ξt+1 = F ξt + vt+1
yt+1 = ρyt + εt+1
Consider an AR(1)yt = ρyt−1 + εt
Define ξt = yt , vt = εt and yt = yt , wt = 0, xt = 0 andF = ρ, H = 1 and A = 0
Observation Equation:
yt = Axt +Hξt + wt
yt = yt
State Equation
ξt+1 = F ξt + vt+1
yt+1 = ρyt + εt+1
Consider a MA(1):
yt = µ + ut + θut−1
Define ξt = (ut , ut−1), yt = yt , xt = 1, wt = 0,vt = [ut , 0]′
Observation Equation:
yt = Axt +Hξt + wt
yt = µ + [ 1 θ ]
(utut−1
)State Equation
ξt+1 = F ξt + vt+1(ut+1
ut
)=
(0 01 0
)(utut−1
)+
(ut+1
0
)
Generally any ARMA(p,q) model can be put in state space form.
Solutions to linearized DSGE models can also be put in state spaceform.
Kalman Filter
The goal is to find the distribution of ξt given all informationavailable upto time t, Ωt = yt , yt−1, ..., y1, xt , xt−1, ..., x1
p(ξt |Ωt)
Assume ξ0 ∼ N(ξ0|0,P0|0)
• ξ0|0: Best guess of ξ0• P0|0: Uncertainty about this guess
With normal errors and linear structure we get
ξt |Ωt ∼ N(ξt|t ,Pt|t)
Goal is to find ξt|t and Pt|t
Kalman Filter
The goal is to find the distribution of ξt given all informationavailable upto time t, Ωt = yt , yt−1, ..., y1, xt , xt−1, ..., x1
p(ξt |Ωt)
Assume ξ0 ∼ N(ξ0|0,P0|0)
• ξ0|0: Best guess of ξ0• P0|0: Uncertainty about this guess
With normal errors and linear structure we get
ξt |Ωt ∼ N(ξt|t ,Pt|t)
Goal is to find ξt|t and Pt|t
Kalman Filter
The goal is to find the distribution of ξt given all informationavailable upto time t, Ωt = yt , yt−1, ..., y1, xt , xt−1, ..., x1
p(ξt |Ωt)
Assume ξ0 ∼ N(ξ0|0,P0|0)
• ξ0|0: Best guess of ξ0• P0|0: Uncertainty about this guess
With normal errors and linear structure we get
ξt |Ωt ∼ N(ξt|t ,Pt|t)
Goal is to find ξt|t and Pt|t
Kalman Filter
Two main steps:
1 Predict: Using all information upto time t − 1 we want toobtain an optimal forecast of ξt , let’s call it ξt|t−1.
2 Update: Once yt is realized, we update our forecast, call it ξt|t
Intuition for Kalman Filter
Assume z1 and z2 are jointly normal.[z1z2
]∼ N
([µ1
µ2
],
[Ω11 Ω12
Ω21 Ω22
])Suppose now you observe z1.Now what is the conditional distribution of z2?
z2|z1 ∼ N(mz2 ,Vz2)
mz2 = µ2 + Ω21Ω−111 [µ1 − z1]
Vz2 = Ω22 −Ω21Ω−111 Ω12
Intuition for Kalman Filter
Assume z1 and z2 are jointly normal.[z1z2
]∼ N
([µ1
µ2
],
[Ω11 Ω12
Ω21 Ω22
])Suppose now you observe z1.Now what is the conditional distribution of z2?
z2|z1 ∼ N(mz2 ,Vz2)
mz2 = µ2 + Ω21Ω−111 [µ1 − z1]
Vz2 = Ω22 −Ω21Ω−111 Ω12
1st Predict step
ξ1|Ω0 ∼ N(ξ1|0,P1|0)
ξ1|0 = F ξ0|0
P1|0 = FP0|0F′ +Q
1st Update step
[y1|x1, Ω0
ξ1|x1, Ω0
]∼ N
([µ1
µ2
],
[Ω11 Ω12
Ω21 Ω22
])where
µ1 = Ax1 +Hξ1|0µ2 = ξ1|0
Ω11 = HP1|0H′ + R
Ω21 = P1|0H′
Ω22 = P1|0
1st Update step
ξ1|y1, x1, Ω0 = ξ1|Ω1 ∼ N(ξ1|1,P1|1)
ξ1|1 = ξ1|0 +Kη1|0P1|1 = P1|0 −KHP1|0
where
K = P1|0H′(HP1|0H
′ + R)−1
η1|0 = y1 − Ax1 −Hξ1|0
Kalman Filter Recursions
Predict:
ξt|t−1 = F ξt−1|t−1
Pt|t−1 = FPt−1|t−1F′ +Q
Update:
ξt|t = ξt|t−1 +Kηt|t−1Pt|t = Pt|t−1 −KHPt|t−1
where
ηt|t−1 = yt − Axt −Hξt|t−1
ft|t−1 = HPt|t−1H′ + R
K = Pt|t−1H′f −1t|t−1
How do we specify ξ0|0 and P0|0?
ξt+1 = F ξt + vt
If this is stationary, i.e. eigenvalues of F are inside the unit circle,then
ξ0|0 = E (ξ0) = 0
P0|0 = E (ξ0ξ ′0)
vec(P0|0) = (I − F ⊗ F )−1vec(Q)
If not, set diagonal elements of P0|0 to large numbers
Bayesian Perspective
ξ0 ∼ N(ξ0|0,P0|0) is the prior.
ξt|t is the posterior Bayesian expectation of ξt |Ωt for given valuesof F,Q,A,H,R
So how do we estimate the parameters?
Parameter Estimation from classical perspective
What is the distribution of yt |Ωt−1, xt , θ where θ contains all theparameters.
yt |Ωt−1, xt , θ ∼ N(yt|t−1, ft|t−1)
where
yt|t−1 = Axt +Hξt|t−1
ft|t−1 = HPt|t−1H′ + R
Likelihood (based on ”Prediction Error Decomposition”)
lnL = −1
2
T
∑t=1
ln(2πft|t−1)−1
2
T
∑t=1
η′t|t−1f−1t|t−1ηt|t−1
Parameter Estimation from classical perspective
lnL = −1
2
T
∑t=1
ln(2πft|t−1)−1
2
T
∑t=1
η′t|t−1f−1t|t−1ηt|t−1
Note ft|t−1 and ηt|t−1 are functions of θ
Maximize the log-likelihood w.r.t θ.
Note ξt|t is our best guess given all data upto time t.
But as econometricians we observe T ≥ t
• Can we improve our best guess using all available data?
Yes!
We can use ”smoothed” inference.
ξt |ΩT ∼ N(ξt|T ,Pt|T )
Same idea used as Kalman filter derivations.
Note ξt|t is our best guess given all data upto time t.
But as econometricians we observe T ≥ t
• Can we improve our best guess using all available data?
Yes!
We can use ”smoothed” inference.
ξt |ΩT ∼ N(ξt|T ,Pt|T )
Same idea used as Kalman filter derivations.
Smoothing
ξt|T = ξt|t + Jt(ξt+1|T − ξt+1|t
)Pt|T = Pt|t + Jt(Pt+1|T − Pt+1|t)J
′t
Jt = Pt|tF′P−1
t+1|t
For smoothed inference.
1 Filter forward to get ξt|t and Pt|t2 Smooth backward using above formulae to get ξt|T and Pt|T
What if Pt+1|t is singular?
See Durbin & Koopman (2002)Time Series Analysis by State Space Methods
• Alternative formulation of Kalman filter recursion in Ch 4.
• Does not require inversion of Pt+1|t for smoothing
• Their method is also computationally more efficient