Lecture 02 EAILC
Transcript of Lecture 02 EAILC
Lecture 2: Real-Time Parameter Estimation
• Least Squares and Regression Models
• Estimating Parameters in Dynamical Systems
• Examples
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 1/25
Preliminary Comments
Real-time parameter estimation is one of methods of systemidentification
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 2/25
Preliminary Comments
Real-time parameter estimation is one of methods of systemidentification
The key elements of system identification are:
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 2/25
Preliminary Comments
Real-time parameter estimation is one of methods of systemidentification
The key elements of system identification are:• Selection of model structure (linear, nonlinear, linear in
parameters, . . . )
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 2/25
Preliminary Comments
Real-time parameter estimation is one of methods of systemidentification
The key elements of system identification are:• Selection of model structure (linear, nonlinear, linear in
parameters, . . . )• Design of experiments (input signal, sampling period, . . . )
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 2/25
Preliminary Comments
Real-time parameter estimation is one of methods of systemidentification
The key elements of system identification are:• Selection of model structure (linear, nonlinear, linear in
parameters, . . . )• Design of experiments (input signal, sampling period, . . . )• Parameter estimation (off-line, on-line, . . . )
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 2/25
Preliminary Comments
Real-time parameter estimation is one of methods of systemidentification
The key elements of system identification are:• Selection of model structure (linear, nonlinear, linear in
parameters, . . . )• Design of experiments (input signal, sampling period, . . . )• Parameter estimation (off-line, on-line, . . . )• Validation mechanism (depends on application)
All these step should be present
in Real-Time Parameter Estimation Algorithms!
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 2/25
Parameter Estimation: Problem Formulation
Suppose that a system (a model of experiment) is
y(i) = φ1(i)θ01 + φ2(i)θ
02 + · · ·+ φn(i)θ
0n
=[
φ1(i), φ2(i), . . . , φn(i)]
︸ ︷︷ ︸
φ(i)T ←− 1×n
θ01...θ0n
︸ ︷︷ ︸
θ0 ←− n×1
= φ(i)T θ0
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 3/25
Parameter Estimation: Problem Formulation
Suppose that a system (a model of experiment) is
y(i) = φ1(i)θ01 + φ2(i)θ
02 + · · ·+ φn(i)θ
0n
=[
φ1(i), φ2(i), . . . , φn(i)]
︸ ︷︷ ︸
φ(i)T ←− 1×n
θ01...θ0n
︸ ︷︷ ︸
θ0 ←− n×1
= φ(i)T θ0
Given the data{
[y(i), φ(i)]}N
i=1={
[y(1), φ(1)], . . . , [y(N), φ(N)]}
The task is to find (estimate) the n unknown values
θ0 =[
θ01, θ
02, . . . , θ
0n
]T
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 3/25
What kind of solution is expected?
Algorithm solving the estimation problem should be as follows.
• Estimates should be computed in a numerically efficientway (an analytical formula is preferable!)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 4/25
What kind of solution is expected?
Algorithm solving the estimation problem should be as follows.
• Estimates should be computed in a numerically efficientway (an analytical formula is preferable!)
• Computed estimates (the mean value) should be close tothe real estimated values (be unbiased), provided that themodel is correct.
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 4/25
What kind of solution is expected?
Algorithm solving the estimation problem should be as follows.
• Estimates should be computed in a numerically efficientway (an analytical formula is preferable!)
• Computed estimates (the mean value) should be close tothe real estimated values (be unbiased), provided that themodel is correct.
• Estimates should have small variances, preferablydecreasing with N →∞ (more data –> better estimate).
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 4/25
What kind of solution is expected?
Algorithm solving the estimation problem should be as follows.
• Estimates should be computed in a numerically efficientway (an analytical formula is preferable!)
• Computed estimates (the mean value) should be close tothe real estimated values (be unbiased), provided that themodel is correct.
• Estimates should have small variances, preferablydecreasing with N →∞ (more data –> better estimate).
• The algorithm, if possible, should have intuitive clearmotivation.
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 4/25
Least Squares Algorithm:
Consider the function (loss-function) to be minimized
VN(θ) =1
2
{
(y(1)− φ(1)Tθ)2 + · · ·+ (y(N)− φ(N)Tθ)2}
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 5/25
Least Squares Algorithm:
Consider the function (loss-function) to be minimized
VN(θ) =1
2
{
(y(1)− φ(1)Tθ︸ ︷︷ ︸
ε(1)
)2 + · · ·+ (y(N)− φ(N)Tθ︸ ︷︷ ︸
ε(N)
)2}
=1
2
{
ε(1)2 + ε(2)2 + · · ·+ ε(N)2}
=1
2ETE
Here
E =[
ε(1), ε(2), . . . , ε(N)]
T
= Y − Φ θ
and
Y =[
y(1), y(2), . . . , y(N)]
T
, Φ =
φ(1)T
φ(2)T
...φ(N)T
←− N×n
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 5/25
Theorem (Least Square Formula):
The function VN(θ) is minimal at θ = θ, which satisfies theequation
ΦTΦ θ = ΦT Y
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 6/25
Theorem (Least Square Formula):
The function VN(θ) is minimal at θ = θ, which satisfies theequation
ΦTΦ θ = ΦT Y
If det(
ΦTΦ)
6= 0, then
• the minimum of the function VN(θ) is unique,
• the minimum value is 12Y TY − 1
2Y TΦ
(
ΦTΦ)−1
ΦTY and
is attained at
θ =(
ΦTΦ)−1
ΦTY
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 6/25
Proof:
The loss-function can be written as
VN(θ) =1
2(Y − Φθ)T (Y − Φθ)
It reaches its minimal value at θ if
∇θVN(θ) =
[∂VN(θ)
∂θ1, . . . ,
∂VN(θ)
∂θn
]
= 0 with θ = θ
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 7/25
Proof:
The loss-function can be written as
VN(θ) =1
2(Y − Φθ)T (Y − Φθ)
It reaches its minimal value at θ if
∇θVN(θ) =
[∂VN(θ)
∂θ1, . . . ,
∂VN(θ)
∂θn
]
= 0 with θ = θ
This equation is
∇θVN(θ) =1
2·2·∇θ (−Φθ)T (Y − Φθ) = −ΦT (Y − Φθ) = 0
( ∇θ (θT a) = ∇θ (aT θ) = aT , ∇θ (θT Aθ) = θT (A + AT ) )
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 7/25
Proof:
The loss-function can be written as
VN(θ) =1
2(Y − Φθ)T (Y − Φθ)
It reaches its minimal value at θ if
∇θVN(θ) =
[∂VN(θ)
∂θ1, . . . ,
∂VN(θ)
∂θn
]
= 0 with θ = θ
This equation is
∇θVN(θ) =1
2·2·∇θ (−Φθ)T (Y − Φθ) = −ΦT (Y − Φθ) = 0
( ∇θ (θT a) = ∇θ (aT θ) = aT , ∇θ (θT Aθ) = θT (A + AT ) ) Hence
ΦT Φ θ = ΦT Y
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 7/25
Proof (con’d):
If the matrix ΦT Φ is invertible, then we solve can find thesolution θ of ΦT Φ θ = ΦT Y :
θ =(
ΦTΦ)−1
ΦTY
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 8/25
Proof (con’d):
If the matrix ΦT Φ is invertible, then we solve can find thesolution θ of ΦT Φ θ = ΦT Y :
θ =(
ΦTΦ)−1
ΦTY
VN(θ) =1
2(Y − Φθ)T (Y − Φθ) =
1
2Y TY−Y TΦθ+
1
2θTΦTΦθ
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 8/25
Proof (con’d):
If the matrix ΦT Φ is invertible, then we solve can find thesolution θ of ΦT Φ θ = ΦT Y :
θ =(
ΦTΦ)−1
ΦTY
VN(θ) =1
2(Y − Φθ)T (Y − Φθ) =
1
2Y TY−Y TΦθ+
1
2θTΦTΦθ
and since
Y TΦθ = Y TΦ(
ΦTΦ)−1 (
ΦTΦ)
θ = θT
(
ΦTΦ)
θ
by completing the square
VN(θ) =1
2Y TY −
1
2θT
(
ΦTΦ)
θ
︸ ︷︷ ︸
independent of θ
+1
2(θ − θ)T
(
ΦTΦ)
(θ − θ)︸ ︷︷ ︸
≥ 0, · · · > 0 for θ 6= θ
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 8/25
Comments:
The formula can be re-written in terms of y(i), φ(i) as
θ =(
ΦTΦ)−1
ΦTY =
(N∑
i=1
φ(i)φ(i)
)−1( N∑
i=1
φ(i)y(i)
)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 9/25
Comments:
The formula can be re-written in terms of y(i), φ(i) as
θ =(
ΦTΦ)−1
ΦTY = P (N)
(N∑
i=1
φ(i)y(i)
)
where P (N) =(
ΦTΦ)−1
=
(N∑
i=1
φ(i)φ(i)
)−1
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 9/25
Comments:
The formula can be re-written in terms of y(i), φ(i) as
θ =(
ΦTΦ)−1
ΦTY = P (N)
(N∑
i=1
φ(i)y(i)
)
where P (N) =(
ΦTΦ)−1
=
(N∑
i=1
φ(i)φ(i)
)−1
The condition det(
ΦTΦ)
6= 0 is called an excitation condition.
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 9/25
Comments:
The formula can be re-written in terms of y(i), φ(i) as
θ =(
ΦTΦ)−1
ΦTY = P (N)
(N∑
i=1
φ(i)y(i)
)
where P (N) =(
ΦTΦ)−1
=
(N∑
i=1
φ(i)φ(i)
)−1
The condition det(
ΦTΦ)
6= 0 is called an excitation condition.
To assign different weights for different time instants, consider
VN(θ) = 12
{
w1(y(1)−φ(1)Tθ)2+ · · ·+wN(y(N)−φ(N)Tθ)2}
= 12
{
w1ε(1)2+w2ε(2)
2+ · · ·+wNε(N)2}
=1
2ETWE
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 9/25
Comments:
The formula can be re-written in terms of y(i), φ(i) as
θ =(
ΦTΦ)−1
ΦTY = P (N)
(N∑
i=1
φ(i)y(i)
)
where P (N) =(
ΦTΦ)−1
=
(N∑
i=1
φ(i)φ(i)
)−1
The condition det(
ΦTΦ)
6= 0 is called an excitation condition.
To assign different weights for different time instants, consider
VN(θ) = 12
{
w1(y(1)−φ(1)Tθ)2+ · · ·+wN(y(N)−φ(N)Tθ)2}
= 12
{
w1ε(1)2+w2ε(2)
2+ · · ·+wNε(N)2}
=1
2ETWE
⇒ θ =(
ΦTWΦ)−1
ΦTWY
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 9/25
Exponential forgetting
The common way to assign the weights is Least Squares withExponential Forgetting:
VN(θ) =1
2
N∑
i=1
λN−i(
y(i)− φ(i)T θ)2
with 0 < λ < 1. What is the solution formula?
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 10/25
Stochastic Notation
Uncertainty is often convenient to represent stochastically.
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 11/25
Stochastic Notation
Uncertainty is often convenient to represent stochastically.• e is random if it is not known in advance and could take
values from the set {e1, . . . , ei, . . . } with probabilities pi.• The mean value or expected value is
E e =∑
i
pi ei, E is the expectation operator.
For a realization {ei}Ni=1 : E e ≈ 1N
∑Ni=1 ei
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 11/25
Stochastic Notation
Uncertainty is often convenient to represent stochastically.• e is random if it is not known in advance and could take
values from the set {e1, . . . , ei, . . . } with probabilities pi.• The mean value or expected value is
E e =∑
i
pi ei, E is the expectation operator.
For a realization {ei}Ni=1 : E e ≈ 1N
∑Ni=1 ei
• The variance or dispersion characterizes possibledeviations from the mean value:
σ2 = var(e) = E (e− E e)2.
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 11/25
Stochastic Notation
Uncertainty is often convenient to represent stochastically.• e is random if it is not known in advance and could take
values from the set {e1, . . . , ei, . . . } with probabilities pi.• The mean value or expected value is
E e =∑
i
pi ei, E is the expectation operator.
For a realization {ei}Ni=1 : E e ≈ 1N
∑Ni=1 ei
• The variance or dispersion characterizes possibledeviations from the mean value:
σ2 = var(e) = E (e− E e)2.
• Two random variables e and f are independent if
E (e f) = (E e) · (E f) or cov(e, f) = 0.
where cov(e, f) = E((e−E e)(f −E f)
)is covariance.
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 11/25
Statistical Properties of LS Estimate:
Assume that the model of the system is stochastic, i.e.
y(i) = φ(i)Tθ0 + e(i){
⇔ Y (N) = Φ(N)θ0 + E(N)}
.
Here θ0 is vector of true parameters, 0 ≤ i ≤ N ≤ +∞,
•{
e(i)}
={
e(0), e(1), . . .}
is a sequence of independent
equally distributed random variables with Ee(i) = 0,
•{
φ(i)}
is either a sequence of random variables
independent of e or deterministic.
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 12/25
Statistical Properties of LS Estimate:
Assume that the model of the system is stochastic, i.e.
y(i) = φ(i)Tθ0 + e(i){
⇔ Y (N) = Φ(N)θ0 + E(N)}
.
Here θ0 is vector of true parameters, 0 ≤ i ≤ N ≤ +∞,
•{
e(i)}
={
e(0), e(1), . . .}
is a sequence of independent
equally distributed random variables with Ee(i) = 0,
•{
φ(i)}
is either a sequence of random variables
independent of e or deterministic.
Pre-multiplying the model by(
Φ(N)TΦ(N))−1
Φ(N)T , gives(
Φ(N)TΦ(N))−1
Φ(N)T×Y (N) = θ̂
θ̂ =(
Φ(N)TΦ(N))−1
Φ(N)T ×(
Φ(N)θ0 + E(N))
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 12/25
Statistical Properties of LS Estimate:
Assume that the model of the system is stochastic, i.e.
y(i) = φ(i)Tθ0 + e(i){
⇔ Y (N) = Φ(N)θ0 + E(N)}
.
Here θ0 is vector of true parameters, 0 ≤ i ≤ N ≤ +∞,
•{
e(i)}
={
e(0), e(1), . . .}
is a sequence of independent
equally distributed random variables with Ee(i) = 0,
•{
φ(i)}
is either a sequence of random variables
independent of e or deterministic.
Pre-multiplying the model by(
Φ(N)TΦ(N))−1
Φ(N)T , gives
θ̂ = θ0 +(
Φ(N)TΦ(N))−1
Φ(N)T × E(N)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 12/25
Theorem: Suppose the data are generated by
y(i) = φ(i)Tθ0 + e(i){
⇔ Y (N) = Φ(N)θ0 + E(N)}
where{
e(i)}
={
e(0), e(1), . . .}
is a sequence of
independent random variables with zero mean and variance σ2.
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 13/25
Theorem: Suppose the data are generated by
y(i) = φ(i)Tθ0 + e(i){
⇔ Y (N) = Φ(N)θ0 + E(N)}
where{
e(i)}
={
e(0), e(1), . . .}
is a sequence of
independent random variables with zero mean and variance σ2.
Consider the least square estimate
θ̂ =(
Φ(N)TΦ(N))−1
Φ(N)TY (N).
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 13/25
Theorem: Suppose the data are generated by
y(i) = φ(i)Tθ0 + e(i){
⇔ Y (N) = Φ(N)θ0 + E(N)}
where{
e(i)}
={
e(0), e(1), . . .}
is a sequence of
independent random variables with zero mean and variance σ2.
Consider the least square estimate
θ̂ =(
Φ(N)TΦ(N))−1
Φ(N)TY (N).
If detΦ(N)TΦ(N) 6= 0, then
• E θ̂ = θ0, i.e. the estimate is unbiased;• the covariance of the estimate is
cov θ̂ = E(
θ̂ − θ0) (
θ̂ − θ0)
T
= σ2(
Φ(N)TΦ(N))−1
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 13/25
Regression Models for FIR
FIR (Finite Impulse Response) or MA (Moving Average Model):
y(t) = b1 u(t− 1) + b2 u(t− 2) + · · ·+ bn u(t− n)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 14/25
Regression Models for FIR
FIR (Finite Impulse Response) or MA (Moving Average Model):
y(t) = b1 u(t− 1) + b2 u(t− 2) + · · ·+ bn u(t− n)
y(t) = [u(t− 1), u(t− 2), . . . , u(t− n)]︸ ︷︷ ︸
=φ(t−1)T
b1
b2...bn
︸ ︷︷ ︸
=θ
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 14/25
Regression Models for FIR
FIR (Finite Impulse Response) or MA (Moving Average Model):
y(t) = b1 u(t− 1) + b2 u(t− 2) + · · ·+ bn u(t− n)
y(t) = [u(t− 1), u(t− 2), . . . , u(t− n)]︸ ︷︷ ︸
=φ(t−1)T
b1
b2...bn
︸ ︷︷ ︸
=θ
Note the change of notation for the typical case when the RHSdoes no have the term b0 u(t): φ(i) −→ φ(t− 1) to indicatedependence of date on input signals up to t− 1th.
However, we keep: Y (N) = Φ(N) θ
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 14/25
Regression Models for ARMA
IIR (Infite Impulse Response) or ARMA (Autoregressive MovingAverage Model):
(qn + a1 q
n−1 + · · ·+ an
)
︸ ︷︷ ︸
A(q)
y(t) =(b1 q
m−1 + b2 qm−2 + · · ·+ bm
)
︸ ︷︷ ︸
B(q)
u(t)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 15/25
Regression Models for ARMA
IIR (Infite Impulse Response) or ARMA (Autoregressive MovingAverage Model):
(qn + a1 q
n−1 + · · ·+ an
)
︸ ︷︷ ︸
A(q)
y(t) =(b1 q
m−1 + b2 qm−2 + · · ·+ bm
)
︸ ︷︷ ︸
B(q)
u(t)
q is the forward shift operator:
q u(t) = u(t + 1), q−1 u(t) = u(t− 1)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 15/25
Regression Models for ARMA
IIR (Infite Impulse Response) or ARMA (Autoregressive MovingAverage Model):
(qn + a1 q
n−1 + · · ·+ an
)
︸ ︷︷ ︸
A(q)
y(t) =(b1 q
m−1 + b2 qm−2 + · · ·+ bm
)
︸ ︷︷ ︸
B(q)
u(t)
q is the forward shift operator:
q u(t) = u(t + 1), q−1 u(t) = u(t− 1)
y(t) = [−y(t− 1), . . . ,−y(t− n), u(t− n + m− 1), . . . , u(t− n)]︸ ︷︷ ︸
=φ(i−1)T
a1
...an
b1...
bm
︸ ︷︷ ︸
=θ
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 15/25
Problem 2.2
Consider the FIR model
y(t) = b0u(t) + b1u(t− 1) + e(t), t = 1, 2, 3, . . . , N
where{e(t)
}is a sequence of independent normalN (0, σ)
random variables
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 16/25
Problem 2.2
Consider the FIR model
y(t) = b0u(t) + b1u(t− 1) + e(t), t = 1, 2, 3, . . . , N
where{e(t)
}is a sequence of independent normalN (0, σ)
random variables
• Determine LS estimates for b0, b1 when u(t) is a step.Analyze the covariance of the estimate, when N →∞
• Make the same investigation when u(t) is white noise withunit variance.
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 16/25
Solution (Problem 2.2):
For the model
y(t) = b0u(t) + b1u(t− 1) + e(t)
the regression form is readily seen
y(t) =[u(t), u(t− 1)
]
[
b0
b1
]
+ e(t) = φ(t)Tθ + e(t)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 17/25
Solution (Problem 2.2):
For the model
y(t) = b0u(t) + b1u(t− 1) + e(t)
the regression form is readily seen
y(t) =[u(t), u(t− 1)
]
[
b0
b1
]
+ e(t) = φ(t)Tθ + e(t)
Then LS estimate is
θ̂ =(
Φ(N)TΦ(N))−1
Φ(N)TY (N)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 17/25
Solution (Problem 2.2):
For the model
y(t) = b0u(t) + b1u(t− 1) + e(t)
the regression form is readily seen
y(t) =[u(t), u(t− 1)
]
[
b0
b1
]
+ e(t) = φ(t)Tθ + e(t)
Then LS estimate is
θ̂ =
u(1) u(0)
u(2) u(1)
...
u(N) u(N−1)
T
u(1) u(0)
u(2) u(1)
...
u(N) u(N − 1)
−1
u(1) u(0)
u(2) u(1)
...
u(N) u(N−1)
T
y(1)
y(2)
...
y(N)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 17/25
Solution (Con’d):
The formula for an arbitrary input signal u(t)
θ̂ =
u(1) u(0)
u(2) u(1)
...
u(N) u(N−1)
T
u(1) u(0)
u(2) u(1)
...
u(N) u(N − 1)
−1
u(1) u(0)
u(2) u(1)
...
u(N) u(N−1)
T
y(1)
y(2)
...
y(N)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 18/25
Solution (Con’d):
The formula for an arbitrary input signal u(t)
θ̂ =
u(1) u(0)
u(2) u(1)
...
u(N) u(N−1)
T
u(1) u(0)
u(2) u(1)
...
u(N) u(N − 1)
−1
u(1) u(0)
u(2) u(1)
...
u(N) u(N−1)
T
y(1)
y(2)
...
y(N)
becomes
θ̂ =
∑Nt=1 u(t)
2∑N
t=1 u(t)u(t− 1)∑N
t=1 u(t)u(t− 1)∑N
t=1 u(t− 1)2
−1
×
×
∑Nt=1 u(t)y(t)
∑Nt=1 u(t− 1)y(t)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 18/25
Solution (Con’d):
Suppose that u(t) is a unit step applied at t = 1
u(t) =
{
1, t = 1, 2, . . .
0, t = 0, −1, −2, −3, . . .
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25
Solution (Con’d):
Suppose that u(t) is a unit step applied at t = 1
u(t) =
{
1, t = 1, 2, . . .
0, t = 0, −1, −2, −3, . . .
Substituting this signal into the general formula, we obtain
θ̂ =
∑Nt=1 u(t)
2∑N
t=1 u(t)u(t− 1)∑N
t=1 u(t)u(t− 1)∑N
t=1 u(t− 1)2
−1
×
×
∑Nt=1 u(t)y(t)
∑Nt=1 u(t− 1)y(t)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25
Solution (Con’d):
Suppose that u(t) is a unit step applied at t = 1
u(t) =
{
1, t = 1, 2, . . .
0, t = 0, −1, −2, −3, . . .
Substituting this signal into the general formula, we obtain
θ̂ =
N
∑Nt=1 u(t)u(t− 1)
∑Nt=1 u(t)u(t− 1)
∑Nt=1 u(t− 1)2
−1
×
×
∑Nt=1 u(t)y(t)
∑Nt=1 u(t− 1)y(t)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25
Solution (Con’d):
Suppose that u(t) is a unit step applied at t = 1
u(t) =
{
1, t = 1, 2, . . .
0, t = 0, −1, −2, −3, . . .
Substituting this signal into the general formula, we obtain
θ̂ =
N N − 1
N − 1∑N
t=1 u(t− 1)2
−1
×
×
∑Nt=1 u(t)y(t)
∑Nt=1 u(t− 1)y(t)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25
Solution (Con’d):
Suppose that u(t) is a unit step applied at t = 1
u(t) =
{
1, t = 1, 2, . . .
0, t = 0, −1, −2, −3, . . .
Substituting this signal into the general formula, we obtain
θ̂ =
N N − 1
N − 1 N − 1
−1
×
×
∑Nt=1 u(t)y(t)
∑Nt=1 u(t− 1)y(t)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25
Solution (Con’d):
Suppose that u(t) is a unit step applied at t = 1
u(t) =
{
1, t = 1, 2, . . .
0, t = 0, −1, −2, −3, . . .
Substituting this signal into the general formula, we obtain
θ̂ =
N N − 1
N − 1 N − 1
−1
×
×
∑Nt=1 y(t)
∑Nt=2 y(t)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25
Solution (Con’d):
Suppose that u(t) is a unit step applied at t = 1
u(t) =
{
1, t = 1, 2, . . .
0, t = 0, −1, −2, −3, . . .
Substituting this signal into the general formula, we obtain
θ̂ =1
N(N − 1)− (N − 1)2
N − 1 −(N − 1)
−(N − 1) N
×
×
∑Nt=1 y(t)
∑Nt=2 y(t)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25
Solution (Con’d):
Suppose that u(t) is a unit step applied at t = 1
u(t) =
{
1, t = 1, 2, . . .
0, t = 0, −1, −2, −3, . . .
Substituting this signal into the general formula, we obtain
θ̂ =
1 −1
−1N
N − 1
×
×
∑Nt=1 y(t)
∑Nt=2 y(t)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25
Solution (Con’d):
Suppose that u(t) is a unit step applied at t = 1
u(t) =
{
1, t = 1, 2, . . .
0, t = 0, −1, −2, −3, . . .
Substituting this signal into the general formula, we obtain
θ̂ =
∑Nt=1 y(t)−
∑Nt=2 y(t)
−∑N
t=1 y(t) +N
N − 1
N∑
t=2
y(t)
=
y(1)
1
N − 1
N∑
t=1
y(t)− y(1)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25
Solution (Con’d):
How to compute the covariance of θ? Consider
θ − θ0 =
y(1)
1N−1
∑Nt=1 y(t)− y(1)
−
b0
b1
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 20/25
Solution (Con’d):
How to compute the covariance of θ? Consider
θ − θ0 =
y(1)
1
N − 1
N∑
t=1
y(t)− y(1)
−
b0
b1
=
{
b0 · 1 + b1 · 0 + e(1)}
− b0
N∑
t=2
{
b0 · 1 + b1 · 1 + e(t)}
+ y(1)
N − 1− y(1) − b1
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 20/25
Solution (Con’d):
How to compute the covariance of θ? Consider
θ − θ0 =
y(1)
1
N − 1
N∑
t=1
y(t)− y(1)
−
b0
b1
=
e(1)
1
N − 1
N∑
t=2
e(t)− e(1)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 20/25
Solution (Con’d):
The estimation error is
θ − θ0 =
e(1)
1
N − 1
N∑
t=2
e(t)− e(1)
The covariance E(θ − θ0)(θ − θ0)T of estimate is then
cov(θ) =
Ee(1)2 E{
∑
N
t=2e(t)
N−1− e(1)
}
e(1)
E{
∑
N
t=2e(t)
N−1− e(1)
}
e(1) E{
∑
N
t=2e(t)
N−1− e(1)
}2
=
σ2 −σ2
−σ2 σ2(
N−1(N−1)2
− 1)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 20/25
Solution (Con’d):
The estimation error is
θ − θ0 =
e(1)
1
N − 1
N∑
t=2
e(t)− e(1)
The covariance E(θ − θ0)(θ − θ0)T of estimate is then
cov(θ) =
Ee(1)2 E{
∑
N
t=2e(t)
N−1− e(1)
}
e(1)
E{
∑
N
t=2e(t)
N−1− e(1)
}
e(1) E{
∑
N
t=2e(t)
N−1− e(1)
}2
=
σ2 −σ2
−σ2 σ2 NN−1
= σ2
1 −1
−1 NN−1
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 20/25
Solution (Con’d):To conclude with the unit step input signal LS estimate is
θ̂ =
b0
b1
=
y(1)
1
N − 1
N∑
t=1
y(t)− y(1)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 21/25
Solution (Con’d):To conclude with the unit step input signal LS estimate is
θ̂ =
b0
b1
=
y(1)
1
N − 1
N∑
t=1
y(t)− y(1)
The covariance of such estimate looks as
E(θ − θ0)(θ − θ0)T = σ2
1 −1
−1 NN−1
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 21/25
Solution (Con’d):To conclude with the unit step input signal LS estimate is
θ̂ =
b0
b1
=
y(1)
1
N − 1
N∑
t=1
y(t)− y(1)
The covariance of such estimate looks as
E(θ − θ0)(θ − θ0)T = σ2
1 −1
−1 NN−1
As a number of measured data increases N →∞
E(θ − θ0)(θ − θ0)T →
1 −1
−1 1
and does NOT IMPROVE!
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 21/25
Solution (Problem 2.2):
Consider now the model
y(t) = b0u(t) + b1u(t− 1) + e(t)
when the input signal is a white noise with unit variance
Eu(t)2 = 1, Eu(t)u(s) = 0 if t 6= s
and when u and e are independent
Eu(t)e(s) = 0 ∀ t, s
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 22/25
Solution (Problem 2.2):
Consider now the model
y(t) = b0u(t) + b1u(t− 1) + e(t)
when the input signal is a white noise with unit variance
Eu(t)2 = 1, Eu(t)u(s) = 0 if t 6= s
and when u and e are independent
Eu(t)e(s) = 0 ∀ t, s
Such assumptions imply that
Ey(t)u(t) = b0, Ey(t)u(t− 1) = b1
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 22/25
Solution (Problem 2.2):
The LS estimate becomes dependent on a realization of thestochastic variable u(t); we need to consider its mean value
Eθ̂ = E
∑Nt=1 u(t)
2∑N
t=2 u(t)u(t− 1)∑N
t=2 u(t)u(t− 1)∑N
t=2 u(t− 1)2
−1
×
×
∑Nt=1 u(t)y(t)
∑Nt=2 u(t− 1)y(t)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 23/25
Solution (Problem 2.2):
The LS estimate becomes dependent on a realization of thestochastic variable u(t); we need to consider its mean value
Eθ̂ = E
N(
1
N
∑
N
t=1u(t)2
)
(N − 1)(
1
N−1
∑
N
t=2u(t)u(t− 1)
)
(N − 1)(
1
N−1
∑
N
t=2u(t)u(t− 1)
)
(N − 1)(
1
N−1
∑
N
t=2u(t− 1)2
)
−1
×
×
N(
1
N
∑
N
t=1u(t)y(t)
)
(N − 1)(
1
N−1
∑
N
t=2u(t− 1)y(t)
)
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 23/25
Solution (Problem 2.2):
The LS estimate becomes dependent on a realization of thestochastic variable u(t); we need to consider its mean value
Eθ̂ ≈
N Eu(t)2 (N − 1)Eu(t)u(t− 1)
(N − 1)Eu(t)u(t− 1) (N − 1)Eu(t− 1)2
−1
×
×
N Eu(t)y(t)
(N − 1)Eu(t− 1)y(t)
=
N 0
0 (N − 1)
−1
Nb0
(N − 1)b1
=
b0
b1
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 23/25
Solution (Problem 2.2):
To compute the covariance, use the formula in Theorem
cov(θ − θ0) = σ2E (Φ(N)TΦ(N))−1
= σ2
N 0
0 (N − 1)
−1
= σ
1N
0
0 1N−1
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 24/25
Solution (Problem 2.2):
To compute the covariance, use the formula in Theorem
cov(θ − θ0) = σ2E (Φ(N)TΦ(N))−1
= σ2
N 0
0 (N − 1)
−1
= σ
1N
0
0 1N−1
It converges to zero!
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 24/25
Next Lecture / Assignments:
• Next lecture (April 13, 10:00-12:00, in A206Tekn): RecursiveLeast Squares, modifications, continuous-time systems.
c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 25/25