Lecture 02 EAILC

78
Lecture 2: Real-Time Parameter Estimation Least Squares and Regression Models Estimating Parameters in Dynamical Systems Examples c Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 1/25

Transcript of Lecture 02 EAILC

Lecture 2: Real-Time Parameter Estimation

• Least Squares and Regression Models

• Estimating Parameters in Dynamical Systems

• Examples

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 1/25

Preliminary Comments

Real-time parameter estimation is one of methods of systemidentification

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 2/25

Preliminary Comments

Real-time parameter estimation is one of methods of systemidentification

The key elements of system identification are:

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 2/25

Preliminary Comments

Real-time parameter estimation is one of methods of systemidentification

The key elements of system identification are:• Selection of model structure (linear, nonlinear, linear in

parameters, . . . )

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 2/25

Preliminary Comments

Real-time parameter estimation is one of methods of systemidentification

The key elements of system identification are:• Selection of model structure (linear, nonlinear, linear in

parameters, . . . )• Design of experiments (input signal, sampling period, . . . )

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 2/25

Preliminary Comments

Real-time parameter estimation is one of methods of systemidentification

The key elements of system identification are:• Selection of model structure (linear, nonlinear, linear in

parameters, . . . )• Design of experiments (input signal, sampling period, . . . )• Parameter estimation (off-line, on-line, . . . )

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 2/25

Preliminary Comments

Real-time parameter estimation is one of methods of systemidentification

The key elements of system identification are:• Selection of model structure (linear, nonlinear, linear in

parameters, . . . )• Design of experiments (input signal, sampling period, . . . )• Parameter estimation (off-line, on-line, . . . )• Validation mechanism (depends on application)

All these step should be present

in Real-Time Parameter Estimation Algorithms!

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 2/25

Parameter Estimation: Problem Formulation

Suppose that a system (a model of experiment) is

y(i) = φ1(i)θ01 + φ2(i)θ

02 + · · ·+ φn(i)θ

0n

=[

φ1(i), φ2(i), . . . , φn(i)]

︸ ︷︷ ︸

φ(i)T ←− 1×n

θ01...θ0n

︸ ︷︷ ︸

θ0 ←− n×1

= φ(i)T θ0

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 3/25

Parameter Estimation: Problem Formulation

Suppose that a system (a model of experiment) is

y(i) = φ1(i)θ01 + φ2(i)θ

02 + · · ·+ φn(i)θ

0n

=[

φ1(i), φ2(i), . . . , φn(i)]

︸ ︷︷ ︸

φ(i)T ←− 1×n

θ01...θ0n

︸ ︷︷ ︸

θ0 ←− n×1

= φ(i)T θ0

Given the data{

[y(i), φ(i)]}N

i=1={

[y(1), φ(1)], . . . , [y(N), φ(N)]}

The task is to find (estimate) the n unknown values

θ0 =[

θ01, θ

02, . . . , θ

0n

]T

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 3/25

What kind of solution is expected?

Algorithm solving the estimation problem should be as follows.

• Estimates should be computed in a numerically efficientway (an analytical formula is preferable!)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 4/25

What kind of solution is expected?

Algorithm solving the estimation problem should be as follows.

• Estimates should be computed in a numerically efficientway (an analytical formula is preferable!)

• Computed estimates (the mean value) should be close tothe real estimated values (be unbiased), provided that themodel is correct.

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 4/25

What kind of solution is expected?

Algorithm solving the estimation problem should be as follows.

• Estimates should be computed in a numerically efficientway (an analytical formula is preferable!)

• Computed estimates (the mean value) should be close tothe real estimated values (be unbiased), provided that themodel is correct.

• Estimates should have small variances, preferablydecreasing with N →∞ (more data –> better estimate).

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 4/25

What kind of solution is expected?

Algorithm solving the estimation problem should be as follows.

• Estimates should be computed in a numerically efficientway (an analytical formula is preferable!)

• Computed estimates (the mean value) should be close tothe real estimated values (be unbiased), provided that themodel is correct.

• Estimates should have small variances, preferablydecreasing with N →∞ (more data –> better estimate).

• The algorithm, if possible, should have intuitive clearmotivation.

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 4/25

Least Squares Algorithm:

Consider the function (loss-function) to be minimized

VN(θ) =1

2

{

(y(1)− φ(1)Tθ)2 + · · ·+ (y(N)− φ(N)Tθ)2}

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 5/25

Least Squares Algorithm:

Consider the function (loss-function) to be minimized

VN(θ) =1

2

{

(y(1)− φ(1)Tθ︸ ︷︷ ︸

ε(1)

)2 + · · ·+ (y(N)− φ(N)Tθ︸ ︷︷ ︸

ε(N)

)2}

=1

2

{

ε(1)2 + ε(2)2 + · · ·+ ε(N)2}

=1

2ETE

Here

E =[

ε(1), ε(2), . . . , ε(N)]

T

= Y − Φ θ

and

Y =[

y(1), y(2), . . . , y(N)]

T

, Φ =

φ(1)T

φ(2)T

...φ(N)T

←− N×n

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 5/25

Theorem (Least Square Formula):

The function VN(θ) is minimal at θ = θ, which satisfies theequation

ΦTΦ θ = ΦT Y

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 6/25

Theorem (Least Square Formula):

The function VN(θ) is minimal at θ = θ, which satisfies theequation

ΦTΦ θ = ΦT Y

If det(

ΦTΦ)

6= 0, then

• the minimum of the function VN(θ) is unique,

• the minimum value is 12Y TY − 1

2Y TΦ

(

ΦTΦ)−1

ΦTY and

is attained at

θ =(

ΦTΦ)−1

ΦTY

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 6/25

Proof:

The loss-function can be written as

VN(θ) =1

2(Y − Φθ)T (Y − Φθ)

It reaches its minimal value at θ if

∇θVN(θ) =

[∂VN(θ)

∂θ1, . . . ,

∂VN(θ)

∂θn

]

= 0 with θ = θ

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 7/25

Proof:

The loss-function can be written as

VN(θ) =1

2(Y − Φθ)T (Y − Φθ)

It reaches its minimal value at θ if

∇θVN(θ) =

[∂VN(θ)

∂θ1, . . . ,

∂VN(θ)

∂θn

]

= 0 with θ = θ

This equation is

∇θVN(θ) =1

2·2·∇θ (−Φθ)T (Y − Φθ) = −ΦT (Y − Φθ) = 0

( ∇θ (θT a) = ∇θ (aT θ) = aT , ∇θ (θT Aθ) = θT (A + AT ) )

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 7/25

Proof:

The loss-function can be written as

VN(θ) =1

2(Y − Φθ)T (Y − Φθ)

It reaches its minimal value at θ if

∇θVN(θ) =

[∂VN(θ)

∂θ1, . . . ,

∂VN(θ)

∂θn

]

= 0 with θ = θ

This equation is

∇θVN(θ) =1

2·2·∇θ (−Φθ)T (Y − Φθ) = −ΦT (Y − Φθ) = 0

( ∇θ (θT a) = ∇θ (aT θ) = aT , ∇θ (θT Aθ) = θT (A + AT ) ) Hence

ΦT Φ θ = ΦT Y

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 7/25

Proof (con’d):

If the matrix ΦT Φ is invertible, then we solve can find thesolution θ of ΦT Φ θ = ΦT Y :

θ =(

ΦTΦ)−1

ΦTY

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 8/25

Proof (con’d):

If the matrix ΦT Φ is invertible, then we solve can find thesolution θ of ΦT Φ θ = ΦT Y :

θ =(

ΦTΦ)−1

ΦTY

VN(θ) =1

2(Y − Φθ)T (Y − Φθ) =

1

2Y TY−Y TΦθ+

1

2θTΦTΦθ

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 8/25

Proof (con’d):

If the matrix ΦT Φ is invertible, then we solve can find thesolution θ of ΦT Φ θ = ΦT Y :

θ =(

ΦTΦ)−1

ΦTY

VN(θ) =1

2(Y − Φθ)T (Y − Φθ) =

1

2Y TY−Y TΦθ+

1

2θTΦTΦθ

and since

Y TΦθ = Y TΦ(

ΦTΦ)−1 (

ΦTΦ)

θ = θT

(

ΦTΦ)

θ

by completing the square

VN(θ) =1

2Y TY −

1

2θT

(

ΦTΦ)

θ

︸ ︷︷ ︸

independent of θ

+1

2(θ − θ)T

(

ΦTΦ)

(θ − θ)︸ ︷︷ ︸

≥ 0, · · · > 0 for θ 6= θ

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 8/25

Comments:

The formula can be re-written in terms of y(i), φ(i) as

θ =(

ΦTΦ)−1

ΦTY =

(N∑

i=1

φ(i)φ(i)

)−1( N∑

i=1

φ(i)y(i)

)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 9/25

Comments:

The formula can be re-written in terms of y(i), φ(i) as

θ =(

ΦTΦ)−1

ΦTY = P (N)

(N∑

i=1

φ(i)y(i)

)

where P (N) =(

ΦTΦ)−1

=

(N∑

i=1

φ(i)φ(i)

)−1

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 9/25

Comments:

The formula can be re-written in terms of y(i), φ(i) as

θ =(

ΦTΦ)−1

ΦTY = P (N)

(N∑

i=1

φ(i)y(i)

)

where P (N) =(

ΦTΦ)−1

=

(N∑

i=1

φ(i)φ(i)

)−1

The condition det(

ΦTΦ)

6= 0 is called an excitation condition.

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 9/25

Comments:

The formula can be re-written in terms of y(i), φ(i) as

θ =(

ΦTΦ)−1

ΦTY = P (N)

(N∑

i=1

φ(i)y(i)

)

where P (N) =(

ΦTΦ)−1

=

(N∑

i=1

φ(i)φ(i)

)−1

The condition det(

ΦTΦ)

6= 0 is called an excitation condition.

To assign different weights for different time instants, consider

VN(θ) = 12

{

w1(y(1)−φ(1)Tθ)2+ · · ·+wN(y(N)−φ(N)Tθ)2}

= 12

{

w1ε(1)2+w2ε(2)

2+ · · ·+wNε(N)2}

=1

2ETWE

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 9/25

Comments:

The formula can be re-written in terms of y(i), φ(i) as

θ =(

ΦTΦ)−1

ΦTY = P (N)

(N∑

i=1

φ(i)y(i)

)

where P (N) =(

ΦTΦ)−1

=

(N∑

i=1

φ(i)φ(i)

)−1

The condition det(

ΦTΦ)

6= 0 is called an excitation condition.

To assign different weights for different time instants, consider

VN(θ) = 12

{

w1(y(1)−φ(1)Tθ)2+ · · ·+wN(y(N)−φ(N)Tθ)2}

= 12

{

w1ε(1)2+w2ε(2)

2+ · · ·+wNε(N)2}

=1

2ETWE

⇒ θ =(

ΦTWΦ)−1

ΦTWY

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 9/25

Exponential forgetting

The common way to assign the weights is Least Squares withExponential Forgetting:

VN(θ) =1

2

N∑

i=1

λN−i(

y(i)− φ(i)T θ)2

with 0 < λ < 1. What is the solution formula?

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 10/25

Stochastic Notation

Uncertainty is often convenient to represent stochastically.

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 11/25

Stochastic Notation

Uncertainty is often convenient to represent stochastically.• e is random if it is not known in advance and could take

values from the set {e1, . . . , ei, . . . } with probabilities pi.• The mean value or expected value is

E e =∑

i

pi ei, E is the expectation operator.

For a realization {ei}Ni=1 : E e ≈ 1N

∑Ni=1 ei

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 11/25

Stochastic Notation

Uncertainty is often convenient to represent stochastically.• e is random if it is not known in advance and could take

values from the set {e1, . . . , ei, . . . } with probabilities pi.• The mean value or expected value is

E e =∑

i

pi ei, E is the expectation operator.

For a realization {ei}Ni=1 : E e ≈ 1N

∑Ni=1 ei

• The variance or dispersion characterizes possibledeviations from the mean value:

σ2 = var(e) = E (e− E e)2.

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 11/25

Stochastic Notation

Uncertainty is often convenient to represent stochastically.• e is random if it is not known in advance and could take

values from the set {e1, . . . , ei, . . . } with probabilities pi.• The mean value or expected value is

E e =∑

i

pi ei, E is the expectation operator.

For a realization {ei}Ni=1 : E e ≈ 1N

∑Ni=1 ei

• The variance or dispersion characterizes possibledeviations from the mean value:

σ2 = var(e) = E (e− E e)2.

• Two random variables e and f are independent if

E (e f) = (E e) · (E f) or cov(e, f) = 0.

where cov(e, f) = E((e−E e)(f −E f)

)is covariance.

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 11/25

Statistical Properties of LS Estimate:

Assume that the model of the system is stochastic, i.e.

y(i) = φ(i)Tθ0 + e(i){

⇔ Y (N) = Φ(N)θ0 + E(N)}

.

Here θ0 is vector of true parameters, 0 ≤ i ≤ N ≤ +∞,

•{

e(i)}

={

e(0), e(1), . . .}

is a sequence of independent

equally distributed random variables with Ee(i) = 0,

•{

φ(i)}

is either a sequence of random variables

independent of e or deterministic.

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 12/25

Statistical Properties of LS Estimate:

Assume that the model of the system is stochastic, i.e.

y(i) = φ(i)Tθ0 + e(i){

⇔ Y (N) = Φ(N)θ0 + E(N)}

.

Here θ0 is vector of true parameters, 0 ≤ i ≤ N ≤ +∞,

•{

e(i)}

={

e(0), e(1), . . .}

is a sequence of independent

equally distributed random variables with Ee(i) = 0,

•{

φ(i)}

is either a sequence of random variables

independent of e or deterministic.

Pre-multiplying the model by(

Φ(N)TΦ(N))−1

Φ(N)T , gives(

Φ(N)TΦ(N))−1

Φ(N)T×Y (N) = θ̂

θ̂ =(

Φ(N)TΦ(N))−1

Φ(N)T ×(

Φ(N)θ0 + E(N))

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 12/25

Statistical Properties of LS Estimate:

Assume that the model of the system is stochastic, i.e.

y(i) = φ(i)Tθ0 + e(i){

⇔ Y (N) = Φ(N)θ0 + E(N)}

.

Here θ0 is vector of true parameters, 0 ≤ i ≤ N ≤ +∞,

•{

e(i)}

={

e(0), e(1), . . .}

is a sequence of independent

equally distributed random variables with Ee(i) = 0,

•{

φ(i)}

is either a sequence of random variables

independent of e or deterministic.

Pre-multiplying the model by(

Φ(N)TΦ(N))−1

Φ(N)T , gives

θ̂ = θ0 +(

Φ(N)TΦ(N))−1

Φ(N)T × E(N)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 12/25

Theorem: Suppose the data are generated by

y(i) = φ(i)Tθ0 + e(i){

⇔ Y (N) = Φ(N)θ0 + E(N)}

where{

e(i)}

={

e(0), e(1), . . .}

is a sequence of

independent random variables with zero mean and variance σ2.

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 13/25

Theorem: Suppose the data are generated by

y(i) = φ(i)Tθ0 + e(i){

⇔ Y (N) = Φ(N)θ0 + E(N)}

where{

e(i)}

={

e(0), e(1), . . .}

is a sequence of

independent random variables with zero mean and variance σ2.

Consider the least square estimate

θ̂ =(

Φ(N)TΦ(N))−1

Φ(N)TY (N).

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 13/25

Theorem: Suppose the data are generated by

y(i) = φ(i)Tθ0 + e(i){

⇔ Y (N) = Φ(N)θ0 + E(N)}

where{

e(i)}

={

e(0), e(1), . . .}

is a sequence of

independent random variables with zero mean and variance σ2.

Consider the least square estimate

θ̂ =(

Φ(N)TΦ(N))−1

Φ(N)TY (N).

If detΦ(N)TΦ(N) 6= 0, then

• E θ̂ = θ0, i.e. the estimate is unbiased;• the covariance of the estimate is

cov θ̂ = E(

θ̂ − θ0) (

θ̂ − θ0)

T

= σ2(

Φ(N)TΦ(N))−1

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 13/25

Regression Models for FIR

FIR (Finite Impulse Response) or MA (Moving Average Model):

y(t) = b1 u(t− 1) + b2 u(t− 2) + · · ·+ bn u(t− n)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 14/25

Regression Models for FIR

FIR (Finite Impulse Response) or MA (Moving Average Model):

y(t) = b1 u(t− 1) + b2 u(t− 2) + · · ·+ bn u(t− n)

y(t) = [u(t− 1), u(t− 2), . . . , u(t− n)]︸ ︷︷ ︸

=φ(t−1)T

b1

b2...bn

︸ ︷︷ ︸

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 14/25

Regression Models for FIR

FIR (Finite Impulse Response) or MA (Moving Average Model):

y(t) = b1 u(t− 1) + b2 u(t− 2) + · · ·+ bn u(t− n)

y(t) = [u(t− 1), u(t− 2), . . . , u(t− n)]︸ ︷︷ ︸

=φ(t−1)T

b1

b2...bn

︸ ︷︷ ︸

Note the change of notation for the typical case when the RHSdoes no have the term b0 u(t): φ(i) −→ φ(t− 1) to indicatedependence of date on input signals up to t− 1th.

However, we keep: Y (N) = Φ(N) θ

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 14/25

Regression Models for ARMA

IIR (Infite Impulse Response) or ARMA (Autoregressive MovingAverage Model):

(qn + a1 q

n−1 + · · ·+ an

)

︸ ︷︷ ︸

A(q)

y(t) =(b1 q

m−1 + b2 qm−2 + · · ·+ bm

)

︸ ︷︷ ︸

B(q)

u(t)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 15/25

Regression Models for ARMA

IIR (Infite Impulse Response) or ARMA (Autoregressive MovingAverage Model):

(qn + a1 q

n−1 + · · ·+ an

)

︸ ︷︷ ︸

A(q)

y(t) =(b1 q

m−1 + b2 qm−2 + · · ·+ bm

)

︸ ︷︷ ︸

B(q)

u(t)

q is the forward shift operator:

q u(t) = u(t + 1), q−1 u(t) = u(t− 1)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 15/25

Regression Models for ARMA

IIR (Infite Impulse Response) or ARMA (Autoregressive MovingAverage Model):

(qn + a1 q

n−1 + · · ·+ an

)

︸ ︷︷ ︸

A(q)

y(t) =(b1 q

m−1 + b2 qm−2 + · · ·+ bm

)

︸ ︷︷ ︸

B(q)

u(t)

q is the forward shift operator:

q u(t) = u(t + 1), q−1 u(t) = u(t− 1)

y(t) = [−y(t− 1), . . . ,−y(t− n), u(t− n + m− 1), . . . , u(t− n)]︸ ︷︷ ︸

=φ(i−1)T

a1

...an

b1...

bm

︸ ︷︷ ︸

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 15/25

Problem 2.2

Consider the FIR model

y(t) = b0u(t) + b1u(t− 1) + e(t), t = 1, 2, 3, . . . , N

where{e(t)

}is a sequence of independent normalN (0, σ)

random variables

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 16/25

Problem 2.2

Consider the FIR model

y(t) = b0u(t) + b1u(t− 1) + e(t), t = 1, 2, 3, . . . , N

where{e(t)

}is a sequence of independent normalN (0, σ)

random variables

• Determine LS estimates for b0, b1 when u(t) is a step.Analyze the covariance of the estimate, when N →∞

• Make the same investigation when u(t) is white noise withunit variance.

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 16/25

Solution (Problem 2.2):

For the model

y(t) = b0u(t) + b1u(t− 1) + e(t)

the regression form is readily seen

y(t) =[u(t), u(t− 1)

]

[

b0

b1

]

+ e(t) = φ(t)Tθ + e(t)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 17/25

Solution (Problem 2.2):

For the model

y(t) = b0u(t) + b1u(t− 1) + e(t)

the regression form is readily seen

y(t) =[u(t), u(t− 1)

]

[

b0

b1

]

+ e(t) = φ(t)Tθ + e(t)

Then LS estimate is

θ̂ =(

Φ(N)TΦ(N))−1

Φ(N)TY (N)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 17/25

Solution (Problem 2.2):

For the model

y(t) = b0u(t) + b1u(t− 1) + e(t)

the regression form is readily seen

y(t) =[u(t), u(t− 1)

]

[

b0

b1

]

+ e(t) = φ(t)Tθ + e(t)

Then LS estimate is

θ̂ =

u(1) u(0)

u(2) u(1)

...

u(N) u(N−1)

T

u(1) u(0)

u(2) u(1)

...

u(N) u(N − 1)

−1

u(1) u(0)

u(2) u(1)

...

u(N) u(N−1)

T

y(1)

y(2)

...

y(N)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 17/25

Solution (Con’d):

The formula for an arbitrary input signal u(t)

θ̂ =

u(1) u(0)

u(2) u(1)

...

u(N) u(N−1)

T

u(1) u(0)

u(2) u(1)

...

u(N) u(N − 1)

−1

u(1) u(0)

u(2) u(1)

...

u(N) u(N−1)

T

y(1)

y(2)

...

y(N)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 18/25

Solution (Con’d):

The formula for an arbitrary input signal u(t)

θ̂ =

u(1) u(0)

u(2) u(1)

...

u(N) u(N−1)

T

u(1) u(0)

u(2) u(1)

...

u(N) u(N − 1)

−1

u(1) u(0)

u(2) u(1)

...

u(N) u(N−1)

T

y(1)

y(2)

...

y(N)

becomes

θ̂ =

∑Nt=1 u(t)

2∑N

t=1 u(t)u(t− 1)∑N

t=1 u(t)u(t− 1)∑N

t=1 u(t− 1)2

−1

×

×

∑Nt=1 u(t)y(t)

∑Nt=1 u(t− 1)y(t)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 18/25

Solution (Con’d):

Suppose that u(t) is a unit step applied at t = 1

u(t) =

{

1, t = 1, 2, . . .

0, t = 0, −1, −2, −3, . . .

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25

Solution (Con’d):

Suppose that u(t) is a unit step applied at t = 1

u(t) =

{

1, t = 1, 2, . . .

0, t = 0, −1, −2, −3, . . .

Substituting this signal into the general formula, we obtain

θ̂ =

∑Nt=1 u(t)

2∑N

t=1 u(t)u(t− 1)∑N

t=1 u(t)u(t− 1)∑N

t=1 u(t− 1)2

−1

×

×

∑Nt=1 u(t)y(t)

∑Nt=1 u(t− 1)y(t)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25

Solution (Con’d):

Suppose that u(t) is a unit step applied at t = 1

u(t) =

{

1, t = 1, 2, . . .

0, t = 0, −1, −2, −3, . . .

Substituting this signal into the general formula, we obtain

θ̂ =

N

∑Nt=1 u(t)u(t− 1)

∑Nt=1 u(t)u(t− 1)

∑Nt=1 u(t− 1)2

−1

×

×

∑Nt=1 u(t)y(t)

∑Nt=1 u(t− 1)y(t)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25

Solution (Con’d):

Suppose that u(t) is a unit step applied at t = 1

u(t) =

{

1, t = 1, 2, . . .

0, t = 0, −1, −2, −3, . . .

Substituting this signal into the general formula, we obtain

θ̂ =

N N − 1

N − 1∑N

t=1 u(t− 1)2

−1

×

×

∑Nt=1 u(t)y(t)

∑Nt=1 u(t− 1)y(t)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25

Solution (Con’d):

Suppose that u(t) is a unit step applied at t = 1

u(t) =

{

1, t = 1, 2, . . .

0, t = 0, −1, −2, −3, . . .

Substituting this signal into the general formula, we obtain

θ̂ =

N N − 1

N − 1 N − 1

−1

×

×

∑Nt=1 u(t)y(t)

∑Nt=1 u(t− 1)y(t)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25

Solution (Con’d):

Suppose that u(t) is a unit step applied at t = 1

u(t) =

{

1, t = 1, 2, . . .

0, t = 0, −1, −2, −3, . . .

Substituting this signal into the general formula, we obtain

θ̂ =

N N − 1

N − 1 N − 1

−1

×

×

∑Nt=1 y(t)

∑Nt=2 y(t)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25

Solution (Con’d):

Suppose that u(t) is a unit step applied at t = 1

u(t) =

{

1, t = 1, 2, . . .

0, t = 0, −1, −2, −3, . . .

Substituting this signal into the general formula, we obtain

θ̂ =1

N(N − 1)− (N − 1)2

N − 1 −(N − 1)

−(N − 1) N

×

×

∑Nt=1 y(t)

∑Nt=2 y(t)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25

Solution (Con’d):

Suppose that u(t) is a unit step applied at t = 1

u(t) =

{

1, t = 1, 2, . . .

0, t = 0, −1, −2, −3, . . .

Substituting this signal into the general formula, we obtain

θ̂ =

1 −1

−1N

N − 1

×

×

∑Nt=1 y(t)

∑Nt=2 y(t)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25

Solution (Con’d):

Suppose that u(t) is a unit step applied at t = 1

u(t) =

{

1, t = 1, 2, . . .

0, t = 0, −1, −2, −3, . . .

Substituting this signal into the general formula, we obtain

θ̂ =

∑Nt=1 y(t)−

∑Nt=2 y(t)

−∑N

t=1 y(t) +N

N − 1

N∑

t=2

y(t)

=

y(1)

1

N − 1

N∑

t=1

y(t)− y(1)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 19/25

Solution (Con’d):

How to compute the covariance of θ? Consider

θ − θ0 =

y(1)

1N−1

∑Nt=1 y(t)− y(1)

b0

b1

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 20/25

Solution (Con’d):

How to compute the covariance of θ? Consider

θ − θ0 =

y(1)

1

N − 1

N∑

t=1

y(t)− y(1)

b0

b1

=

{

b0 · 1 + b1 · 0 + e(1)}

− b0

N∑

t=2

{

b0 · 1 + b1 · 1 + e(t)}

+ y(1)

N − 1− y(1) − b1

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 20/25

Solution (Con’d):

How to compute the covariance of θ? Consider

θ − θ0 =

y(1)

1

N − 1

N∑

t=1

y(t)− y(1)

b0

b1

=

e(1)

1

N − 1

N∑

t=2

e(t)− e(1)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 20/25

Solution (Con’d):

The estimation error is

θ − θ0 =

e(1)

1

N − 1

N∑

t=2

e(t)− e(1)

The covariance E(θ − θ0)(θ − θ0)T of estimate is then

cov(θ) =

Ee(1)2 E{

N

t=2e(t)

N−1− e(1)

}

e(1)

E{

N

t=2e(t)

N−1− e(1)

}

e(1) E{

N

t=2e(t)

N−1− e(1)

}2

=

σ2 −σ2

−σ2 σ2(

N−1(N−1)2

− 1)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 20/25

Solution (Con’d):

The estimation error is

θ − θ0 =

e(1)

1

N − 1

N∑

t=2

e(t)− e(1)

The covariance E(θ − θ0)(θ − θ0)T of estimate is then

cov(θ) =

Ee(1)2 E{

N

t=2e(t)

N−1− e(1)

}

e(1)

E{

N

t=2e(t)

N−1− e(1)

}

e(1) E{

N

t=2e(t)

N−1− e(1)

}2

=

σ2 −σ2

−σ2 σ2 NN−1

= σ2

1 −1

−1 NN−1

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 20/25

Solution (Con’d):To conclude with the unit step input signal LS estimate is

θ̂ =

b0

b1

=

y(1)

1

N − 1

N∑

t=1

y(t)− y(1)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 21/25

Solution (Con’d):To conclude with the unit step input signal LS estimate is

θ̂ =

b0

b1

=

y(1)

1

N − 1

N∑

t=1

y(t)− y(1)

The covariance of such estimate looks as

E(θ − θ0)(θ − θ0)T = σ2

1 −1

−1 NN−1

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 21/25

Solution (Con’d):To conclude with the unit step input signal LS estimate is

θ̂ =

b0

b1

=

y(1)

1

N − 1

N∑

t=1

y(t)− y(1)

The covariance of such estimate looks as

E(θ − θ0)(θ − θ0)T = σ2

1 −1

−1 NN−1

As a number of measured data increases N →∞

E(θ − θ0)(θ − θ0)T →

1 −1

−1 1

and does NOT IMPROVE!

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 21/25

Solution (Problem 2.2):

Consider now the model

y(t) = b0u(t) + b1u(t− 1) + e(t)

when the input signal is a white noise with unit variance

Eu(t)2 = 1, Eu(t)u(s) = 0 if t 6= s

and when u and e are independent

Eu(t)e(s) = 0 ∀ t, s

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 22/25

Solution (Problem 2.2):

Consider now the model

y(t) = b0u(t) + b1u(t− 1) + e(t)

when the input signal is a white noise with unit variance

Eu(t)2 = 1, Eu(t)u(s) = 0 if t 6= s

and when u and e are independent

Eu(t)e(s) = 0 ∀ t, s

Such assumptions imply that

Ey(t)u(t) = b0, Ey(t)u(t− 1) = b1

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 22/25

Solution (Problem 2.2):

The LS estimate becomes dependent on a realization of thestochastic variable u(t); we need to consider its mean value

Eθ̂ = E

∑Nt=1 u(t)

2∑N

t=2 u(t)u(t− 1)∑N

t=2 u(t)u(t− 1)∑N

t=2 u(t− 1)2

−1

×

×

∑Nt=1 u(t)y(t)

∑Nt=2 u(t− 1)y(t)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 23/25

Solution (Problem 2.2):

The LS estimate becomes dependent on a realization of thestochastic variable u(t); we need to consider its mean value

Eθ̂ = E

N(

1

N

N

t=1u(t)2

)

(N − 1)(

1

N−1

N

t=2u(t)u(t− 1)

)

(N − 1)(

1

N−1

N

t=2u(t)u(t− 1)

)

(N − 1)(

1

N−1

N

t=2u(t− 1)2

)

−1

×

×

N(

1

N

N

t=1u(t)y(t)

)

(N − 1)(

1

N−1

N

t=2u(t− 1)y(t)

)

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 23/25

Solution (Problem 2.2):

The LS estimate becomes dependent on a realization of thestochastic variable u(t); we need to consider its mean value

Eθ̂ ≈

N Eu(t)2 (N − 1)Eu(t)u(t− 1)

(N − 1)Eu(t)u(t− 1) (N − 1)Eu(t− 1)2

−1

×

×

N Eu(t)y(t)

(N − 1)Eu(t− 1)y(t)

=

N 0

0 (N − 1)

−1

Nb0

(N − 1)b1

=

b0

b1

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 23/25

Solution (Problem 2.2):

To compute the covariance, use the formula in Theorem

cov(θ − θ0) = σ2E (Φ(N)TΦ(N))−1

= σ2

N 0

0 (N − 1)

−1

= σ

1N

0

0 1N−1

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 24/25

Solution (Problem 2.2):

To compute the covariance, use the formula in Theorem

cov(θ − θ0) = σ2E (Φ(N)TΦ(N))−1

= σ2

N 0

0 (N − 1)

−1

= σ

1N

0

0 1N−1

It converges to zero!

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 24/25

Next Lecture / Assignments:

• Next lecture (April 13, 10:00-12:00, in A206Tekn): RecursiveLeast Squares, modifications, continuous-time systems.

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 25/25

Next Lecture / Assignments:

• Next lecture (April 13, 10:00-12:00, in A206Tekn): RecursiveLeast Squares, modifications, continuous-time systems.

• Solve problems 2.5 and 2.6

c©Leonid Freidovich. March 31, 2010. Elements of Iterative Learning and Adaptive Control: Lecture 2 – p. 25/25