Time to Equilibrium for Finite State Markov Chain

許元春（交通大學應用數學系）

a finite set ( state space) a sequence of valued random variables ( random process, stochastic process)

( finite-dimensional distribution)

( Here )

:SS

),,,( 1100 nn iXiXiXP

),|()|()( 110022001100 iXiXiXPiXiXPiXP

),,,|(),,,|( 111100110011 nnnnkkkk iXiXiXiXPiXiXiXiXP

)(

)()|(

AP

BAPABP

:,,,,, 210 nXXXX

What is ?Among all possibilities, the following two

are the simplest： (i.i.d.)

where is a probability measure on

• Example: ( Black-Scholes-Merton Model)

the price of some asset at time t

),,,|( 110011 kkkk iXiXiXiXP

)(),,|( 10011 kkkkk iPiXiXiXPP

S

)(

))1((ln

nS

nSX n

)(tS

Here is a stochastic matrix

(i.e. and )

In this case,

is the transition probability for the Markov chain

),(),,|( 10011 kkkkkk iiKiXiXiXP

K

|||| SS0),( jiK

i

j,

Sj

jiK 1),(

),( yxK

0}{ nnX

Example: ( Riffle Shuffles )

(Gilbert, Shannon ‘55, Reeds ‘81) n

k kn

n

k

n

kn

a

b

ba

a

ba

b

n

n

k

n

2

binomial

n

k

Markov Chain with transition kernel K and initial distribution λ

This implies

In particular, we observe

Here and

),(),(),()(),....,,( 1211001100 nnnn iiKiiKiiKiiXiXiXP

),(),(),()|,....,( 121100011 nnnn iiKiiKiiKiXiXiXP

),()|( 0 jiKiXjXP nn KK 1

KKK nn 1

What is the limiting distribution of given ? (i.e. What is the limiting behavior for ?)

Example: ( Two State Chain )

1

0

1

1

0

1

,10

10

nX iX 0

nK

K

0 I

20 II

1 III

10

01nK

nn

nn

nK)1()1(

)1()1(

n

IK n 2

KK n 12

n

nK

lim

does not exist

invariant/equilibrium/stationary distribution

Suppose for some , that

for all

Then

Sx)(),( yyxK

nn

Sy

K ..ei

)(),()( yyxKxSx

Sy

Ergodic Markov Chain

Assume is aperiodic and irreducible.

Then there admits a unique invariant

distribution λ and

How the distribution of converge to its

limiting distribution?

K

)(),( yyxK n Syx ,

nX

Distance between two probability measures ν and μ on S . ( total variation distance )

( distance )

( Note that )

)}()({sup AASA

TV

PL

Px

x

x

x

Pxx

x

x

xP

Sx

P

P

,)(

)(

)(

)(sup

1,)()(

)(

)(

)(/1

,

1,2

1

TV

For

is a non-increasing sub-additive function

( )

This implies that if

for some and

then

p1n

P

n

SxxK

,)(),(sup

..ei )()()( mfnfmnf

mn,

P

m

SxxK

,)(),(sup

m

10

nn

m

n

P

n

Sx

m

n

xK

,)(),(sup

1

lnexpm

n

n

n

mn

1

ln/

exp

We say is reversible if it satisfies the detailed balance condition

Assume is reversible, irreducible and aperiodic.Then there exists eigenvalue and for any corresponding orthonormal basis of eigenvectors with , we have

and

K

),()(),()( xyKyyxKx Syx ,K

1....1 012||1|| SS

1||

0

)()()(

),( S

iii

ni

n

yxy

yxK

1||

1

222

2,|)(|||),(

S

ii

ni

n xxK

}{ i 10

the smallest non-zero eigenvalue of = the spectral gap of

where

is the smallest constant satisfying the Poincare inequality

Holding for all

11 KI

),( K

0)(|)(

),(inf fVar

fVar

ff

2)()( fEfEfVar

yx

xyxKxfyfff,

2 )(),(|)()(|2

1),(

"1

" A

),()( ffAfVar f

'

Setting .Then

( The Divichlet form associated with the semigroup )

and

Note that

Hence

0

)(

!m

mtKIt

t m

KeeH

),)((1

lim),(200

2

2

ffHIt

fHt

ff tttt

2

2

2

2)()( ffH

dt

dffH

dt

dtt

)(2

))(),((2

fHVar

ffHffH

t

tt

)( fHVardt

dt

)()( 22

2fVareffH t

t

tH

),(22

2fHfHfH

t ttt

Theorem:

The mixing time is given by

• Theorem:

where

)(),(

2, x

exH

t

t

tt e

x

yyyxH

)(

)()(),(

2T

exHtT t

Sx

1),(max|0inf

2,2

)1

log2

11(

11

*2

T

)(min* xSx

Consider the entropy – like quantity

And

The log-Sobolev constant is given by the

Formula Hence is the smallest constant satisfying

the log-Sobolev inequality

holding for all function

)(|)(|

log|)(|)(2

2

22 s

f

sfsff

Ss

,

)(

),(inf{

f

ff

}0)( f

1 A

),()( ffAf

f

Theorem:

• •

2

)1

loglog4

11(

1

2

1*2 T

2T

)1

loglog2

12(

1*

)1

log2

11(

1*

1

21

)1

loglog4

11(

1*

Can one compute or estimate the constant ?The present answer is that it seems to be a very difficult

problem to estimate . Lee-Yau(1998), Ann. of probability symmetric simple exclusion/random transpositionDiaconis-Saloff-Coste(1996), Ann. Of Applied

Probability . For simple random walk on cycle,

. The exact value of for with all rows equal to Chen-Sheu(03), Journal of Functional Analysis when and is even

n

2

1~nn

K

)2

log1(2

1

nn

4n

n

Who Cares ?

a set. a group.

Action of group on set :

Orbit of for some

What’s the number of orbits (or patterns) ?

:S

:G

yxSyOx gx |{

}Gg

Z

G S

SxSGxg g ),(

Example ( balls, boxes, Bose-Einstein distribution)

Polya’s theory of counting

(See Enumerative Combinatorics, Vol II, by R. Stanley, Sec7.24)

Burnside Process (Jerrum and Goldberg)

n

k,][ nkS

},...,2,1{][ kk

nSG

}|{ xxGgG gx

}|{ ssSsS gg

n

l

Diaconis (‘03) ( balls, boxes)

for all

yx GGg g

x

SG

OyxK

||

1

||

||),(

zOx

x

1

||

1)(

zOx

1)( x

|1

),(||)(),(|z

OxKOOxK yl

yyl

TV

l xK ),(

n

k

,))(1(),0( l

TV

l kCK !

1~)(k

kC

)( nk

0ddK

TV

l ),0(

nl log

Cut-off phenomenon

Bayer and Diacoins (’86)

The total variation distance for riffle shuffles of 52 cards

“neat riffle shuffles”?

l