A Rank-Revealing Method for Low Rank Matrices withUpdating, Downdating, and Applications

Tsung-Lin Lee(Michigan State University)

2007 AMS Session Meeting, Chicago

joint work with Tien-Yien Li and Zhonggang Zeng

Rank determination problems appear in

1. Image Processing

2. Information Retrieval

3. Matrix Approximation

4. Least Squares Problems

5. Numerical Polynomial Algebra

……

1

k

k

The rank

gap:

kAranknkk

)(11

Numerical rank: nmR nm ,

the rank decision threshold :

the approxi-rank w.r.t. the threshold : (numerical rank)

)(rank

2k

kr k,rank(A) Assume

2})(|{12})(|{

minmin BABArBrankB

rrBrankB

Mirsky Theorem:

1k

A

k

2k

The numerical rank w.r.t. threshold :

)(min2

BrankArankAB

1k

A

k

SVD Algorithm (Golub-Reinsch)

In some applications, the matrix is large.

-The rank is close to full. (high rank)

-The rank is close to zero. (low rank)

=> efficient when the matrix size is moderate.

The goal: An efficient and stable algorithm

The updating and downdating problems

=> It can’t solve them efficiently.

1989 Tony Chan => Rank Revealing QR algorithm for high rank matrices

Updating problem:

krank )(

?)ˆ( rank

?)ˆ( rank

Downdating problem:

krank )(

?)ˆ( rank

?)ˆ( rank

1992 G.W. Stewart => rank revealing UTV decomposition. (URV/ULV)

1. Updating problems are applicable.

2. Downdating problems are difficult.

=> re-compute the UTV decomposition

F.D. Fierro, P.C. Hansen and P.S. K. Hansen (1999)UTV tools: Matlab templates for rank-revealing UTV decomposition

2005, T.Y. Li and Zhonggang Zeng

=> rank-revealing algorithm for high rank matrices

1. The approxi-rank.2. The approxi-kernel.3. The method is more efficient and robust.4. Algorithms for updating and downdating problems are straightforward, stable and efficient.

kernel-approxileftkernel-approxi

range-approxi rowspace-approxi

Tnkk

n

k

k

mkk vvvvuuuu

111

1

11

nkk

11

Tsung-Lin Lee, T.Y. Li and Zhonggang Zeng

=> rank-revealing algorithm for low rank matrices

1. The approxi-rank.2. The approxi-range.3. The approxi-rowspace.4. The projections of left and right kernel.5. USV+E decomposition.6. The method is robust and more efficient.7. Algorithms for updating and downdating problems are straightforward, stable and efficient.

= +

2

21

1

j

jj

jT

jT

j

xA

xy

yA

yx

1v

1u

Stop when thresholdepsyA

j

jT

:,

2

21

Power iteration on T

Random 0y

,3,2,1for j

0y

1y

2y

4y1u

epsO

0

mR

3y

},,,{ 21 kuuuspan approxi-range

},,,{)( 211 kuuuspanrangez

nnkk 1121

0' k

The implicit singular value deflation:

0'''1121

nkk

Tzz 11'

Tp UUE kzzU ,,1

USV+E decomp.

TT LQU LQ pT EULQ

)(1 rangez

Tzz 11' )'(2 rangez

TT zzzz 2211'' )''(3 rangez

Tkk

Tp zzzzE 11

2pE

perturbation= +

USV+E decomposition

kk

approxi-range

approxi-rowspace

Numerical experiments and comparisonsMatlab 7.0, on Dell PC Pentium D 3.2MHz CPU, 1GB RAM

2n

n,20

2

400x200 800x400 1600x800 3200x1600

time error time error time error time error

SVD 0.31 3e-9 2.19 4e-9 16.6 3e-9 144. 7e-9

lurv 0.66 3e-9 1.52 4e-9 5.97 3e-9 32.5 7e-9

lulv 0.56 4e-9 1.52 6e-9 6.03 5e-9 31.9 5e-9

larank 0.05 3e-9 0.11 4e-9 0.39 3e-9 1.81 4e-9

3111

10 e

10)(,81 ranke

A U= + pE

TV

Row updating

A U= + pE

TV

1

TV~

A U= + pE

TV

1

Tv~

2aVVa T

aVVa

vT

~

Ta

row downdating

1)ˆ( korkrank )()ˆ( rowrow

QRUV )(min

R

deflate R

min

210

0ˆ

R

RGG

pT EQRV

= +

EUSV T

Dominant(signal)A Perturbation

(noise)

= +

USV+E decomposition

)( 2nO )(nO

Information retrieval

Latent Semantic Indexing method (LSI)

Library database

Webpage search engine (Google)

…

rank, revealing, updating, downdating, application

1

0

1

1

0

0

0

0

0

1

0

1

||

q12x8 term by document matrix

2222

,cos

j

jT

j

j

jAeq

Aeq

Aeq

Aeq

EUSVA T

= +

222222

~cos

j

jT

jT

jTT

jT

jTT

jWeq

WeUq

eSVq

eSVUq

eUSVq

eUSVq

TSVW

Image processing

Saving storage of photographs

FBI Fingerprint Image Database

Face Image Database

…

A 480x640 monochrome (baseball picture)

Grey levels: 0 => 1

black white

j

j

Rank 20 approximation imageRank 480 image

20)(%35.1 1 kArank

7.477.920.735.38 : 1510.8%

2.014.945.140.388.07 : 1341.3%

k SVD

2.87

lulv

3.02

lurvlarank

0.17

Running time (seconds)

15.2 : 1

Compression ratio

18

Approxi-rank

2.1%

Threshold

1

1

1

1:)( knm

mn

http://www.msu.edu/~leetsung/Software.htm

HighRankRev and LowRankRev Package

Thank you