資料結構與演算法 ( 上 )

2008 Fall Semester2008 Fall Semester Data Structures and Algorithms (I)Data Structures and Algorithms (I) 11

資料結構與演算法 ( 上 )

呂學一 (Hsueh-I Lu)http://www.csie.ntu.edu.tw/~hil/

Outline of this slideOutline of this slide Dynamic programming

– Fibonacci sequence– Stamp problem– Sequence alignment– Matrix multiplication

Leornardo FibonacciLeornardo Fibonacci1170-1250 1170-1250

Old hens never die …Old hens never die …They just lay eggs!They just lay eggs! At the beginning of Day 1, there is a hen. Each hen lays an egg every 24 hours. Each egg takes 24 hours to become a hen. F(n) = the number of hens at the end of

day n. Give an algorithm to compute F(n).

Day 1Day 1

Day 2Day 2

Day 3Day 3

Day 4Day 4

Day 5Day 5

E(n) = the number of E(n) = the number of eggs at the end of Day eggs at the end of Day nn E(n) = ? F(n) = ?

The recurrence The recurrence relationrelation

F (n) = F (n ¡ 1) +E (n ¡ 1)= F (n ¡ 1) +F (n ¡ 2):

F (1) = F (2) = 1.

The recursive The recursive algorithmalgorithmint F (n) fif (n · 2)return 1;

elsereturn F (n ¡ 1) +F (n ¡ 2);

int F (n) fif (n · 2)return 1;

elsereturn F (n ¡ 1) +F (n ¡ 2);

Very Inefficient!Very Inefficient!F(8)

F(7) F(6)

F(6) F(5) F(5) F(4)

F(5) F(4) F(4) F(3) F(4) F(3) F(3) F(2)

1:61800365 = 1:89303571£ 10761:6180030 = 1859325:9

It takes ((1:618)n) time to compute F (n) using therecursivealgorithm.

Dynamic-Programming Dynamic-Programming ApproachApproachint F (n) flet F [1]= 1;let F [2]= 1;for i = 3 to n dolet F [i]= F [i ¡ 1]+F [i ¡ 2];

return F [n];g

int F (n) flet F [1]= 1;let F [2]= 1;for i = 3 to n dolet F [i]= F [i ¡ 1]+F [i ¡ 2];

return F [n];g

The DP-algorithm takes only O(n) time

and space!

IllustrationIllustration

n 1 2 3 4 5 6 7

F[n] 1 1 2 3 5 8 13

Dynamic ProgrammingDynamic Programming A clever way to implement recursion:

– Using storage to avoid unnecessarily duplicated efforts.

– 讓走過的留下痕跡

QuestionQuestion 有沒有可能維持線性的時間，卻將空間降低到 O(1)?

Another example

Choosing stamps

The problemThe problem If the postage is n, what is the minimum

number of stamps to cover the postage?

A recursive algorithmA recursive algorithmint stamp(n) fif (n == 0) return 0;if (n < 0) return 1 ;if (n 2 f2;5;8;14g) return 1;let n2 = stamp(n ¡ 2);let n5 = stamp(n ¡ 5);let n8 = stamp(n ¡ 8);let n14 = stamp(n ¡ 14);return 1+min(n2;n5;n8;n14);

int stamp(n) fif (n == 0) return 0;if (n < 0) return 1 ;if (n 2 f2;5;8;14g) return 1;let n2 = stamp(n ¡ 2);let n5 = stamp(n ¡ 5);let n8 = stamp(n ¡ 8);let n14 = stamp(n ¡ 14);return 1+min(n2;n5;n8;n14);

The DP-versionThe DP-versionint stamp(n) flet S[0]= 0;for i = ¡ 1 to ¡ 13 dolet S[i]= 1 ;

for i = 1 to n dolet S[i]= 1+min(S[i ¡ 2];S[i ¡ 5];S[i ¡ 8];S[i ¡ 14]);

return S[n];g

int stamp(n) flet S[0]= 0;for i = ¡ 1 to ¡ 13 dolet S[i]= 1 ;

for i = 1 to n dolet S[i]= 1+min(S[i ¡ 2];S[i ¡ 5];S[i ¡ 8];S[i ¡ 14]);

return S[n];g

The DP-algorithm takes only O(n) time

and space!

IllustrationIllustration0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4

0 * 1 * 2 1 3 2 1 3 2 4 3 2 1 3 2 4 3 2 4 3 2 4 3

QuestionQuestion 剛剛只是問幾張郵票 . 如果我們想要知道最少張郵票的貼法，究竟是每一種面額的郵票各幾張，應該如何處理？需要額外再花空間嗎？

Sequence Alignment

Aligning two stringsAligning two strings A = attgatcctag B = acttagtccttcgc

A → a-ttga-tcc-tag- B → actt-agtccttcgc

gapgapgapgapgap

Measuring an Measuring an alignmentalignment

a t g c -a 2 -2 -2 -2 -1t -2 2 -2 -2 -1g -2 -2 2 -2 -1c -2 -2 -2 2 -1- -1 -1 -1 -1 -1

Scoring matrix

Other scoring matricesOther scoring matrices

a t g c -a 5 -4 -4 -4 -4t -4 5 -4 -4 -4g -4 -4 5 -4 -4c -4 -4 -4 5 -4- -4 -4 -4 -4 -4

BLAST matrix Transition/Transversion matrix

a t g c -a 0 5 5 1 -1t 5 0 1 5 -1g 5 1 0 5 -1c 1 5 5 0 -1- -1 -1 -1 -1 -1

Scoring matrix is an Scoring matrix is an artart Log odds matrix

– score[i, j] = log (q(i, j) / p(i) p(j)). PAM matrix

– Point accepted mutations BLOSOM matrix

– Block substitution matrix– Steven Henikoff and Jorja G. Henikoff (1992).

Other specialized scoring matrices– Domenico Bordo and Patrick Argos (1991). – Jean-Michael Claverie (JCB 1993). – Lee F. Kowlakowski and Kenneth A. Rice (Nature 1994)

Scoring an Scoring an alignmentalignment

a – t t g a – t c c – t a g - c c t t – a g t c c t t c g c -2-1+2+2-1+2-1+2+2+2-1+2-2+2-1

score = 7

a t g c -a 2 -2 -2 -2 -1t -2 2 -2 -2 -1g -2 -2 2 -2 -1c -2 -2 -2 2 -1- -1 -1 -1 -1 -1

String alignment String alignment problemproblem Input:

– two strings A and B; and– a scoring table 分 .

Output: – an alignment of A and B that has the

maximum score with respect to 分 .

Q: Any naïve methods?Q: Any naïve methods? A = attgatcctag B = ccttagtccttcgc

分 a t g c -a 2 -2 -2 -2 -1t -2 2 -2 -2 -1g -2 -2 2 -2 -1c -2 -2 -2 2 -1- -1 -1 -1 -1 -1

Q: Is there a recursive Q: Is there a recursive method?method? A = attgatcctag B = ccttagtccttcgc

分 a t g c -a 2 -2 -2 -2 -1t -2 2 -2 -2 -1g -2 -2 2 -2 -1c -2 -2 -2 2 -1- -1 -1 -1 -1 -1

Yes, but very Yes, but very inefficient!inefficient!int align(m;n) fif (m= n = 0) return 0;let x = y = z = 1 ;if (m> 0 and n > 0)let x = align(m¡ 1;n ¡ 1) +Score[A[m];B[n]];

if (m> 0)let y = align(m¡ 1;n) +Score[A[m];¡ ];

if (n > 0)let z = align(m;n ¡ 1) +Score[¡ ;B[n]];

return max(x;y;z);g

Alignment graphAlignment graph

c gc t t a t c

ObservationsObservations Each alignment

corresponds to a maximal path on the alignment graph.

The score of an alignment is the score of its corresponding maximal path.

c gc t t a t c

c c t t - a g t ca - t t g a - - -

前無古人後無來者

Score of edgesScore of edges

B[j]分 [-, B[j]]

分 [A[i], -] 分 [A[i], B[j]]

The graph problem

Finding a maximal path with maximum score on the alignment graph (a directed acyclic graph)

IdeaIdea For each i = 0, 1,

…, |A| and each j = 0, 1,…, |B|, let 點[i, j] keep the maximum score of aligning A[1…i] and B[1…j].

1 j |B|

An observationAn observation

點 [i, j] = the maximum of – 點 [i-1, j-1] + 分 [A[i],

B[j]]– 點 [i-1, j] + 分 [A[i], -]– 點 [i, j-1] + 分 [-, B[j]]點 [i, j-1] 點 [i, j]

點 [i-1, j-1] 點 [i-1, j]

分 [-, B[j]]

分 [A[i], -]分 [A[i], B[j]]

For exampleFor example

0 -1 -2 -3 -4 -5 -6 -7 -8

-3 -4 -5 -2 1 0 -1 -2 -3

-1 -2 -3 -4 -5 -2 -3 -4 -5

-2 -3 -4 -1 -2 -3 -4 -1 -2

-4 -5 -6 -3 0 -1 2 1 0

-5 -6 -7 -4 -1 2 1 0 -1

c gc t t a t c 分 a t g c -a 2 -2 -2 -2 -1t -2 2 -2 -2 -1g -2 -2 2 -2 -1c -2 -2 -2 2 -1- -1 -1 -1 -1 -1

The DP-version.The DP-version.int align(m;n) flet C[0;0]= 0;for i = 1 to m let C[i;0]= C[i ¡ 1;0]+Score[A[i]; ¡ ];for j = 1 to n let C[0; j ]= C[0; j ¡ 1]+ScoreS[¡ ;B [j ]];for i = 1 to mfor j = 1 to n flet x = C[i ¡ 1; j ¡ 1]+Score[A[i];B [j ]];let y = C[i ¡ 1; j ]+Score[A[i]; ¡ ];let z = C[i; j ¡ 1]+Score[¡ ;B[j ]];let C[i; j ] = max(x;y;z);

greturn C[m;n];

ComplexityComplexity Space = O(|A|×|B|).

– Each node keeps a score and a pointer, and thus requires only O(1) space.

Time = O(|A|×|B|). – The content of each node can be obtained

from those of at most three nodes in O(1) time.

QuestionQuestion 剛剛只是算出最佳的成績 . 如果我們想要知道得到這個最佳成績的

alignment 應該如何處理？需要額外再花空間嗎？

For exampleFor example

0 -1 -2 -3 -4 -5 -6 -7 -8

-3 -4 -5 -2 1 0 -1 -2 -3

-1 -2 -3 -4 -5 -2 -3 -4 -5

-2 -3 -4 -1 -2 -3 -4 -1 -2

-4 -5 -6 -3 0 -1 2 1 0

-5 -6 -7 -4 -1 2 1 0 -1

c gc t t a t c 分 a t g c -a 2 -2 -2 -2 -1t -2 2 -2 -2 -1g -2 -2 2 -2 -1c -2 -2 -2 2 -1- -1 -1 -1 -1 -1

回顧來時徑

ComplexityComplexity Space = O(|A|×|B|).

– Each node keeps a score and a pointer, and thus requires only O(1) space.

Time = O(|A|×|B|). – The content of each node can be obtained

from those of at most three nodes in O(1) time.

Application 1

Longest common subsequence

SubsequenceSubsequence

For any indices 1 ≤ i1 < i2 < … <ik ≤ |A|, A[i1] A[i2] A[i3]…A[ik] is a subsequence of A.

For example, A = 0 1 1 0 1 0 1– 0 1 1 1, 0 0 0, and 1 0 1 0 1 are subsequences

of A.– 0 1 0 1 1 0 is not a subsequence of A.

Longest Common Longest Common SubsequenceSubsequence

Input: two strings A and B Output: a longest string C that is a

subsequence of both A and B.

Any naïve algorithm?

It’s an alignment It’s an alignment problem…problem… …with respect to the following scoring

matrix:

分 a t g c -a 1 0 0 0 0t 0 1 0 0 0g 0 0 1 0 0c 0 0 0 1 0- 0 0 0 0 0

Why?Why? Each alignment with score k corresponds

to a common subsequence of length k.

0 1 1 – 1 0 - - 0 1 1 - - 1 0 1 1 0 1 0 – 1 1 0 1 1 0 1 1

Application 2

Edit distance between two strings

Edit operationsEdit operations Inserting a character at position i Deleting a character at position i Replacing a character at position i by a

new character

Edit distanceEdit distance The edit distance between two strings A

and B is the minimum number of edit operations required to turn A into B.

The edit distance The edit distance problemproblem Input: two strings A and B Output: the edit distance of A and B.

Any naïve algorithm?

It’s an alignment It’s an alignment problem…problem… …with respect to the following scoring

matrix:

分 a t g c -a 0 -1 -1 -1 -1t -1 0 -1 -1 -1g -1 -1 0 -1 -1c -1 -1 -1 0 -1- -1 -1 -1 -1 -1

Why?Why? Each alignment with score -k corresponds

to a sequence of k edit operations that turns A into B.

0 1 1 – 1 0 - - 0 1 1 - - 1 0 1 1 0 1 0 – 1 1 0 -1 -1-1 -1-1-1 -1

d r i i i d i

Matrix multiplication

Multiplying two Multiplying two matricesmatricesLet A bea p£ qmatrix. Let B bea q£ r matrix. Then theproduct A £ B of A and B is thep£ r matrix M such that

M [i; j ]=X

1· k· qA[i;k]¢B[k;j ]

holds for all indices i and j with 1· i · p and 1· j · r.

It takesO(pqr) timeto obtain A £ B fromA and B. (Why?)

Multiplying 3 matricesMultiplying 3 matrices

A £ B £ C = (A £ B) £ C= A £ (B £ C):

The time required by obtaining A £ B £ C could be affectedby which twomatricesmultiply ¯rst.

An exampleAn example

n £ 1 1£ n n £ n

n £ n n £ n n £ n

£(n3)£ (n2)

Theoverall time is

£(n2)+£(n3) = £(n3):

An exampleAn example

n £ 1 1£ n n £ n

n £ n

£(n2)

n £ 1

£(n2)

Theoverall time is

£(n2)+£(n2) = £(n2):

The problemThe problem

F Input: A sequence 0; 1; : : : ; n of positive integers,whereI i ¡ 1 is thenumber of rows of matrix M i , andI i is thenumber of columns of matrix M i .

F Output: An order of performing those n ¡ 1 matrixmultiplications in theminimumnumber of operationsto obtain theproduct

M1 £ M2 £ ¢¢¢£ Mn:

Any naïve algorithm?Any naïve algorithm?

C(i,j)C(i,j)

Let C(i; j ) be the minimum number of operationsrequired to obtain theproduct

M i £ M i+1 £ ¢¢¢£ M j :

Clearly, theminimumnumber of operations requiredto obtain the product of all n matrices is exactlyC(1;n).

RecurrenceRecurrence

C(i; j )

=( 0 if i ¸ j

mini · k<j

(C(i;k) +C(k+1;j ) + i ¡ 1 k j ) otherwise:

Dynamic programmingDynamic programming

C 1 2 3 4 5 6 … n

如何找出矩陣相乘的順序？

ComplexityComplexity

F Timecomplexity: O(n3).F Spacecomplexity: O(n2).

I Can this be reduced to o(n2)?

Have a wonderful weekend!!

See you next time

資料結構與演算法 ( 上 )

Documents

Transcript of 資料結構與演算法 ( 上 )

複数個のJoinpoint 対象としたPointcut 記述可能なアスペクト ......Juggler はAspectJ の文法を基にして、pointcut 式に新しい演算子であ る連結演算子(##)

インテル(R) 数値演算ライブラリ(MKL) リファレンス・マニュアル · インテル® 数値演算ライブラリ －リファレンス・マニュアル－ 目次

いろいろな演算（四則演算） int以外の変数の型）yaoki/pro2/pro2-02.pdf · 2008-09-28 · 1 第2回 プログラミング入門2 2 演算と変数の型 いろいろな演算（四則演算）

量子計算の基礎Pauli演算子，ブロッホ球，Clifford演算子，non-Clifford演算, Solovay-Kitaevアルゴリズム，直積空間，CNOT演算，万能量子計算 量子状態

螞蟻演算法 (ant colony optimization, ACO)

mit 演算法 Lec2

FP - growth / FP 演算法簡介

5アナログ演算ブロック 5.1四則演算式の係数とバイアス … · 74 5アナログ演算ブロック 5.1四則演算式の係数とバイアスの求め方 規準化演算式の係数とバイアスの計算は大変間違い

資料結構 與 演算法

回溯演算法 (Backtracking)

第22 章章 演算法分析el.fotech.edu.tw/localuser/eetuml/web1/PG31221-DOC/ch02.pdf · 第22 章章 演算法分析 本章 學習目標. 理解資料結構是否優良與演算法的效能有關

第一章 資料結構導論 結束 2015/6/28 1 本章重點 1-1 認識資料與資訊 認識資料與資訊 1-2 解析演算法 (Algorithm) 解析演算法 (Algorithm) 1-3 程式

Asakusa Framework 演算子の処方箋

貪婪演算法 與 動態規劃演算法

字串演算法 String Algorithms 本次大綱 1 - 一些基本的字串演算法符號名詞介紹 2 - Rabin Karp 簡介 3 - 自動機的演算法簡介 4 - KMP 演算法本體介紹

第 7 章 2D 遊戲演算法

物理学情報処理演習 5. C++言語②変数・演算・制御文 - Kobe ...pp物理学情報処理演習 5. C++言語②変数・演算・制御文 5.1 変数 5.2 演算 5.3

資料結構與演算法 ( 下 )

1-1 · 演算法兩個運算思維開始。那什麼是資料結構與演算法，分別說 明如下： y 資料結構 什麼是資料結構呢？以解一元二次方程式 ax2+bx+c=0

基因演算法 2014-02-21

複数個のJoinpoint 対象としたPointcut 記述可能なアスペクト ......Juggler はAspectJ の文法を基にして、pointcut 式に新しい演算子である連結演算子(##)

インテル(R) 数値演算ライブラリ(MKL) リファレンス・マニュアル · インテル® 数値演算ライブラリ－リファレンス・マニュアル－目次

いろいろな演算（四則演算） int以外の変数の型）yaoki/pro2/pro2-02.pdf · 2008-09-28 · 1 第2回プログラミング入門2 2 演算と変数の型いろいろな演算（四則演算）

量子計算の基礎Pauli演算子，ブロッホ球，Clifford演算子，non-Clifford演算, Solovay-Kitaevアルゴリズム，直積空間，CNOT演算，万能量子計算量子状態

5アナログ演算ブロック 5.1四則演算式の係数とバイアス … · 74 5アナログ演算ブロック 5.1四則演算式の係数とバイアスの求め方規準化演算式の係数とバイアスの計算は大変間違い

資料結構與演算法

第22 章章演算法分析el.fotech.edu.tw/localuser/eetuml/web1/PG31221-DOC/ch02.pdf · 第22 章章演算法分析本章學習目標. 理解資料結構是否優良與演算法的效能有關

第一章資料結構導論結束 2015/6/28 1 本章重點 1-1 認識資料與資訊認識資料與資訊 1-2 解析演算法 (Algorithm) 解析演算法 (Algorithm) 1-3 程式

貪婪演算法與動態規劃演算法

1-1 · 演算法兩個運算思維開始。那什麼是資料結構與演算法，分別說明如下： y 資料結構什麼是資料結構呢？以解一元二次方程式 ax2+bx+c=0