1

Graphs: shortest paths & Minimum Spanning Tree(MST)

15-211 Fundamental Data Structures and Algorithms

Ananda Guna

April 8, 2003

2

Announcements

Homework #5 is due Tuesday April 15th.

Quiz #3 feedback is enabled.

Final Exam is Tuesday May 8th at 8AM

Recap

4

Dijkstra’s algorithm

S = {1}

for i = 2 to n do D[i] = C[1,i] if there is an edge from 1 to i, infinity otherwise

for i = 1 to n-1

{ choose a vertex w in V-S such that D[w] is min

add w to S (where S is the set of visited nodes)

for each vertex v in V-S do

D[v] = min(D[v], D[w]+c[w,v])

}

Where |V| = n

5

Features of Dijkstra’s Algorithm

•A greedy algorithm

•“Visits” every vertex only once, when it becomes the vertex with minimal distance amongst those still in the priority queue

•Distances may be revised multiple times: current values represent ‘best guess’ based on our observations so far

•Once a vertex is visited we are guaranteed to have found the shortest path to that vertex…. why?

6

Correctness (via contradiction)

•D(x) must represent a shortest path to x, and D(x) Dshortest(u).

•However, Dijkstra’s always visits the vertex with the smallest distance next, so we can’t possibly visit u before we visit x

u

xsvisited

unvisited

• Prove D(u) represent the shortest path to u (visited node)

• Assume u is the first vertex visited such that D(u) is not a shortest path (thus the true shortest path to u must pass through some unvisited vertex)

• Let x represent the first unvisited vertex on the true shortest path to u

7

Quiz break

Would it be better to use an adjacency list or an adjacency matrix for Dijkstra’s algorithm?

What is the running time of Dijkstra’s algorithm, in terms of |V| and |E| in each case?

8

Complexity of Dijkstra

Adjacency matrix version Dijkstra finds shortest path from one vertex to all others in O(|V|2) time

If |E| is small compared to |V|2, use a priority queue to organize the vertices in V-S, where V is the set of all vertices and S is the set that has already been explored

So total of |E| updates each at a cost of O(log |V|)

So total time is O(|E| log|V|)

Negative Weighted Single-SourceShortest Path Algorithm

(Bellman-Ford Algorithm)

10

The Bellman-Ford algorithm

(see Weiss, Section 14.4)

Returns a boolean:•TRUE if and only if there is no negative-weight

cycle reachable from the source: a simple cycle <v0, v1,…,vk>, where v0=vk and

•FALSE otherwise

If it returns TRUE, it also produces the shortest paths

k

iii vvweight

11 0),(

11

Example

For each edge (u,v), let's denote its length by C(u,v))

Let d[i][v] = distance from start to v using the shortest path out of all those that use i or fewer edges, or infinity if you can't get there with <= i edges.

12

Example ctd..

How can we fill out the rows?

0 1 2 3 4 5

0 0 1 0 50 15 2 0 50 80 15 45

V

i

13

Example ctd..

Can we get ith row from i-1th row?

for v != start,

d[v][i] = MIN d[x][i-1] + len(x,v)

x->v

We know minimum path to come to x using < i nodes.So for all x that can reach v, find the minimum such sum (in blue) among all x

Assume d[start][i] = 0 for all i

14

Completing the table

0 1 2 3 4 5

0 0 1 0 50 15 2 0 50 80 15 45 3 0 25 80 15 45 75

4 0 25 55 15 45 755 0 25 55 15 45 65

d[v][i] = MIN d[x][i-1] + len(x,v)

x->v

15

Key features

• If the graph contains no negative-weight cycles reachable from the source vertex, after |V| - 1 iterations all distance estimates represent shortest paths…why?

16

Correctness

By induction:• D(s) = 0 after initialization• Assume D(vi-1) is a shortest path after iteration (i-1)• Since edge (vi-1,vi) is updated on the ith pass, D(vi)

must then reflect the shortest path to vi.• Since we perform |V| - 1 iterations, D(vi) for all

reachable vertices vi must now represent shortest paths

The algorithm will return true because on the |V|th iteration, no distances will change

Case 1: Graph G=(V,E) doesn’t contain any negative-weight cycles reachable from the source vertex s

Consider a shortest path p = < v0, v1,..., vk>, which must have k |V| - 1 edges

17

Correctness

Proof by contradiction:

Assume the algorithm returns TRUEThus, D(vi-1) + weight(vi-1, vi) D(vi) for i = 1,…,k

Summing the inequalities for the cycle:

leads to a contradiction since the first sums on each side are equal (each vertex appears exactly once) and the sum of weights must be less than 0.

k

ii

k

iii

k

ii vDvvweightvD

111

11 )(),()(

Case 2: Graph G=(V,E) contains a negative-weight cycle < v0, v1,..., vk> reachable from the source vertex s

18

Performance

Initialization: O(|V|)

Path update and cycle check: |V| calls checking |E| edges, O(|VE|)

Overall cost: O(|VE|)

The All PairsShortest Path Algorithm

(Floyd’s Algorithm)

20

Finding all pairs shortest paths

Assume G=(V,E) is a graph such that c[v,w] 0, where C is the matrix of edge costs.

Find for each pair (v,w), the shortest path from v to w. That is, find the matrix of shortest paths

Certainly this is a generalization of Dijkstra’s.

Note: For later discussions assume |V| = n and |E| = m

21

Floyd’s Algorithm

A[i][j] = C(i,j) if there is an edge (i,j)

A[i][j] = infinity(inf) if there is no edge (i,j)

Graph

“adjacency” matrix

A is the shortest path matrix that uses 1 or fewer edges

22

Floyd ctd..

To find shortest paths that uses 2 or fewer edges find A2, where multiplication defined as min of sums instead sum of products

That is (A2)ij = min{ Aik + Akj | k =1..n}

This operation is O(n3)

Using A2 you can find A4 and then A8 and so on

Therefore to find An we need log n operations

Therefore this algorithm is O(log n* n3)

We will consider another algorithm next

23

Floyd-Warshall Algorithm

This algorithm uses nxn matrix A to compute the lengths of the shortest paths using a dynamic programming technique.

Let A[i,j] = c[i,j] for all i,j & ij

If (i,j) is not an edge, set A[i,j]=infinity and A[i,i]=0

Ak[i,j] =

min (Ak-1[i,j] , Ak-1[i,k]+ Ak-1[k,j])

Where Ak is the matrix after k-th iteration and path from i to j does not pass through a vertex higher than k

24

Example – Floyd-Warshall Algorithm

1 2 3

8 2

3

5

Find the all pairs shortest path matrix

Ak[i,j] =

min (Ak-1[i,j] , Ak-1[i,k]+ Ak-1[k,j]) Where Ak is the matrix after k-th iteration and path from i to j does not pass through a vertex higher than k

25

Floyd-Warshall Implementation

initialize A[i,j] = C[i,j]

initialize all A[i,i] = 0

for k from 1 to n

for i from 1 to n

for j from 1 to n

if (A[i,j] > A[i,k]+A[k,j])

A[i,j] = A[i,k]+A[k,j];

The complexity of this algorithm is O(n3)

26

Questions

Question: What is the asymptotic run time of Dijkstra (adjacency matrix version)?

O(n2)

Question: What is the asymptotic running time of Floyd-Warshall?

27

Minimum Spanning Trees(some material adapted from slides by Peter Lee)

28

Problem: Laying Telephone Wire

Central office

29

Wiring: Naïve Approach

Central office

Expensive!

30

Wiring: Better Approach

Central office

Minimize the total length of wire connecting the customers

31

Minimum Spanning Tree (MST)(see Weiss, Section 24.2.2)

it is a tree (i.e., it is acyclic)

it covers all the vertices V

contains |V| - 1 edges

the total cost associated with tree edges is the minimum among all possible spanning trees

not necessarily unique

A minimum spanning tree is a subgraph of an undirected weighted graph G, such that

32

Applications of MST

Any time you want to visit all vertices in a graph at minimum cost (e.g., wire routing on printed circuit boards, sewer pipe layout, road planning…)

Internet content distribution$$$, also a hot research topic

Idea: publisher produces web pages, content distribution network replicates web pages to many locations so consumers can access at higher speed

MST may not be good enough! content distribution on minimum cost tree may take

a long time!

33

How Can We Generate a MST?

a

ce

d

b2

45

9

6

4

5

5

a

ce

d

b2

45

9

6

4

5

5

34

Prim’s Algorithm

Let V ={1,2,..,n} and U be the set of vertices that makes the MST and T be the MST

Initially : U = {1} and T = while (U V)

let (u,v) be the lowest cost edge such that

u U and v V-U

T = T {(u,v)}

U = U {v}

35

Prim’s Algorithm implementation

Initializationa. Pick a vertex r to be the root

b. Set D(r) = 0, parent(r) = nullc. For all vertices v V, v r, set D(v) = d. Insert all vertices into priority queue P, using distances as the keys

a

ce

d

b2

45

9

6

4

5

5

e a b c d

0

Vertex Parent e -

36

Prim’s Algorithm

While P is not empty:

1. Select the next vertex u to add to the treeu = P.deleteMin()

2. Update the weight of each vertex w adjacent to u which is not in the tree (i.e., w P) If weight(u,w) < D(w),

a. parent(w) = u b. D(w) = weight(u,w)

c. Update the priority queue to reflect new distance for w

37

Prim’s algorithm

a

ce

d

b2

45

9

6

4

5

5

d b c a

4 5 5

Vertex Parente -b ec ed e

The MST initially consists of the vertex e, and we updatethe distances and parent for its adjacent vertices

38

Prim’s algorithm

a

ce

d

b2

45

9

6

4

5

5

a c b

2 4 5

Vertex Parente -b ec dd ea d

39

Prim’s algorithm

a

ce

d

b2

45

9

6

4

5

5

c b

4 5

Vertex Parente -b ec dd ea d

40

Prim’s algorithm

a

ce

d

b2

45

9

6

4

5

5

b

5

Vertex Parente -b ec dd ea d

41

Prim’s algorithm

Vertex Parente -b ec dd ea d

a

ce

d

b2

45

9

6

4

5

5

The final minimum spanning tree

42

Prim’s Algorithm Invariant

At each step, we add the edge (u,v) s.t. the weight of (u,v) is minimum among all edges where u is in the tree and v is not in the tree

Each step maintains a minimum spanning tree of the vertices that have been included thus far

When all vertices have been included, we have a MST for the graph!

43

Running time of Prim’s algorithm

Initialization of priority queue (array): O(|V|)

Update loop: |V| calls• Choosing vertex with minimum cost edge: O(|V|)• Updating distance values of unconnected

vertices: each edge is considered only once during entire execution, for a total of O(|E|) updates

Overall cost: O(|E| + |V| 2)

44

Another Approach – Kruskal’s

a

ce

d

b2

45

9

6

4

5

5

• Create a forest of trees from the vertices• Repeatedly merge trees by adding “safe edges”

until only one tree remains• A “safe edge” is an edge of minimum weight which

does not create a cycle

forest: {a}, {b}, {c}, {d}, {e}

45

Kruskal’s algorithm

Initializationa. Create a set for each vertex v Vb. Initialize the set of “safe edges” A

comprising the MST to the empty setc. Sort edges by increasing weight

a

ce

d

b2

45

9

6

4

5

5

{a}, {b}, {c}, {d}, {e}A = E = {(a,d), (c,d), (d,e), (a,c), (b,e), (c,e), (b,d), (a,b)}

46

Kruskal’s algorithm

For each edge (u,v) E in increasing order while more than one set remains:

If u and v, belong to different sets a. A = A {(u,v)} b. merge the sets containing u and v

Return A

Use Union-Find algorithm to efficiently determine if u and v belong to different sets

47

Kruskal’s algorithm

E = {(a,d), (c,d), (d,e), (a,c), (b,e), (c,e), (b,d), (a,b)}

Forest{a}, {b}, {c}, {d}, {e}{a,d}, {b}, {c}, {e}{a,d,c}, {b}, {e}{a,d,c,e}, {b}{a,d,c,e,b}

A{(a,d)}{(a,d), (c,d)}{(a,d), (c,d), (d,e)}{(a,d), (c,d), (d,e), (b,e)}

a

ce

d

b2

45

9

6

4

5

5

48

After each iteration, every tree in the forest is a MST of the vertices it connects

Algorithm terminates when all vertices are connected into one tree

Kruskal’s Algorithm Invariant

49

Greedy Approach

Like Dijkstra’s algorithm, both Prim’s and Kruskal’s algorithms are greedy algorithms

The greedy approach works for the MST problem; however, it does not work for many other problems!

50

Thursday

P vs NP

Models of Hard Problems

Work on Homework 5