LN12_Optimization.pdf

1NTUEEAlgorithms

Algorithms演算法

Professor Chien-Mo James Li 李建模

Graduate Institute of Electronics EngineeringNational Taiwan University

Useful Combinatorial Optimization Algorithms *not in exam

2NTUEEAlgorithms

Outline• Introduction

• Exponential time Algorithms

♦ Exhaustive search / Branch and Bound

• Mathematical Programming

• Randomized Algorithms

♦ Simulated annealing

♦ Genetic algorithm

• Heuristic

♦ Tseng and Siewiorek Algorithm (minimum clique partition)

♦ Quine-McCluskey Algorithm (minimum set covering)

3NTUEEAlgorithms

Combinatorial Optimization• Combinatorial Optimization

♦ Find an optimal solution from a finite set of objects♦ instances and solutions are discrete

∗ exhaustive search is often not feasible• Continuous optimization

♦ Input instance and the solution set can be continuous

• Examples of Combinatorial Optimization♦ minimum spanning tree problem (polynomial time)♦ traveling salesman problem (NP-hard)

• Applications of Combinatorial Optimization♦ Electronic Design Automation♦ Control♦ Operational research♦ Artificial intelligence♦ ….

4NTUEEAlgorithms

Decision Problems vs. Optimization Problems• Decision problems: find answers to yes/no questions

♦ MST: Given a graph G=(V, E) and a bound K, is there a spanning tree with a cost at most K?

♦ TSP: Given cities, distance between each pair of cities, and a bound B, is there a route that starts and ends at a given city, visits every city exactly once, and has total distance at most B ?

• Optimization problems: find a legal answer to minimize cost

♦ MST: Given a graph G=(V, E), find the cost of a minimum spanning tree of G

♦ TSP: Given a set of cities and that distance between each pair of cities, find the distance of a “minimum route” starts and ends at a given city and visits every city exactly once

5NTUEEAlgorithms

NP-complete and NP-Hard• A problem L is NPC if both conditions are true

1. L ∈ NP

2. L' ≤ P L for every L' ∈ NP

• A problem L is NP-hard if this condition is true

1. L' ≤ P L for every L' ∈ NP

• NP-hard problems are more difficult than NPC

• Example: TSP

♦ decision problem is NPC,

♦ optimization problem is NP-hard

[Wikipedia]

6NTUEEAlgorithms

Coping with NPC, NP-hard Problems• Approximation algorithms:

♦ Guarantee to be a fixed percentage away from the optimum

• Exponential algorithms

♦ Branch and Bound/Exhaustive search

♦ (Feasible only when the problem size is small)

• Mathematical programming:

♦ Integer Linear programming , 0/1 ILP

• Local search:

♦ Simulated annealing (hill climbing), genetic algorithms, etc

♦ Randomized algorithms: Make use of a randomizer for operation

• Heuristics: No formal guarantee of performance

• Others:

♦ Pseudo-polynomial time algorithms: e.g. Dynamic programming for 0-1 Knapsack problem

♦ Probabilistic algorithms: Assume some probabilistic distribution of the instances.

♦ Restriction: Work on some special cases of the original problem.

7NTUEEAlgorithms








• Heuristic

8NTUEEAlgorithms 8

Exhaustive search• General principle

♦ Systematically assign values to unspecified variables until

∗ a single point in search space is identified , or

∗ An implicit constraints makes it impossible to continue

− backtracking

• Example: TSP

♦ X means backtracking

9NTUEEAlgorithms 9

Branch and Bound• General principle

♦ Estimate the cost lower bound

♦ Kill partial solutions higher than the lowest cost

• Example: TSP

♦ Use MST as cost lower bound

♦ E.g. A B C F = 22; MST of {DEFA} = 9

∗ 22+9 > 27 Killed

10NTUEEAlgorithms 10

Exhaustive Search vs. Branch and Bound

exhaustive: 72 nodes!

B&B: only 27 nodes


Pseudo Code of B&B• Disadvantage:

♦ Finding lower bound is not easy all the time

♦ Still too many branches when problem is large

12NTUEEAlgorithms








• Heuristic


Linear Programming• General principle

♦ Convert problems into a mathematical LP problem

♦ Canonical form of LP

∗ AX≤b

∗ X≥0

• Integer Linear Programming (ILP)

♦ Variables are restricted to integers

• 0-1 ILP

♦ Solutions are restricted to 0,1

• Why ILP useful for optimization?

♦ Problem independent

♦ ILP solvers are widely available

∗ Runs fast in most cases (although worst case not polynomial)


ILP Example : TSP (optimization version)

• Variables x1 … x12

♦ Xi = 1 means the edge is traveled♦ Xi = 0 means the edge is not traveled

• Minimize cost of travel

• Subject to constraints♦ Every vertex has exactly two edges traveled

♦ are these constraints enough?

2:

2:

2:

2:

2:

2:

1211966

1210835

1110744

98523

76512

43211

=+++=+++=+++=+++=+++=+++

xxxxv

xxxxv

xxxxv

xxxxv

xxxxv

xxxxv

∑=

12

1

)(i

ii xew


TSP: Subtour Exclusion Constraints(1)

• Multiple disjoint tour should be prevented

• Add more constraints to avoid multiple disjoint tour

2:},,}{,,{

2:},,}{,,{

2:},,}{,,{

11106532653421

12109541642531

987643654321

≥+++++≥+++++≥+++++

xxxxxxvvvvvv

xxxxxxvvvvvv

xxxxxxvvvvvv

16NTUEEAlgorithms

TSP: Subtour Exclusion Constraints(2)

• Type I: for a non-trivial subset S of vertex {1, 2, ... n}

• Type II:

♦ Type I&II: number of constraints grows exponentially with n

• Type III: add aux variables ui = (2, 3, ...n)

♦ every vertex i has one unique number ui

♦ Number of constraints: n2

17NTUEEAlgorithms








• Heuristic

♦ Quine-McCluskey for SOP minimization

18NTUEEAlgorithms

Local Search• General principle

♦ Search only the neighbors of the current solution

♦ Move to the neighbor with lower cost than the current solution

• Problem

♦ Can be trapped in local minimum

fg

19NTUEEAlgorithms

Simulated Annealing (SA)• General Principle: mimic material cooling process

♦ Search the neighbors of current solution

♦ A good move means the cost decreases

∗ always accepted

♦ A bad move means the cost increases

∗ Accepted with probability exp (-∆cost/T)

• Analogy

♦ Energy = cost function

♦ Temperature = controlling parameter T

• Advantage: can escape local minimum

♦ ‘Uphill climbing’ is possible

f

20NTUEEAlgorithms

Pseudo Code of SA

f

21NTUEEAlgorithms

Tabu Search• General Principle: avoid staying in the taboo solutions

♦ Keep a list of visited solutions : taboo list

♦ Always move to a new solution other than the taboo list

∗ even the new solution is poor than the current solution

22NTUEEAlgorithms

Genetic Algorithms (GA)• General Principle

♦ Survival of the fittest (s)

♦ Keep a group of feasible solutions

∗ ‘population’

♦ ‘Parent’ population generates the ‘child’ population

∗ Keep only the best children

23NTUEEAlgorithms

Important steps in GA• Cross over: Two feasible solutions generate their child by switching

chromosomes

• Mutation: some chromosomes can change by probability

24NTUEEAlgorithms

Pseudo Code of GA

25NTUEEAlgorithms








• Heuristic

♦ Tseng and Siewiorek Algorithm (minimum clique partition)


* Heuristic refers to experience-based techniques for problem solving, where the exhaustive search is impractical. [Wikipedia]

26NTUEEAlgorithms

Test Cube Compaction Problem• Problem: given a set of test cubes {1, 0, X}

• Output: a minimum set of compacted test cubes

• Two test cubes are compatible iff

♦ no position is 0 in one cube and 1 in the other cube

• Compatible test cubes can be merged into one test cube

♦ merged test cubes has X elements only where both cubes are X

• Example: a set of 4 test cubes

♦ t4+t5, t3+t6, t0+ t1+t2, t7

• Any better solution?

t0 0xx10

t1 0xx1x

t2 0x01x

t3 01xx0

t4 x0xx0

t5 1xxxx

t6 x1x00

t7 11xx0

27NTUEEAlgorithms

Compatibility Graph• Compatibility graph G(V, E)

♦ vertex vi represents a test cube ti

♦ edge eij between vi and vj means two test cubes are compatible

♦ test cubes in a clique can be merged into one test cube

• Example

• This is a minimum clique partition problem

♦ NP-hard

t0 0xx10

t1 0xx1x

t2 0x01x

t3 01xx0

t4 x0xx0

t5 1xxxx

t6 x1x00

t7 11xx0

28NTUEEAlgorithms

Tseng and Siewiorek Algorithm

keep those edges that do not touch vi and vj

keep edges of common neighbors

Q: why remove other edges?

GKC = compatibility graph in Kth iteration

Vc = set of vertex in GKC

Ec = set of edges in GKC

29NTUEEAlgorithms

• combine vertices step by step

• select two vertices of maximum common neighbors

• supervertices is a set of vertices

♦ e.g., combine v0 v1 v0,1 supervertices

T & S Example

t0,1,2,3 01010

t4 x0xx0

t5, 6, 7 11x00

30NTUEEAlgorithms








• Heuristic

♦ Tseng and Siewiorek Algorithm (maximum clique )


31NTUEEAlgorithms

Set-Covering Problem• Given a finite set X, and a family F of subsets of X

• subset S∈ F covers its elements

• Set covering problem: find a minimum number of subset C⊆ F that cover all X

• Optimization problem

♦ minimum size of C

• Decision problem

♦ exist a covering C ≤ size k?

♦ exercise: prove set covering is NPC (Hint: use vertex covering)

• Applications of Set-Covering

♦ find minimum of people (C) that perform all tasks (X)

♦ minimum 2-level combinational circuit design

UFS

S X∈

=

UCS

S X∈

=

32NTUEEAlgorithms

Greedy-Set-Cover• Each iteration chooses the subset that covers most elements

♦ running time O(|X||F| min(|X|,|F|))

• example: |X|=12, |F|=6

♦ greedy solution

∗ {S1, S4, S5, S6} size =4

♦ optimal solution

∗ C={S3, S4, S5} size=3

• (Corollary 35.5) GREEDY-SET-COVER is a polynomial time (ln|X|+1)-approximation algorithm

33NTUEEAlgorithms

SOP Minimization * not in exam

• Two-level AND-OR Combinational Circuit

• Sum of product (SOP)

♦ f = a’c’d + bc’d + bcd’ + acd’

• Disjunctive normal form (DNF)

♦ f=(¬a∧¬c ∧d)∨(b∧¬c ∧d)∨(b∧c ∧ ¬ d)∨(a∧c ∧¬d)

• Very important EDA problem:

♦ Given f, find minimize number of gates

♦ if gates are the same, minimize number of inputs

34NTUEEAlgorithms

Quine-McCluskey Algorithm• Invented in 1956, one of the earliest EDA algorithms

• Two-step process

♦ 1. Generate all Prime Implicants (PI)

∗ PI = product term that cannot be combined with other terms

♦ 2. Select minimum number of PI

∗ set-covering

35NTUEEAlgorithms

• f(abcd)=Σm(4,5,6,8,9,10,13)+d(0,7,15)

♦ at least ONE of the following terms must be covered

∗ a’bc’d’ + a’bc’d + a’bcd’ + ab’c’d’ + ab’c’d + ab’cd’ + abc’d’

∗ called On-set

♦ a’b’c’d’+ab’c’d’+a’b’c’d’ may or may not be covered

∗ called Don’t-care set

♦ None of the others can be covered

∗ called Off-set

Prime Implicant Generation (1/5)

abcd 00 01 11 10

00

01

11

10

0

1

0

1

10

X

0

0

1

1

1

0

X

1

X

a

d

b

c

36NTUEEAlgorithms

Prime Implicant Generation (2/5)Implicant Table

Column I

0000

0100

1000

0101

0110

1001

0111

1101

1111

1010

zero “1”

one “1”

two “1”

three “1”

four “1”

37NTUEEAlgorithms


Column I

0000 |

0100 |

1000 |

0101 |

0110 |

1001 |

0111 |

1101 |

1111 |

1010 |

Column II

0-00

-000

010-

01-0

100-

10-0

01-1

-101

011-

1-01

-111

11-1

√

√√

√

√

√

√

√√

√

- are don’t cares

38NTUEEAlgorithms


Column I

0000 |

0100 |

1000 |

0101 |

0110 |

1001 |

0111 |

1101 |

1111 |

1010 |

Column II

0-00 *

-000 *

010- |

01-0 |

100- *

10-0 *

01-1 |

-101 |

011- |

1-01 *

-111 |

11-1 |

Column III

01-- *

-1-1 *

√

√

√

√

√

√

√

√√

√

√

√

√

√

√

√

√

7 Prime Implicants:0-00 = a'c'd'100- = ab'c'1-01 = ac'd-1-1 = bd -000 = b'c'd'10-0 = ab'd'01- - = a'b

39NTUEEAlgorithms

Karnaugh Map

abcd

00 01 11 10

00

01

11

10

0

1

0

1

10

X

0

0

1

1

1

0

X

1

X

a

d

b

c

7 Prime Implicants : 0-00 = a'c'd'100- = ab'c'1-01 = ac'd-1-1 = bd

-000 = b'c'd' 10-0 = ab'd' 01-- = a'b

• PI on Karnaugh map

♦ see logic design textbook

40NTUEEAlgorithms

PI Table 4 5 6 8 9 10 13

0,4 (0-00)

0,8 (-000)

8,9 (100-)

8,10 (10-0)

9,13 (1-01)

4,5,6,7 (01- -)

5,7,13,15 (-1-1)

rows = prime implicants

columns = ON-set elements

“X" means ON-set element is covered by the prime implicant

41NTUEEAlgorithms

Essential Rows and Distinguished Column4 5 6 8 9 10 13

0,4 (0-00)

0,8 (-000)

8,9 (100-)

8,10 (10-0)

9,13 (1-01)

4,5,6,7 (01- -)

5,7,13,15 (-1-1)

If column has a single X, it is distinguished.The implicant associated with the row is essential.

Essential rows must appear in the minimum cover .

essential rows

distinguished columns

42NTUEEAlgorithms

Select Essential Rows

4 5 6 8 9 10 13

0,4 (0-00)

0,8 (-000)

8,9 (100-)

8,10 (10-0)

9,13 (1-01)

4,5,6,7 (01- -)

5,7,13,15 (-1-1)

Eliminate all columns covered by essential rows

43NTUEEAlgorithms

Final Result

4 5 6 8 9 10 13

0,4 (0-00)

0,8 (-000)

8,9 (100-)

8,10 (10-0)

9,13 (1-01)

4,5,6,7 (01- -)

5,7,13,15 (-1-1)

Find minimum set of rows that cover the remaining columns

f = ab'd' + ac'd + a'b

44NTUEEAlgorithms

FFT• Can Quine-McCluskey find optimal solution for all cases?

♦ obviously not ... set-covering is NPC

• Give an example where PI table has neither essential rows nor distinguished columns

♦ Quine-McCluskey has trouble finding optimal solution

LN12_Optimization.pdf

Documents

Transcript of LN12_Optimization.pdf