Complexity Classes
Complexity Classes
What is a problem?
Given: a set of variables, a set of values that are possible for those variables, a set of constraints.Question: does there exist an assignment of values to the variables that satisfies the constraints?
Complexity Classes
What is a problem?
Given: a set of variables, a set of values that are possible for those variables, a set of constraints.Question: does there exist an assignment of values to the variables that satisfies the constraints?
What happened to objective functions?
Complexity Classes
What is a problem?
Given: a set of variables, a set of values that are possible for those variables, a set of constraints.Question: does there exist an assignment of values to the variables that satisfies the constraints?
What is the size of the input? A minimal encoding under a given alphabet, usually {0,1} Example: the number 25 could be written like this 11111111111111111111111110 or it could be written like this: 11001
Complexity Classes
What does it mean for a problem to be solved in polynomial time?
There exists an algorithm whose running time on an instance of size n is bounded by nk for some constant k.
Complexity Classes
What does it mean for a problem to be solved in polynomial time?
There exists an algorithm whose running time on an instance of size n is bounded by nk for some constant k.
Consider the problem of determining whether a numberis prime. An algorithm considered earlier takes time n0.5
Is this a polynomial time algorithm?
Complexity Classes
What does it mean for a problem to be solved in polynomial time?
There exists an algorithm whose running time on an instance of size n is bounded by nk for some constant k.
Consider the problem of determining whether a numberis prime. An algorithm considered earlier takes time n0.5
Is this a polynomial time algorithm? No Why? Length of input: log(n) = L Running time: n0.5 = (2log(n))0.5 = 20.5L
Complexity Classes
What does it mean for a problem to be solved in polynomial time?
There exists an algorithm whose running time on an instance of size n is bounded by nk for some constant k.
Consider the problem of finding the greatest numberin a list of n numbers. An algorithm considered earlier takes time bounded by n Is this a polynomial time algorithm?
Complexity Classes
What does it mean for a problem to be solved in polynomial time?
There exists an algorithm whose running time on an instance of size n is bounded by nk for some constant k.
Consider the problem of finding the greatest numberin a list of n numbers. An algorithm considered earlier takes time bounded by n Is this a polynomial time algorithm? Yes Why? Length of input: nlog(n) = L Running time: n = L/log(n) < L
Complexity Classes
Where do optimization problems fit in?
Optimization problem: find an assignment that Satisfies constraints and for which some objective Function is minimum (or maximum)
Optimization problem revised: the objective function becomes a constraint and the question is does there exist an assignment for which the objective function is less than k?
Complexity Classes
Where do optimization problems fit in?
Optimization problem: find an assignment that Satisfies constraints and for which some objective Function is minimum (or maximum)
Optimization problem revised: the objective function becomes a constraint and the question is does there exist an assignment for which the objective function is less than k?
Example: Min Cost Network – Old: Given graph G with edge weights, find a spanning tree of least total weight New: Given graph G with edges weights and a number k, does there exist a spanning tree of total weight ≤ k?
Complexity Classes
How does this help anything?
There is always an upper bound on the objective function Example: Min Cost Network – no greater than n max(w
i)
Proceed with binary search for the answer starting at the maximum value.
For Min Cost Network: result is obtained in log(n max(wi))
reruns of the algorithm. Total running time: nlog(n)log(n max(w
i))
= nlog(n)log(n) + nlog(n)log( max(wi))
Length of input: nlog(max(wi)) = L
Total running time: nlog2(n) + Llog(n) ≤ Llog2(L) + 2L Conclude: polynomial complexity is preserved
Complexity Classes
A problem instance: Some sort of structure (e.g. Graph, Boolean formula, list)
A problem: Some constraints that must hold for any structure of some specific type. Is there an assignment of values to the variables of a given instance obeying the specified structure of a problem for which all constraints hold?
Worstcase running time: An upper bound on the number of operations needed for an algorithm to solve a problem as a function of input length, assuming the most efficient encoding.
Complexity Classes
The complexity class P: A problem X (variables, constraints, yes/no result) is in complexity class P if and only if there is an algorithm which takes an instance of X as input, always correctly answers yes/no depending on whether the constraints can be satisfied, and has a worstcase running time that is bounded from above by a polynomial on the length of the instance.
Complexity Classes
Examples of problems in class P: Minimum Cost Network (greedy) Integer Deadline Scheduling (greedy) Single source shortest path (greedy) Huffman tree encoding (greedy) System of linear equality constraints (Gaussian elimination) System of linear constraints and objective function (Lin Prog) Biconnected components of a graph (depth first search) Graph connectedness (depth first search) Maximum matching in a graph (Edmond’s algorithm) Horn satisfiability (binary constraint propagation) 2SAT (binary constraint propagation)
Not an example: Knapsack: O(nm) dynamic programming not poly time
Complexity Classes
Reasonable statement of tractability: A problem is considered tractable if it is a member of complexity class P
Complexity Classes
Suppose God were helping us: Suppose God gives us a solution and all we have to do is check that the solution satisfies all the constraints. If we can make the check in polynomial time, the problem is said to be nondeterministic polynomial time solvable or in the class NP (all bets off for nonsolution)
Complexity Classes
Suppose God were helping us: Suppose God gives us a solution and all we have to do is check that the solution satisfies all the constraints. If we can make the check in polynomial time, the problem is said to be nondeterministic polynomial time solvable or in the class NP (all bets off for nonsolution)
Example: Satisfiability: does assignment 000100 satisfy?
Vertex cover: is 1,3,5,7 a vertex cover of size 4?
2
3
4
5
1
2
3
4
56
7
Complexity Classes
What does it mean to be the “hardest” problem in the world?
YX
1
2
3
4
56
7
∃ a vertex cover ofsize 5?
T = O(nk) algorithmto translate X to Y
∃ assignment of values tov
0... that causes formula
to evaluate to 1?YES iff YES
Complexity Classes
YX
T = O(nk) algorithmto translate X to Y
∃ a O(nk) algorithm for Y → ∃ a O(nk) algorithm for X
Hence X is no harder than Y
What does it mean to be the “hardest” problem in the world?
Complexity Classes
YX
T = O(nk) algorithmto translate X to Y
∃ a O(nk) algorithm for X → ∃ a O(nk) algorithm for Y
But Y can be harder than X
What does it mean to be the “hardest” problem in the world?
Complexity Classes
Y
X1
NPcomplete: the hardest problems in the world (for us, anyway) A problem is NPcomplete if it is a member of NP and there is a polynomial time translation from every NP problem to it
X2
X3 . . .
Complexity Classes
Examples of problems in class NPcomplete: Steiner Tree Integer Linear Programming Bin Packing Most Scheduling Problems Most Transportation Problems Satisfiability Deadlock Avoidance Production Planning Most Motion Planning Problems Traveling Salesman (Hamiltonian Cycle) Dynamic Storage Allocation
Not an example: Eulerian Path – visit every edge exactly once
Complexity Classes
coNP: A problem whose complement is in NP
Example: Satisfiability: given a CNF formula f, does there not exist an assignment that satisfies f ? Minimum Cost Network: given graph with edge weights is there no spanning network of weight k?
Complexity Classes
coNP: A problem whose complement is in NP
Example: Satisfiability: given a CNF formula f, does there not exist an assignment that satisfies f ? Minimum Cost Network: given graph with edge weights is there no spanning network of weight k?
Observe: Since there is a polytime algorithm for MCN, just reverse the answer to solve the complementary problem
Complexity Classes
coNP: A problem whose complement is in NP
Example: Satisfiability: given a CNF formula f, does there not exist an assignment that satisfies f ? Minimum Cost Network: given graph with edge weights is there no spanning network of weight k?
Observe: Since there is a polytime algorithm for MCN, just reverse the answer to solve the complementary problem
But for Satisfiability, we cannot rely on God for an answer so determining that f is unsatisfiable may be really hard
Complexity Classes
coNPcomplete: A problem is coNPcomplete if it is a member of coNP and there is a polynomial time translation from every coNP problem to it
Y
X1
X2
X3 . . .
Complexity ClassesWhat we know: A problem that is in P is also in coNP A single problem in P that is NPcomplete means P=NP
Complexity ClassesWhat we know: A problem that is in P is also in coNP A single problem in P that is NPcomplete means P=NP If P=NP then all hell breaks loose (a name goes up in lights)
Complexity ClassesWhat we know: A problem that is in P is also in coNP A single problem in P that is NPcomplete means P=NP If P=NP then all hell breaks loose (a name goes up in lights) Either the coNPcomplete and NPcomplete classes are equal or disjoint (no overlap)
Complexity ClassesWhat we know: A problem that is in P is also in coNP A single problem in P that is NPcomplete means P=NP If P=NP then all hell breaks loose (a name goes up in lights) Either the coNPcomplete and NPcomplete classes are equal or disjoint (no overlap)
What we do not know: Whether P=NP Whether graph isomorphism is in P or NPcomplete Whether factoring a large number is NPcomplete or in P
Complexity ClassesWhat we know: A problem that is in P is also in coNP A single problem in P that is NPcomplete means P=NP If P=NP then all hell breaks loose (a name goes up in lights) Either the coNPcomplete and NPcomplete classes are equal or disjoint (no overlap)
What we do not know: Whether P=NP Whether graph isomorphism is in P or NPcomplete Whether factoring a large number is NPcomplete or in P
What we think: P ≠ NP coNP ≠ NP coNPcomplete ≠ NPcomplete
Complexity ClassesWhat was the first problem shown to be NPcomplete? Hint: first public presentation in Cleveland, Ohio, 1971
Complexity ClassesWhat was the first problem shown to be NPcomplete? Correct: Satisfiability
Complexity ClassesWhat was the first problem shown to be NPcomplete? Correct: Satisfiability
How was it done?
Complexity ClassesWhat was the first problem shown to be NPcomplete? Correct: Satisfiability
How was it done? Rough idea: All we have to do is provide a circuit that outputs 1 if and only if the original problem has a solution that is, we need to express the quick “check” of the constraints in propositional logic
Complexity ClassesWhat was the first problem shown to be NPcomplete? Correct: Satisfiability
How was it done? Rough idea: All we have to do is provide a circuit that outputs 1 if and only if the original problem has a solution that is, we need to express the quick “check” of the constraints in propositional logic
1
2
3
k =1 Vertex cover
Complexity ClassesWhat was the first problem shown to be NPcomplete? Correct: Satisfiability
How was it done? Rough idea: All we have to do is provide a circuit that outputs 1 if and only if the original problem has a solution
1
2
3
∨ ∨
1 23
∨∨ ∨
1 2 3
∧∧∧
Make sure only one vertex is in the cover
Make sure alledges are covered
k =1
Complexity ClassesWhat was the first problem shown to be NPcomplete? Correct: Satisfiability
How was it done? Rough idea: All we have to do is provide a circuit that outputs 1 if and only if the original problem has a solution
1
2
3
∨ ∨
1 23
∨∨ ∨
1 2 3
∧∧∧
Make sure only one vertex is in the cover
Make sure alledges are covered
k =1
Complexity ClassesEven if the circuit uses gates other than ‘and’ and ‘or’any circuit can be expressed as an equivalent CNF formula
How so? 1st create a formula digraph – vertices are operators, edges connect to evaluation of nesting
Complexity ClassesEven if the circuit uses gates other than ‘and’ and ‘or’any circuit can be expressed as an equivalent CNF formula
How so? Then create a new variable foreach operator vertex.
vx10
vx9
vx6vx8
vx4vx7
vx5
vx3vx2vx1
Complexity ClassesFinally, add clause sets for all new variables according to this table:
Complexity ClassesFor this example we get the following:
Note: there are at most 4 clauses for each vertex – hence thisis a polynomial time translation from any Boolean expressionto CNFNote: no such translation exists to DNF
Complexity ClassesIf you have one NPcomplete problem you have them all
Y Z
If there is a polynomial time translation from Y to Z andY is NPcomplete
Complexity ClassesIf you have one NPcomplete problem you have them all
Y
X1
X2
X3 . . .
Z
If there is a polynomial time translation from Y to Z andY is NPcomplete then all NPproblems polytime translate to Z through Y
Complexity ClassesSimple transformation: Show 3CNF is NPcomplete
Any assignment that satisfies (1) can be extended to satisfy (2):if the left clause of (2) cannot be satisfied by the literal assignments in (1), then set vx1 to 1 otherwise, set it to 0.Any assignment not satisfying (1) also does not satisfy (2)
For any length clause, repeat the above until clause width is 3.It is not possible using this reduction to get equivalence with 2 literal clauses.
Complexity ClassesHarder transformation: Show Vertex Cover is NPcomplete Transformation from 3SAT:
v1
v2
v3
v4
v5
v6
k = 2(# clauses) + number variables 14 for this example
Complexity ClassesHarder transformation: Show Vertex Cover is NPcomplete Transformation from 3SAT:
v1
v2
v3
v4
v5
v6
Any satisfying assignment: There is a cover of size k = 2(# clauses) + number variables
Complexity ClassesHarder transformation: Show Vertex Cover is NPcomplete Transformation from 3SAT:
v1
v2
v3
v4
v5
v6
Any nonsatisfying assignment: There is no cover of size k = 2(# clauses) + number variables
Complexity ClassesHarder still: Show Hamiltonian Cycle is NPcomplete Transformation from Vertex Cover Hamiltonian Cycle: Given undirected graph G = (V,E), does there exist a cycle in G that passes through all vertices in G?
Complexity ClassesHarder still: Show Hamiltonian Cycle is NPcomplete Transformation from Vertex Cover Hamiltonian Cycle: Given undirected graph G = (V,E), does there exist a cycle in G that passes through all vertices in G?
For each edge ⟨u,v⟩ ∈ E create an “edge gadget”:
u
v
Complexity ClassesHarder still: Show Hamiltonian Cycle is NPcomplete Transformation from Vertex Cover Hamiltonian Cycle: Given undirected graph G = (V,E), does there exist a cycle in G that passes through all vertices in G?
For each edge ⟨u,v⟩ ∈ E create an “edge gadget”:
u
v
1st way a Hamiltonian cycle can go through this gadget corresponds to v in the cover and not u
Complexity ClassesHarder still: Show Hamiltonian Cycle is NPcomplete Transformation from Vertex Cover Hamiltonian Cycle: Given undirected graph G = (V,E), does there exist a cycle in G that passes through all vertices in G?
For each edge ⟨u,v⟩ ∈ E create an “edge gadget”:
u
v
2nd way a Hamiltonian cycle can go through this gadget corresponds to u in the cover and not v
Complexity ClassesHarder still: Show Hamiltonian Cycle is NPcomplete Transformation from Vertex Cover Hamiltonian Cycle: Given undirected graph G = (V,E), does there exist a cycle in G that passes through all vertices in G?
For each edge ⟨u,v⟩ ∈ E create an “edge gadget”:
u
v
3rd way a Hamiltonian cycle can go through this gadget corresponds to v and u both in the cover
Complexity Classes1
2
3
4
56
7
⟨5,7⟩
⟨6,7⟩⟨2,7⟩
⟨1,7⟩
⟨4,5⟩
⟨5,6⟩
⟨3,4⟩
⟨2,3⟩
⟨1,2⟩
Complexity Classes1
2
3
4
56
7
5
5
⟨5,7⟩
⟨6,7⟩⟨2,7⟩
⟨1,7⟩
⟨4,5⟩
⟨5,6⟩
⟨3,4⟩
⟨2,3⟩
⟨1,2⟩
Complexity Classes1
2
3
4
56
7
75
21
3
4
6
5
617
43
2
⟨5,7⟩
⟨6,7⟩⟨2,7⟩
⟨1,7⟩
⟨4,5⟩
⟨5,6⟩
⟨3,4⟩
⟨2,3⟩
⟨1,2⟩
Complexity Classes1
2
3
4
56
7
...
⟨5,7⟩
⟨6,7⟩⟨2,7⟩
⟨1,7⟩
Complexity Classes1
2
3
4
56
7
...
⟨5,7⟩
⟨6,7⟩⟨2,7⟩
⟨1,7⟩
7
Complexity Classes1
2
3
4
56
7
...
⟨5,7⟩
⟨6,7⟩⟨2,7⟩
⟨1,7⟩
7
⟨4,5⟩
⟨5,6⟩
Complexity Classes1
2
3
4
56
7
...
⟨5,7⟩
⟨6,7⟩⟨2,7⟩
⟨1,7⟩
⟨4,5⟩
⟨5,6⟩
75
Complexity Classes1
2
3
4
56
7
...
⟨5,7⟩
⟨6,7⟩⟨2,7⟩
⟨1,7⟩
⟨4,5⟩
⟨5,6⟩
75
⟨3,4⟩
⟨2,3⟩
Complexity Classes1
2
3
4
56
7
...
⟨5,7⟩
⟨6,7⟩⟨2,7⟩
⟨1,7⟩
⟨4,5⟩
⟨5,6⟩
⟨3,4⟩
⟨2,3⟩
⟨5,7⟩
753
Complexity Classes1
2
3
4
56
7
...
⟨5,7⟩
⟨6,7⟩⟨2,7⟩
⟨1,7⟩
⟨4,5⟩
⟨5,6⟩
⟨3,4⟩
⟨2,3⟩
⟨5,7⟩
753
⟨1,2⟩
Complexity Classes1
2
3
4
56
7
...
⟨5,7⟩
⟨6,7⟩⟨2,7⟩
⟨1,7⟩
⟨4,5⟩
⟨5,6⟩
⟨3,4⟩
⟨2,3⟩
⟨1,2⟩
7531
Complexity Classes1
2
3
4
56
7
...
⟨5,7⟩
⟨6,7⟩⟨2,7⟩
⟨1,7⟩
⟨4,5⟩
⟨5,6⟩
⟨3,4⟩
⟨2,3⟩
⟨1,2⟩⟨1,2⟩
7531
Complexity Classes1
2
3
4
56
7
⟨5,7⟩
⟨6,7⟩⟨2,7⟩
⟨1,7⟩
⟨4,5⟩
⟨5,6⟩
⟨3,4⟩
⟨2,3⟩
⟨1,2⟩
7531
Complexity ClassesApple pie simple: Show Traveling Salesman is NPcomplete Transformation from Hamiltonian Cycle Traveling Salesman: Given undirected graph G = (V,E), and a weight function w
G: E → N
+ mapping edges to positive integers, does there exist a Hamiltonian cycle in G whose total weight is B?
1
2
3
4
56
7
Complexity Classes
1
2
3
4
56
7
Create a complete graph H that has a number of vertices equal to G. Labelthe vertices of H exactly as vertices arelabeled in G. For all edges ⟨x,y⟩ in G let w
H(⟨x,y⟩) = 1 for the corresponding
edge in H.
G:
H:
11
11
11
11
1
1
2
3
4
56
7
Complexity Classes
1
2
3
4
56
7
Create a complete graph H that has a number of vertices equal to G. Labelthe vertices of H exactly as vertices arelabeled in G. For all edges ⟨x,y⟩ in G let w
H(⟨x,y⟩) = 1 for the corresponding
edge in H. For every other edge e in H, assign w(e) = number of vertices+1. LetB = number of vertices.
G:
H:
88
8
8
1
2
3
4
56
7
Complexity Classes
Create a complete graph H that has a number of vertices equal to G. Labelthe vertices of H exactly as vertices arelabeled in G. For all edges ⟨x,y⟩ in G let w
H(⟨x,y⟩) = 1 for the corresponding
edge in H. Let n be the number of verticesin G and H. For every other edge e in H, assign w(e) = n+1. Let B = n.
Any Hamiltonian Cycle in H of total weight n corresponds to a HC in G.
G:
H:
1
2
3
4
56
7
11
1
11
11
1
2
3
4
56
7
Complexity Classes
Create a complete graph H that has a number of vertices equal to G. Labelthe vertices of H exactly as vertices arelabeled in G. For all edges ⟨x,y⟩ in G let w
H(⟨x,y⟩) = 1 for the corresponding
edge in H. Let n be the number of verticesin G and H. For every other edge e in H, assign w(e) = n+1. Let B = n.
Any Hamiltonian Cycle in H of total weight n corresponds to a HC in G.
Any Hamiltonian Cycle in H of weight other than n must include an edge that does not exist in G
G:
H:
1
2
3
4
56
7
11
111
1
8
Complexity Classes
Shortest Path – two verticesIs there a simple path from a to bin G(V,E) having length B or less?
Longest path – two verticesIs there a simple path from a to bin G(V,E) having length B or more?
Edge Cover Is there a subset of edges of sizeno greater than k in G(V,E) that covers all vertices in G?
Vertex Cover Is there a subset of vertices of sizeno greater than k in G(V,E) that covers all edges in G?
Transitive ReductionIs there a set of directed edges A onvertices in G(V, E) of size no greater than k s.t. G(V, A) has paths between all vertices iff G does?
Minimum Equivalent DigraphIs there a subset E’ of dir. edgesin G(V,E) of size no greater than ksuch that G’(V,E’) has paths between all vertices iff G does?
Intree SchedulingProcesses: unit time, partial order with at most one successor. Is there a schedule that meets all deadlines?
Outtree SchedulingProcesses: unit time, partial order with at most one predecessor. Is there a schedule that meets all deadlines?
P NPcomplete
Top Related