Extreme Programming Extreme Programming Método Ágil Introdução.
[PPT]Linear Programming (Optimization) - SOLABsolab.kaist.ac.kr/files/LP/LP2015/LP_Chap_04.pptx ·...
Transcript of [PPT]Linear Programming (Optimization) - SOLABsolab.kaist.ac.kr/files/LP/LP2015/LP_Chap_04.pptx ·...
Linear Programming 2015 1
Chapter 4. Duality Theory Given min , s.t. , (called primal problem)
there exists another LP derived from the primal problem using the same data, but with different structure (called the dual prob-lem, 쌍대문제 ).
The relation between the primal and the dual problem consti-tutes very important basis for understanding the deeper struc-ture of LP (compared to systems of linear equations).It provides numerous insights in theory and algorithms and im-portant ingredients in the theory of LP.
Objective value of the dual problem of a feasible dual solution provides lower bound on the optimal primal objective value and the dual problem can be derived for this purpose. However, the text derives it as a special case of the Lagrangian dual problem.
Consider an optimization problem (OPT) , . A relaxation of OPT is any min-imization problem
(RP) with the following two properties: (R1) (R2) for
Prop: If RP is infeasible, so is OPT. If OPT is feasible, then . Pf) From (R1), if , so is OPT. Now suppose that is finite, and let be an optimal solution to OPT. Then , where the first inequality follows from (R2) and the second one follows from (R1). Finallly, if , (R1) and (R2) imply that . Relaxation for a maximization problem also can be defined similarly.
Linear Programming 2015 2
Linear Programming 2015 3
Given min , s.t. , (called primal problem)consider a relaxed problem in which the constraints is elimi-nated and instead included in the objective with penalty , where is a price vector of the same dimension as .
Lagrangian function : The problem becomes min
s.t. (called Lagrangian relaxation)
Optimal value of this problem for fixed is denoted as .Suppose is an optimal solution to the LP, then
since is a feasible solution to LP. gives a lower bound on the optimal value of the LP. Want close lower bound.
Linear Programming 2015 4
Lagrangian dual problem :max s.t. no constraints on ,
where
Hence the dual problem ismax s.t. ( , )
Linear Programming 2015 5
Remark :(1) If LP has inequality constraints
dual constraints are
Or the dual can be derived directly min , s.t. ( Let )
max , s.t.
Hence the dual problem ismax , s.t.
Linear Programming 2015 6
(2) If are free variables, then
dual constraints are
Linear Programming 2015 7
4.2 The dual problem
min max s.t. s.t.
Linear Programming 2015 8
Table 4.1 : Relation between primal and dual variables and con-straints
Dual of a maximization problem can be obtained by converting it into an equivalent min problem, and then take its dual.
PRIMAL minimize maximize DUAL
constraintsfree
variables
variablesfree
constraints
Linear Programming 2015 9
Vector notationmin max s.t. s.t.
min max s.t. s.t.
Thm 4.1 : If we transform the dual into an equivalent minimiza-tion problem and then form its dual, we obtain a problem equiva-lent to the original problem.( the dual of the dual is primal, involution property)
For simplicity, we call the min form as primal and max form as dual. But any form can be considered as primal and the corre-sponding dual can be defined.
Linear Programming 2015 10
Dual
321 32min xxx
freexx
xxxxx
xx
3
2
1
3
321
21
00
463253
.. ts
321 465max ppp
.. ts
33 2 3120
0
32
21
21
3
2
1
pppppp
pp
freep
321 465min xxx .. ts
33- 2312
00
32
21
21
3
2
1
xxxxxx
xx
freexDual
321 32max ppp
freepp
ppppp
pp
3
2
1
3
321
21
00
46325 3
.. ts
(Ex 4.1)
Linear Programming 2015 11
Ex 4.2 : Duals of equivalent LP’s are equivalent.
min max s.t. s.t. free
min max s.t. s.t. free free,
min max s.t. s.t.
Linear Programming 2015 12
Ex 4.3: Redundant equation can be ignored.
Considermin max s.t. (feasible) s.t.
Suppose .For feasible , get . dual constraints are .
dual objective is .Let . max
s.t.
Linear Programming 2015 13
Thm 4.2 : If we use the following transformations, the corre-sponding duals are equivalent, i. e., they are either both infea-sible or they have the same optimal cost.(a) free variable difference of two nonnegative variables(b) inequality equality by using nonnegative slack var.(c) If LP in standard form (feasible) has redundant equality constraints, eliminate them.
Linear Programming 2015 14
4.3 The duality theorem Thm 4.3 : ( Weak duality )
If is feasible to primal, is feasible to dual, then .pf) Let , , If feasible to P and D respectively, then
Cor 4.1 : If any one of primal, dual is unbounded the other infeasible
Cor 4.2 : If feasible and , then optimal.pf) for all primal feasible . Hence is optimal to the primal problem. Similarly for .
Linear Programming 2015 15
Thm 4.4 : (Strong duality)If a LP has an optimal solution, so does its dual, and the respec-tive optimal costs are equal.
pf) Consider standard form first.Get optimal dual solution from the optimal basis.Suppose have min , , full row rank, and this LP has optimal so-lution.Use simplex to find optimal basis with .Let , then dual feasibleAlso , hence is an optimal solution to the dual and .For general LP, convert it to standard LP with full row rank and apply the result, then convert the dual to the dual for the original LP.
Linear Programming 2015 16
D1 1
2 D2
Fig 4.1 : Proof of the duality theorem for general LP
Duals of equivalent prob-lems are equivalent Equivalent
Duality for standard form problems
Linear Programming 2015 17
Table 4.2 : different possibilities for the primal and the dual
Finite opti-mum
Unbounded Infeasible
Finite opti-mum
Possible Impossible Impossible
Unbounded Impossible Impossible Possible
Infeasible Impossible Possible Possible
(P)
(D)
Linear Programming 2015 18
Note:(1) Later, we will show strong duality without using simplex method. (see example 4.4 later)
(2) Optimal dual solution provides “certificate of optimality”certificate of optimality is (short) information that can be used to check the optimality of a given solution in polynomial time.( Two view points : 1. Finding the optimal solution.
2. Proving that a given solution is optimal.For computational complexity of a problem, the two view points usually give the same complexity (though not proven yet, P = NP ?). Hence, researchers were almost sure that LP can be solved in polynomial time even before the discovery of a polynomial time algorithm for LP.)
(3) The nine possibilities in Table 4.2 can be used importantly in determining the status of primal or dual problem.
Linear Programming 2015 19
Thm 4.5 : (Complementary slackness)Let and be feasible solutions to primal and dual, then and are optimal solutions to the respective problems iff
for all , for all .
pf) Define , Then for feasible .
From strong duality, if optimal, we have .Hence Conversely, if , hence optimal.
Linear Programming 2015 20
Note :(1) CS theorem provides a tool to prove the optimality of a given primal solution. Given a primal solution, we may identify a dual solution that satisfies the CS condition. If the dual solution is feasible, both x and y are feasible solutions that satisfy CS con-ditions, hence optimal (If nondegenerate primal solution, then the system of equations has a unique solution. See ex. 4.6).
(2) CS theorem does not need that and be basic solutions.
(3) CS theorem can be used to design algorithms for special types of LP (e.g. network problems). Also an interior point algo-rithm tries to find a system of nonlinear equations similar to the CS conditions.
(4) See the strict complementary slackness in exercise 4.20.
Linear Programming 2015 21
Geometric view of optimal dual solution
(P) min s.t. , , ( assume span )
(D) max s.t.
Let , such that are linearly independent. has unique solution which is a basic solution. Assume that is nondegenerate, i.e., , for .Let be dual vector.
Linear Programming 2015 22
The conditions that and are optimal are
(a) , ( primal feasibility )(b) ( complementary slackness )(c) ( dual feasibility )
( unique solution )(d) ( dual feasibility )
Linear Programming 2015 23
Figure 4.3
𝑐 𝑎5
𝑎4
𝑎3
𝑎2
𝑎1𝐷
𝐶
𝐵𝐴
𝑐
𝑎2
𝑎1
𝑐𝑎3
𝑎1
𝑐𝑎4 𝑎1
𝑐𝑎5
𝑎1
Linear Programming 2015 24
Figure 4.4 degenerate basic feasible solution
𝑐𝑎3
𝑎2
𝑎1
𝑎3 𝑎2𝑎1
𝑥∗
Linear Programming 2015 25
4.4 Optimal dual var as marginal costs
Suppose standard LP has nondegenerate optimal b.f.s. and is the optimal basis, then for small , . not affected, hence remains optimal basis.Objective value is ( )So is marginal cost (shadow price) of requirement .
Another interpretation of duality, for standard form problems: ex) diet problem (ex. 1.3 in sec.1.1)
Have different foods and different nutrients. Nutrient re-quirement vector , . amount of nutrient in one unit of food.min ()
, nutrient vector of one unit of food.Interpret as the “fair” price of per unit of the nutrient.Cost of one unit of food is but it also has a value of if priced at the nutri-ent market.CS conditions asserts that every food that is used () to synthesize the ideal food, should be consistently priced at the two markets (when ). The value of the ideal food is , which is equal to in the nutrient market.
Also can explain the logic of simplex method. With current basis , have cur-rent value of nutrients (. If have nonbasic with , cost of food is cheaper than the value of nutrients obtained from buying one unit of food. Hence it is better to buy food (try to make , or basic).
Linear Programming 2015 26
Linear Programming 2015 27
4.5 Dual simplex method
For standard problem, a basis gives primal solution and dual solution .At optimal basis, have ( )and , hence optimal.( have primal feasible, dual feasible solutions and objective values are the same).Sometimes, it is easy to find dual feasible basis. Then, start-ing from a dual feasible basis, try to find the basis which sat-isfies the primal feasibility.(Text gives algorithm for tableau form, but revised dual sim-plex algorithm also possible.)
Linear Programming 2015 28
Given tableau, for all , for some .
(1) Find row with . is component of .
(2) For with , find such that
(3) Perform pivot ( enters, leaves basis )( dual feasibility maintained)
Linear Programming 2015 29
Ex 4.7 :
103241011422001062054321 xxxxx
2/102/3122/1215060301014354321 xxxxx
4x5x
4x2x
Linear Programming 2015 30
Note :(1) row 0 row 0 + ( row )
( from the choice of )For , add (nonnegative number) to row 0 For , have . Hence dual feasibility maintained.
Objective value increases by . ( note that )
If , objective value strictly increases.
Linear Programming 2015 31
(2) If in all iterations, objective value strictly increases, hence fi-nite termination. (need lexicographic pivoting rule for general case)At terminationcase a) , optimal solution.case b) entries of row are all , then dual is unbounded. Hence primal is infeasible.Reasoning : (a) Find unbounded dual solution. Let be the current dual feasible solution ( ).Want such that as .In other words, .Suppose . Let (negative of row of ), then and .Hence is dual feasible and as .(Is an extreme ray of dual polyhedron?)
(continued)(b) Current row : Since , and for feasibility, but no feasible solution to primal exists. hence dual unbounded.
Linear Programming 2015 32
Linear Programming 2015 33
The geometry of the dual simplex method For standard LP, a basis gives a basic solution (not necessarily
feasible) .The same basis provides a dual solution by , i.e. .The dual solution is dual feasible if .So, in the dual simplex, we search for dual b.f.s. while corre-sponding primal basic solutions are infeasible until primal feasi-bility (hence optimality) is attained.( see Example 4.8 )
See example 4.9 for the cases when degeneracy exists. Different bases may lead to the same basic solution for the pri-mal, but to different basic solutions for the dual.
Linear Programming 2015 34
4.6 Farkas’ lemma and linear inequalities Thm 4.6 : Exactly one of the following holds:
(a) There exists some such that .(b) There exists some such that and .
(Note that we used (I) (II) earlier. Rows of are considered as generators of a cone. But, here, columns of are considered as generators of a cone.)pf) not both : ( i.e. one holds the other not hold)Text shows (a) ~(b) which is equivalent to (b) ~(a).~(a) (b) : Consider
(P) max (D) min
~(a) primal infeasible dual infeasible or un-bounded
but is dual feasible dual unbounded with and .
Linear Programming 2015 35
Other expression for Farkas’ lemma.
Cor 4.3 ) Suppose that any vector satisfying also satisfy .Then such that .
See ‘Applications of Farkas’ lemma to asset pricing’ and separat-ing hyperplane theorem used for proving Farkas’ lemma.
Linear Programming 2015 36
The duality theorem revisited Proving strong duality using Farkas’ lemma (not using simplex
method as earlier)
(P) min (D) max s.t.
Suppose optimal to (P). Let Then that satisfies , must satisfy ( i. e., is infeasible )( Note that above statement is nothing but saying, if optimal, then there does not exists a descent and feasible direction at . Necessary condition for optimality of . )
Linear Programming 2015 37
(continued)Otherwise, let .Then
for small Hence feasible for small and .Contradiction to optimality of .By Farkas’, such that .Let for . such that and . By weak duality, is an optimal dual solution.
Linear Programming 2015 38
Figure 4.2 strong duality theorem
𝑐
𝑎3
𝑎2𝑎1
𝑝2𝑎2 𝑝1𝑎1
{𝑑 :𝑐 ′𝑑<0 }
{𝑑 :𝑎𝑖 ′ 𝑑≥0 , 𝑖∈𝐼 }
𝑥∗