Random Knapsack Random Knapsack in Expected in Expected

Polynomial TimePolynomial Time

老師：呂學一老師老師：呂學一老師

Team members Team members • R91921028 陳姿樺

• R92921084 何明彥

• R92921083 余宗恩

OutlineOutline

• Introduction………………………… 姿樺• The uniform distribution………… 姿樺• Long-tailed distributions

– Analysis…………………………………… 姿樺– Applications……………………………… 明彥

• Lower bound………………………… 明彥• General distributions……………… 宗恩

IntroductionIntroduction

• Speaker : R91921028 陳姿樺

TodayToday• Random Knapsack in Expected

Polynomial Time

The problemThe problem• Input :

– Given n items with positive weights w1, . . . ,wn

– and profits p1, . . . , pn

– and a knapsack capacity c, • Output :

– find a subset S [n] :={1,2, . . . ,n} such that ∑iS wi c and ∑iS pi is maximized.

Domination concept Domination concept

• Two subset:– A subset S [n] with weight w(S)=∑iSwi

and profit p(S) = ∑iSpi

– Another subset T [n] with weight w(T)=∑iT wi and profit p(T) = ∑iTpi

• We say S dominates T if w(S) w(T) and p(S) p(T).

AssumeAssume

• For simplicity assume that no two subsets have the same profit.

ExampleExample• n=3

• S={1,3}; w(S)=3; p(S)=1.4• T={1,2}; w(T)=5; p(T)=0.8• S dominates T

1 2 3

Profit 0.5 0.3 0.9

weight 2 3 1

observationobservation• No subset dominated by another

subset can be an optimal solution to the knapsack problem, regardless of the specified knapsack capacity.– It suffices to consider those sets that are

not dominated by any other set, the so-called dominating sets.

observationobservation• In the dominating sets, the solutions

that cannot be improved in weight and profit simultaneously by other solutions.

• Why?– By domination concept

The Nemhauser/Ullmann The Nemhauser/Ullmann algorithmalgorithm

• For i [n], let S(i) be the sequence of dominating subsets over the items 1,...,i.

• The sets in S(i) are assumed to be listed in increasing order of their weights.

The Nemhauser/Ullmann The Nemhauser/Ullmann algorithmalgorithm

• Given S(i), S(i+1) can be computed as follows:– duplicate all subsets in S(i) and then add

item i+1 to each of the duplicated sets.– Removing the sets dominated by any

other set.– The result is the ordered sequence

S(i+1) of dominating sets over the items 1, . . . , i+1.

ExampleExample

• n=3

1 2 3

Profit 0.5 0.3 0.9

weight 2 3 1

Naïve algorithmNaïve algorithm

• List total state and find dominating set

• 21=0,1;– S(1)={0,1}

• 22=0,1,2,12;– S(2)={0,1,12}

• 23=0,1,2,3,12,13,23,123;– S(3)={0,3,13,123}

0 1 2 12 3 13 23 123

p 0 0.5 0.3 0.8 0.9 1.4 1.2 1.7

w 0 2 3 5 1 3 4 6

Nemhauser/Ullmann algo.Nemhauser/Ullmann algo.• S(1)={0,1}• Find S(2)

– From {0,1,2,12} to find dominating set– S(2)={0,1,12}

• Find S(3)– From {0,1,12,3,13,123} to find– S(3)={0,3,13,123}

• The same as naïve algorithm

0 1 2 12 3 13 23 123

p 0 0.5 0.3 0.8 0.9 1.4 1.2 1.7

w 0 2 3 5 1 3 4 6

SomethingSomething• Let us assume that add and compare

numbers in constant time.• Then the sequence S(i+1) can be

calculated from the sequence S(i) in time linear.

• The optimal knapsack filling is described by one of the subsets in the list S(n)

• Generating S(n) basically solves the knapsack problem.

LemmaLemma• For every i[n], let q(i) denote an

upper bound on the (expected) number of dominating sets over the items in 1,...,i, and assume q(i+1) q(i). The Nemhauser/Ullmann algorithm computes an optimal knapsack filling in (expected) time

1

( ( )) ( * ( ))n

i

O q i O n q n

By the lemmaBy the lemma• We can only think the number of

dominating sets, i.e., q(n)• Goal: counting E[q]

THE UNIFORM THE UNIFORM DISTRIBUTION DISTRIBUTION

• Assumption:– Profits are assumed to be chosen

uniformly at random from [0,1] and,– In all of our analyses, weights are

chosen by an adversary.

DefinitionDefinition

• Let m=2n and let S1,...,Sm denote the sequence of all subsets of [n] listed in non-decreasing order of their weights.

• Let the profit of subset Su be Pu=ΣiSu

pi.

• For any 2 u m, define Δu=maxv[u]Pv - maxv[u-1] Pv 0.

ObservationObservation

• Observe that S1 is always a dominating set.

• For all u 2, Su is dominating if and only if Δu > 0.

ExampleExample• n=3

• Su

1 2 3

Profit 0.5 0.3 0.9

weight 2 3 1

Su S1 S2 S3 S4 S5 S6 S7 S8

0 3 1 2 13 23 12 123

Pu 0 0.9 0.5 0.3 1.4 1.2 0.8 1.7

wu 0 1 2 3 3 4 5 6

ExampleExample• Δ2=P2 - P1 =0.9• Δ3=P2 - P2 =0• Δ4=P2 - P2 =0• Δ5=P5 - P2 =0.5• Δ6=P5 - P5 =0• Δ7=P5 - P5 =0• Δ8=P8 - P5 =0.3• Dominating sets: S1, S2, S5, S8,

Su S1 S2 S3 S4 S5 S6 S7 S8

0 3 1 2 13 23 12 123

Pu 0 0.9 0.5 0.3 1.4 1.2 0.8 1.7

wu 0 1 2 3 3 4 5 6

LemmaLemma• For every u {2,...,m},

20

1[ | ]

32u uEn

TheoremTheorem• Suppose the weights are arbitrary

positive numbers and profits are chosen according to the uniform distribution over [0,1]. Let q denote the number of dominating sets over all n items. Then E[q] = O(n3).

Proof theoremProof theorem

• Pm=ΣiSmpi=Σi[n]pi and P1=ΣiS1

pi=0

• E[Pm]=n/2 because each individual item has profit 1/2 on expectation.

2

2

22

1[ ] [ ]

Pr[ 0]* [ | 0]

1Pr[ 0]*

32

m

m uu

m

u u uu

m

uu

E P P E

E

n

P1=0

By lemma

Proof theoremProof theorem

• Consequently,

2

2 3 3

[ ] 1 Pr[ 0]

1 32 [ ] 16 1

m

uu

m

E q

n E P n O n

22

2 3

2

1Pr[ 0]*

2 32

Pr[ 0] 32 * 16

m

m uu

m

u mu

nE P

n

n E P n

Proof Lemma(1)Proof Lemma(1)• For every u {2,...,m},

20

1[ | ]

32u uEn

Proof Lemma(2)Proof Lemma(2)• Find a easy way

• The goal is change to

2 2

1 1[ | 0] Pr[ | 0]*

16 16u u u uEn n

2

1 1Pr[ | 0]

16 2u un

DefinitionDefinition• For every v[u-1], define Xv=Su\Sv and

Yv=Sv\Su.

• Lv=∑iXvpi

• Define two set

– Clearly, AB

21

1

1{( *...* ) [0,1] | : }

16

{( *...* ) [0,1] | : }

v

v

n k

i Y

n k

i Y

k

k

n i

n i

v

v

A p p v p Ln

B p p v p L

DefinitionDefinition• For ε[0,1], we define

• Obviously, B=B0 and, in general, Bε can be obtained by shrinking B in each dimension by a factor of 1-ε.

• As the number of dimensions is n-k, it holds

1{( *...* ) [0,1 ] | : (1 ) }v

n k

i Y

k n i vB p p v p L

( )( ) (1 ) ( )n kvol B vol B

Help1 Help1

• Lv1/4n

• Proof:

[ ]

[ ]

1Pr[ [ ] : | : ]

4

1Pr[ | : ]

4

1 1Pr[ ]

4 4 4

v v

v v

i X i Y

j k i X i Y

j k

j

j

j

i i

i i

j k p v p pn

p v p pn

kp

n n

Help1Help1

• we assume pj 1/4n, for every j[k].

• Lv=∑iXvpi, Under our assumption, Lv

1/4n, for every v[u-1],• Why?

– because each set Xv contains at least one element of size at least 1/4n .

Help2 Help2

• Observe that Lv(1-1/4n ) Lv-1/16n2 because Lv1/4n. Thus setting ε=1/4n implies Bε A.

( )( ) ( ) (1 ) ( )

3(1 ( )) ( ) ( )

4

n kvol A vol B vol B

n k vol B vol B

Proof new LemmaProof new Lemma

2

2

2

2

1Pr[ | 0]

161

Pr[ : | : ]16

1Pr[ : | : ]

16

1Pr[ : | : ]

16

( ) ( ) 3

( ) ( ) 4

u v u v

v v v v

v v

i S i S i S i S

i X i Y i X i Y

i Y i Y

i i

u u

i i i i

i i i i

v v

n

v p p v p pn

v p p v p pn

v p L v p Ln

vol A B vol A

vol B vol B

By definitio

n

By help2

Long-tailed vs. short-tailed Long-tailed vs. short-tailed • if the tail function of a distribution

can be lower-bounded by the tail function of the exponential function, then we say the distribution has a “long tail”, and if the tail function can be upper-bounded by the exponential tail function, then we talk about “short tails”.

DefinitionDefinition• Given any continuous probability

distribution with density function f:R0→R0, the tail function T:R0→[0,1] is defined by T(t)=∫t

∞f(x)dx.• We define the slope of T at xR0 to

be the first derivative of the function -ln(T(·)) at x, i.e.,

slopeT(x)=-[ln(T(x))]’.

Long-tailed definitionLong-tailed definition• The tail of a continuous probability

distribution is defined to be long if there exists α>0 such that slopeT(x) α, for every xR 0.

Theorem Theorem • For i[n], let profit pi be a random

variable with tail function Ti:R

0→[0,1]. Define µi = E[pi] and let αi be an appropriate positive real number satisfying slopeTi

(x)αi for every x0, i[n]. Let α=maxi[n] αi and µ=maxi

[n]µi. Finally, let q denote the number of dominating sets over the elements in [n].

• Then2

[ ] [ ]

[ ] ( ) 1 1i n i n

i iE q n

Proof theoremProof theorem

• µi = E[pi], so E[Pm]=∑i[n]µi.

• Help3:

• Proof of theorem

[ ] [ ]2

[ ][ ] 1 1 *

min ( [ | 0]) i imi n i nu

m

u u

E PE q

E

1[ | 0]

i n

iu uE

Help3Help3• Help3 can be proved by the former

lemma

1[ | 0]

i n

iu uE

• Speaker : R92921084 何明彥

Application(1/2)Application(1/2)by theorem:

if profits are chosen according to the exponential distribution ,then

2

[ ] [ ]

[ ] ( ) 1 1i n i n

i iE q n

2[ ] ( )E q n

Application(2/2)Application(2/2)The exponential and other long tailed distribution:

We can generalize the upper bound towards all continuous distributions with finite mean.

2[ ] ( )E q n

Lower BoundLower Bound

GoalGoal• a lower bound for the number of

dominating set for continuous distributions with non-increasing density function.

Theorem(1/4)Theorem(1/4)draw profit randomly according to a continuous probability distribution with non-increasing density function:

then a vector of weight w1,…,wn

Therefore,

0 0:f R R

2[ ] ( )E q n

0: [0,1]F R

Theorem(2/4)Theorem(2/4)

: the weight for i-th item : the profit

For every jn

: the dominating set for item 1,…j-1

2iiw

1

j

i

ijP p

ip

( 1)S j

Theorem(3/4)Theorem(3/4)All those sets of S(j-1) have weight <All sets containing item j have weight at least

These sets can’t dominate the sets in S(j-1).S(j) contains all sets from S(j-1).SЄS(j-1) with profit create

new dominating sets in S(j) with profit

2 j

2 j

1 , 1( ) ( )j j jp S P p P

1( ) j jp S p p

Theorem(4/4)Theorem(4/4)

S(3)=(0;1;3;1,2;1,3;1,2,3)S(4)=(0;1;3;1,2;1,3;1,2,3;

1 , 1( ) ( )j j jp S P p P 1 2 3 4

P 0.5 0.1 1 0.7

W 2 4 8 16

1 1.6jP Ex: j=4 0.7jP

To create new dominating set

00

0.52

18

0.66

1.510

1.614

(3) (0.9,1.6)p

3,4;1,3,4;1,2,3,4)

Lower Bound(1/5)Lower Bound(1/5)Given α>0,

:# of dominating sets in S(j) with profit >(note: don’t count the last set [j] in this sequence)

next, by induction we shows that

jX

jX

jP

[ ] ( )jE X F j

Lower Bound(2/5)Lower Bound(2/5) ; for j>1, it holds

1[ ] ( )E X F 1 1

1

[ ] P( )*( [ ] [ ] 1)

P( )* [ ]

P( )*[ ( )( 1) ( )( 1) 1]

P( )* ( )( 1)

P( )*[ ( )( 1) 1] P( )* ( )( 1)

( )

{ ( ) ( ) ( ) ( ) ( ) ( )}

( 1)

j j

j j jp p

j

j

j

j j j

j

j j

j j

E X p E X E X

p E X

p F p j F p j

p F j

F

p F j p F j

F j

a F b F a b F p F p F

F

( ) ( )F j

Lower Bound(3/5)Lower Bound(3/5) : # of new dominating set in S(j)

(+1 because the set[j-1] doesn’t count in but yields a new set in S(j))

| ( ) | | ( 1) |Yi S j S j

1 1j

jpYi X

1j

jpX

1[ ] [ 1]j

jpE Yi E X

Lower Bound(4/5)Lower Bound(4/5)#of dominating set in S(n) is

therefore, 1

[ ]n

j

jq E Y

1

1 1

1 1

[ ] [ ] [ 1]

[ ( )( 1) 1] [ ( )]

j

n njp

j j

n n

j j

j j

jE q E Y E X

E F p j E F p j

[ ( )]jE F p1

1

( ) ( ( )) ( ( ))

( ( ))

j jF x P F p x P p F x

F F x x

Lower Bound(5/5)Lower Bound(5/5) is uniformly distributed in [0,1] so that

consequently,

( )jF p1

[ ( )]2

jE F p

2

1

1[ ] ( )

2

n

j

E q j n

Conclusion for PartIIConclusion for PartIIthe expected #of dominating sets for the exponential distribution is tight.For the exponential distribution, lower and upper bound deviate by

2( )n

General DistributionsGeneral Distributions

• Speaker : R92921083 余宗恩

ObjectiveObjective

Profit 與以下兩者呈線性關係：

1. 期望收穫的最大值

2. 機率密度函數的上界

MethodMethod• Lemma 9:

• =>

• Lemma 10:

• Theorem

10 0 11

[ | ][ ] [ | ]

2k k k

kk

E q X XE q E q X

5

0[ ] 2 ( )k

kkE q q M

3[ ] min{ 2 1,2 }k nkE q n n

4 2 3)[ ] 32( nE q n

Define:

( 比 x 大的機率 )

因為引入 Xk 的 Ti(t) 並不是可完美 bound E[q] 的 long tailed function ，所以還需要進一步的調整。

Define:

把短尾的 mask 起來，再於 lemma 10 分析 E[qk] 。

Rewrite Equation 2:

其實只要證明 ：

為了套用 Theorem 4 ，再定義：

(by lemma 9)

(by lemma 10)

ConclusionConclusion

ConclusionConclusion