劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O...
Transcript of 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O...
![Page 1: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/1.jpg)
岩田 覚
(京都大学数理解析研究所)
劣モジュラ最適化
![Page 2: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/2.jpg)
Outline• Submodular Functions
ExamplesDiscrete Convexity
• Minimizing Submodular Functions • Symmetric Submodular Functions• Maximizing Submodular Functions• Approximating Submodular Functions
![Page 3: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/3.jpg)
Submodular Functions
• Cut Capacity Functions • Matroid Rank Functions • Entropy Functions
)()()()( YXfYXfYfXf ∪+∩≥+
VYX ⊆∀ ,
VX Y
R2: →Vf:V Finite Set
![Page 4: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/4.jpg)
Cut Capacity Function
Cut Capacity ∑= } leaving :|)({)( XaacXκ
X
s t
Max Flow Value=Min Cut Capacity
0)( ≥ac
![Page 5: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/5.jpg)
Matroid Rank Functions
:)()(
||)(,
ρρρ
ρYXYX
XXVX≤⇒⊆≤⊆∀
],[rank)( XUAX =ρMatrix Rank Function
Submodular
=A
X
V
U
Whitney (1935)
![Page 6: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/6.jpg)
Entropy Functions
:)(Xh
0)( =φh
X
Information Sources
0≥
)()()()( YXhYXhYhXh ∪+∩≥+
Entropy of the Joint Distribution
Conditional Mutual Information
![Page 7: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/7.jpg)
Positive Definite Symmetric Matrices
0)( =φf
=A ][XA][detlog)( XAXf =
X
)()()()( YXfYXfYfXf ∪+∩≥+
Ky Fan’s Inequality
Extension of the Hadamard Inequality∏∈
≤Vi
iiAAdet
![Page 8: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/8.jpg)
Discrete Concavity
})({}){()(}){( ufuTfSfuSfTS
−∪≥−∪⇒⊆
V
T
Diminishing Returns
S u
![Page 9: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/9.jpg)
Discrete Convexity
)()()()( YXfYXfYfXf ∪+∩≥+
V
X Y
Convex Function
x
y
![Page 10: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/10.jpg)
Discrete Convexity
},{ ba
},,{ cbaV =
}{b
}{a}{c
},{ cb
φ
Lovász (1983)
f̂
f
f̂
: Linear Interpolation
: Convex
: Submodular
Murota (2003)
Discrete Convex Analysis
![Page 11: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/11.jpg)
Submodular Function Minimization
X
)(Xf
?}|)(min{ VYYf ⊆
MinimizationAlgorithm
Evaluation Oracle
Minimizer
0)( =φfAssumption:
Ellipsoid MethodGrötschel, Lovász, Schrijver (1981)
![Page 12: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/12.jpg)
Base Polyhedra
Submodular Polyhedron
)}()(,,|{)( YfYxVYxxfP V ≤⊆∀∈= R
∑∈
=Yv
vxYx )()(
}|{ RR →= VxV
)}()(),(|{)( VfVxfPxxfB =∈=Base Polyhedron
![Page 13: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/13.jpg)
Greedy Algorithm
v
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
))((
))(())((
)(
)()(
1110
11001
2
1
2
1
nn vLf
vLfvLf
vy
vyvy
MM
L
OMM
MO
L
}){)(())(()( vvLfvLfvy −−=
Edmonds (1970)Shapley (1971)
Extreme Base
)(vL
)( Vv∈
:y
2v1v nv
![Page 14: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/14.jpg)
Theorem
Min-Max Theorem
)}(|)(max{ )(min fBxVxYfVY
∈= −
⊆
)}(,0min{:)( vxvx =−
)()()( YfYxVx ≤≤−
Edmonds (1970)
![Page 15: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/15.jpg)
Combinatorial Approach
Extreme Base
Convex Combination
Cunningham (1985)
LL
L yx ∑Λ∈
= λ
)( fByL ∈
x
|)(|max XfMVX⊆
=
)log( 6 nMMnO γ
![Page 16: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/16.jpg)
Submodular Function Minimization
)( 87 nnO +γ)log( 5 MnO γ
)log( 7 nnO γ
Grötschel, Lovász, Schrijver (1981, 1988)
Iwata, Fleischer, Fujishige (2000) Schrijver (2000)
Iwata (2003)
Fleischer, Iwata (2000)
)log)(( 54 MnnO +γOrlin (2007)
)( 65 nnO +γ
Iwata (2002)
Fully Combinatorial
Ellipsoid Method
Cunningham (1985)
Iwata, Orlin (2009)
![Page 17: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/17.jpg)
)( subject to Minimize 2
fBxx∈
The Minimum-Norm Base
sol. opt. :∗x}0)(|{: <= ∗
− vxvX}0)(|{:0 ≤= ∗ vxvX
Minimal MinimizerMaximal Minimizer
∗x
Theorem Fujishige (1984)
Wolfe’s Algorithm Practically Efficient
![Page 18: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/18.jpg)
Evacuation Problem (Dynamic Flow)
:)(Xo
:)(aτ
Hoppe, Tardos (2000)
:)(vb
:)(ac
SX ∩ XT \
TSXXoXb ∪⊆∀≤ ),()(
TSCapacity
Transit Time
Supply/Demand
Maximum Amount of Flow from to .
Feasible
![Page 19: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/19.jpg)
Multiterminal Source Coding
X XR
Y
Encoder
Encoder YRDecoder
)(YH
)(XH
)|( XYH
)|( YXH
),( YXH
),( YXHXR
YR
Slepian, Wolf (1973)
![Page 20: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/20.jpg)
Multiclass Queueing Systems
iμiλ
Server
Control Policy
Arrival Rate Service Rate
Multiclass M/M/1Preemptive
Arrival Interval Service Time
![Page 21: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/21.jpg)
Performance Region
VXS
Xii
Xiii
Xiii ⊆∀
−≥
∑∑
∑∈
∈
∈
,1 ρ
μρρ
:s
,: iii μλρ =
YRjs
is
:js Expected Staying Time of a Job in j
Achievable
Coffman, Mitrani (1980)
:s1<∑
∈Viiρ
![Page 22: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/22.jpg)
A Class of Submodular FunctionsVzyx +∈R,,
))(()()()( XxhXyXzXf −=
:h
VXS
Xii
Xiii
Xiii ⊆∀
−≥
∑∑
∑∈
∈
∈
,1 ρ
μρρ
Nonnegative, Nondecreasing, Convex
)( VX ⊆
i
iiy
μρ
=:
xxh
−=
11:)(
iii Sz ρ=:
iix ρ=:
Submodular
Itoko & Iwata (2005)
![Page 23: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/23.jpg)
Zonotope in 3D))(),(),(()( XzXyXxXw =
}|)({conv VXXwZ ⊆=
)(),,(~ xyhzzyxf −=
Zonotope
} ofPoint ExtremeLower :),,(|),,(~min{
}|)(min{
Zzyxzyxf
VXXf
=
⊆
Remark: ),,(~ zyxf is NOT concave!
z
![Page 24: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/24.jpg)
Line Arrangementβ
iii zyx =+ βα
α
Topological Sweeping MethodEdelsbrunner, Guibas (1989) )( 2nO
Enumerating All the Cells
![Page 25: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/25.jpg)
Symmetric Submodular Functions
. ),\()( VXXVfXf ⊆∀=
?},|)(min{ VXVXXf ≠⊂≠φ
)()()()( ,
YXfYXfYfXfVYXYX
∩+∪≥+⇒≠∪≠∩ φ
R→Vf 2:
Symmetric
Crossing Submodular
Symmetric Submodular Function Minimization
![Page 26: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/26.jpg)
Maximum Adjacency Ordering
• Minimum Cut Algorithm by MA-orderingNagamochi & Ibaraki (1992)
• Simpler ProofsFrank (1994), Stoer & Wagner (1997)
• Symmetric Submodular Functions Queyranne (1998)
• Alternative ProofsFujishige (1998), Rizzi (2000)
![Page 27: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/27.jpg)
Minimum Degree OrderingNagamochi (2007)
ISAAC’07, Sendai, Japan
Finding the family of all extreme sets for symmetric crossing submodular functionsin time. )( 3γnO
Symmetric Submodular Function Minimization
![Page 28: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/28.jpg)
Extreme Sets
.: ),()( XZXZXfZf ≠≠⊂∀> φ
)\()\()()( XYfYXfYfXf +≥+
:f Symmetric Crossing Submodular Function
Extreme Set:X
The family of all extreme sets forms a laminar.
X YX Y
![Page 29: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/29.jpg)
Flat Pair for Symmetric Submodular Functions
,}|)(min{)( XxxfXf ∈≥
.1|},{| s.t. =∩⊆∀ vuXVX
)( },{ vuVvu ≠⊆Flat Pair
uv
X
},{ vu
No Extreme SetsSeparate andu .v
Shrink into a single vertex.
},{ vu
![Page 30: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/30.jpg)
MD-Ordering for Symmetric Submodular Functions
X
)\( )()(:)( iii VVXXVfXfXf ⊆∪+=
iV
Symmetric, Crossing Submodular
MD-orderingVvvvv nn ∈− ,,...,, 121
jvEach has minimum value of among )(1 vf j− .\ 1−∈ jVVv
},...,{: 1 ii vvV =
![Page 31: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/31.jpg)
MD-Ordering for Symmetric Submodular Functions
nn vv ,1−
1−nv2−= ni
The last two vertices of an MD-ordering form a flat pair.
Proof by Induction:
:},{ 1 nn vv − if
.0,1,...,2−= ni
Flat Pair for on
nv
iVV \
![Page 32: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/32.jpg)
Time Complexity
• Finding an MD-ordering in time.
• Finding all the extreme sets in time.
• Minimizing symmetric submodular functions in time.
)( 3γnO
)( 2γnO
)( 3γnO
![Page 33: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/33.jpg)
Application to Clustering
A
V
AV \nXX ,...,1
),()()( )|()();(
BABA
ABBBA
XXHXHXHXXHXHXXI−+=
−=
Random variables
VPartition into andas independent as possible
);( \ AVA XXIMinimize subject to VA ≠≠φ
AV \A
![Page 34: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/34.jpg)
Application to Clustering
)( jiVV ji ≠=∩ φ
∑=
k
iiVf
1)(Minimize
subject tokVVV ∪∪= L1
Greedy Split
)12( k− -Approximation
![Page 35: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/35.jpg)
Submodurlar Function Maximization
Approximation AlgorithmsNemhauser, Wolsey, Fisher (1978)
Monotone SF / Cardinality Constraint(1-1/e)-Approximation
Feige, Mirrokni, Vondrák (FOCS 2007)Nonnegative SF2/5-Approximation
Vondrák (STOC 2008) Monotone SF / Matroid Constraint(1-1/e)-Approximation
![Page 36: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/36.jpg)
Approximate Maximization
)()(:
}{:
}){(maxarg::
1
1
1
0
−
−
−
−=
∪=
∪==
jjj
jjj
jj
TfTf
vTT
vTfvT
ρ
φ
1−jv1vGreedy Algorithm
)(SfMaximize subject to kS ≤||
kj ,...,1=
1−jT
( ) kSSSfTf k ≤∀−≥ |:| ),(e11)(
![Page 37: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/37.jpg)
Approximate Maximization
),...,1( kj =
)(: *Sf=η:*S
∑−
=
+≤1
1
j
iijk ρρη
[ ]
jj
TSujjj
j
kTf
TfuTfTf
TSfSf
j
ρ+≤
−∪+≤
∪≤
−
∈−−−
−
∑−
)(
)(}){()(
)()( )
1
\111
1**
1*
Q
Optimal Solution
∑−
=
1
1
j
iiρ
∑=
=k
iikTf
1)( ρ
![Page 38: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/38.jpg)
Approximate Maximization
),...,1( 0 kjj =≥ρ
111:ˆ −
⎟⎠⎞
⎜⎝⎛ −=
j
j kkηρ
),...,1( kj =
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
≥
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
η
ηη
ρ
ρρ
MM
L
OOM
MO
L
kk
kk
2
1
110
100
Minimize ∑=
k
ii
1ρ
subject to
⎟⎠⎞
⎜⎝⎛ −≥
⎥⎥⎦
⎤
⎢⎢⎣
⎡⎟⎠⎞
⎜⎝⎛ −−=∑
= e11111ˆ
1ηηρ
kk
ii k
)(: *Sf=η
∑=
=k
iki Tf
1)(ρ
![Page 39: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/39.jpg)
Feature Selection
nXXY ,...,, 1
AX
)|()(:);( AA XYHYHYXI −=
Random Variables
1XYPredict from subset
2X 3X
Y“Sick”
“Fever” “Rash” “Cough”);( YXI AMaximize
subject to kA ≤|| :X Conditionally Independent
)|()( YXHXH AA −= Submodular
Krause & Guestrin (2005)
![Page 40: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/40.jpg)
Submodular Welfare Problem
(Monotone, Submodular)kff ,...,1Utility Functions
)( jiVV ji ≠=∩ φ
∑=
k
iiVf
1)(Maximize
subject tokVVV ∪∪= L1
e)11( − -ApproximationVondrák (2008)
![Page 41: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/41.jpg)
Approximating Submodular Functions
X
)(XfAlgorithm Evaluation
Oracle
Function f̂ Y)(ˆ Yf
![Page 42: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/42.jpg)
Construct a set function such that
For what function is this possible?
Approximating Submodular Functions
. ),(ˆ)()()(ˆ VXXfnXfXf ⊆∀≤≤ α
α
,0)( =φfAssumption . 0,)( VXXf ⊆∀≥Problem
f̂
1)( =nαnn =)(α
Remarksfor cut capacity functions
for general monotonesubmodular functions
![Page 43: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/43.jpg)
Approximating Submodular Functions
• Algorithm with for matroid rank functions.
• Algorithm with for monotone submodular functions.
• No polynomial algorithm can achieve a factor better than even for matroid rank functions.
)log()( nnOn =α
)log()( nnn Ω=α
1)( += nnα
Goemans, Harvey, Iwata, Mirrokni (2009)
![Page 44: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/44.jpg)
Submodular Load Balancing
?)(maxmin},...,{ 1
jjjVVVf
m
∑∈
=Xv
jvj pXf :)(
:,...,1 mffSvitkina & Fleischer (2008)
Monotone Submodular Functions
Scheduling
2-Approximation AlgorithmLenstra, Shmoys, Tardos (1990)
)log( nnO -Approximation Algorithm
![Page 45: 劣モジュラ最適化Submodular Function Minimization γ+ O n n 7 8 ( ) 5 γ O n M ( log ) 7 γ O n n ( log ) Grötschel, Lovász, Schrijver (1981, 1988) Iwata, Fleischer, Fujishige](https://reader033.fdocument.pub/reader033/viewer/2022050113/5f4a9dfbdedab3445c5fc29f/html5/thumbnails/45.jpg)
Pointers
• S. Fujishige: Submodular Functions and Optimization, Elsevier, 2005.
• ICML 2008 Tutorial Session http://www.submodularity.org
• 永野清仁,河原吉伸,岡本吉央:離散凸最適化手法による機械学習の諸問題へのアプローチ,回路とシステム軽井沢ワークショップ,2009.