Joint Inference of Multiple Label Types in Large Networks
description
Transcript of Joint Inference of Multiple Label Types in Large Networks
![Page 1: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/1.jpg)
1
Joint Inference of Multiple Label Types in Large Networks
Deepayan Chakrabarti ([email protected])
Stanislav Funiak([email protected]
)
Jonathan Chang([email protected]
)
Sofus A. Macskassy ([email protected])
![Page 2: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/2.jpg)
2
Profile Inference
Profile: Hometown: Palo Alto High School: Gunn College: Stanford Employer: Facebook Current city: Sunnyvale
Hobbies, Politics, Music, …
A complete profile is a boon: People are easily
searchable Tailored news
recommendations Group recommendations Ad targeting (especially
local)
How can we fill in missing profile fields?
??
?
![Page 3: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/3.jpg)
3
Profile Inference Use the social
network and the assumption
of homophily Friendships form
between “similar” people Infer missing labels to maximize similarity
u
v1
v2
v3
v4v5
H = Palo AltoE = Microsoft
H = MPKE = FB
H = AtlantaE = Google
H = Palo AltoE = ?
H = ?E = ?
H = ?E = ?
![Page 4: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/4.jpg)
4
Previous Work Random walks [Talukdar+/09, Baluja+/08] Statistical Relational Learning [Lu+/03,
Macskassy+/07] Relational Dependency Networks
[Neville+/07] Latent models [Palla+/12]
Either: too generic; require too much labeled data; do not handle multiple label types; are outperformed by label propagation
[Macskassy+/07]
![Page 5: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/5.jpg)
5
Previous Work Label Propagation
[Zhu+/02, Macskassy+/07] “Propagate” labels
through the network
Probability (I have hometown H)= fraction of my friends whose hometown is H
Iterate until convergence
Repeat for current city, college, and all other label types
u
v1
v2
v3
v4v5
H = Palo Alto
H = MPKH =
Atlanta
H = Palo Alto
H = ?
H = Palo Alto (0.5) MPK (0.25) Atlanta (0.25)
H = Palo Alto (…) MPK (…) Atlanta (…)
![Page 6: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/6.jpg)
6
Problem
u
H = Calcutta
H = CalcuttaCC = Bangalore
CC = Berkeley
H = ?CC = ?H = CalcuttaCC = Bangalore
Interactions between label types are not considered
![Page 7: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/7.jpg)
7
The EdgeExplain Model Instead of taking friendships as given,
explain friendships using labels
A friendship u∼v is explained if:u and v share the same hometown OR current city OR high school OR college OR employer
![Page 8: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/8.jpg)
8
The EdgeExplain Model
u
H = Calcutta
H = CalcuttaCC = Bangalore
CC = Berkeley
H = ?CC = ?H = CalcuttaCC = Berkeley
Hometown friends
Current City
friends
We set H and CC so as to jointly explain all friendships
![Page 9: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/9.jpg)
9
Find f to maximize ∏explained (fu, fv)The EdgeExplain Model
u∼v
“Soft” ORover label
types
Probability distribution for each label type
Explain all
friendships
![Page 10: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/10.jpg)
10
Find f to maximize ∏explained (fu, fv)The EdgeExplain Model
u∼v
“Soft” ORover label
types
explained (fu, fv) = softmax( is_reasont (fut, fvt) )t∊Τis_reasont (fut, fvt) = ∑ futℓ . fvtℓℓ∊L(t)softmax( is_reasont (fut, fvt) ) = σ (α . ∑ is_reasont (fut, fvt) + c) t∊Τ t∊Τ
Is u∼v explained by label type t?Chances of
sharing a label of type tSigmoid for
softmax
![Page 11: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/11.jpg)
11
H = ?CC = ?
The EdgeExplain Model
∑t is_reasont
u
H = Calcutta
CC = Berkeley
H = CalcuttaCC = Bangalore
∑t is_reasont
softmax( is_reasont (fut, fvt) ) = σ (α . ∑ is_reasont (fut, fvt) + c)
![Page 12: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/12.jpg)
12
The EdgeExplain Model
H = CalcuttaCC = ?
∑t is_reasont
u
H = Calcutta
CC = Berkeley
H = CalcuttaCC = Bangalore
∑t is_reasontH=Cal H=Cal
softmax( is_reasont (fut, fvt) ) = σ (α . ∑ is_reasont (fut, fvt) + c)
![Page 13: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/13.jpg)
13
H = CalcuttaCC = Bangalore
The EdgeExplain Model
∑t is_reasont H=Cal
Marginal gain with CC = Bangalore
u
H = Calcutta
CC = Berkeley
H = CalcuttaCC = Bangalore
∑t is_reasont H=CalH=CalCC=B’lore H=CalCC=B’lore
softmax( is_reasont (fut, fvt) ) = σ (α . ∑ is_reasont (fut, fvt) + c)
![Page 14: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/14.jpg)
14
H=CalCC=Berkeley
H = CalcuttaCC = Berkeley
The EdgeExplain Model
∑t is_reasont H=Cal
u
H = Calcutta
CC = Berkeley
H = CalcuttaCC = Bangalore
∑t is_reasont H=Cal H=CalCC=Berkeley
More gain with CC = Berkeley
softmax( is_reasont (fut, fvt) ) = σ (α . ∑ is_reasont (fut, fvt) + c)
![Page 15: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/15.jpg)
15
α controls the slope high α steep one reason per edge is enough low α linear consider multiple reasons per edge
H=CalCC=Berkeley
The EdgeExplain Modelsoftmax( is_reasont (fut, fvt) ) = σ (α . ∑ is_reasont (fut, fvt) + c)
∑t is_reasont H=Cal ∑t is_reasont H=Cal H=CalCC=Berkeley
![Page 16: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/16.jpg)
16
Experiments 1.1B users of the Facebook social network O(10M) labels 5-fold cross-validation Measure recall
Did we get the correct label in our top prediction? Top-3?
Inference: proximal gradient descent implemented via message-passing in Apache Giraph
[Ching/13] Sparsify graph by considering K closest friends by
age
![Page 17: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/17.jpg)
17
Results (varying closest friends K)
K=100 or K=200 closest friends is best K=400 hurts; these friendships are probably
due to other factors
Recall@1 Recall@3Hometo
wn Curre
nt cit
y High
scho
ol Colle
ge Emplo
yer
Lift o
f Ed
geEx
plai
n ov
er
K=20
Lift o
f Ed
geEx
plai
n ov
er
K=20
Hometo
wn Curre
nt cit
y High
scho
ol Colle
ge Emplo
yer
![Page 18: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/18.jpg)
18
Results (versus Label Propagation)
Joint modeling helps most for employer Significant gains for high school and college
as well
Hometo
wn Curre
nt cit
y High
scho
ol Colle
ge Emplo
yer
Lift o
f Edg
eExp
lain
ov
er L
abel
Pr
opag
atio
n
Hometo
wn Curre
nt cit
y High
scho
ol Colle
ge Emplo
yer
Lift o
f Edg
eExp
lain
ov
er L
abel
Pr
opag
atio
n
Recall@1 Recall@3
![Page 19: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/19.jpg)
19
Conclusions Assumption: each friendship has one reason Model: explain friendships via user attributes Results: up to 120% lift for recall@1 and 60%
for recall@3
![Page 20: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/20.jpg)
20
Result (effect of α)
High α is best one reason per friendship is enough
Lift o
f Ed
geEx
plai
n ov
er
α=0.
1Ho
metow
n Curre
nt cit
y High
scho
ol Colle
ge Em
ploy
er
![Page 21: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/21.jpg)
21
Results (varying closest friends K)
K=100 or K=200 closest friends is best K=400 hurts; these friendships are probably
due to other factors
Hometo
wn
Curren
t city
High sc
hool
College
Emplo
yer0%
20%
40%
60%
80%
100%
K=50K=100K=200K=400
Lift
over
K=
20
Hometo
wn
Current c
ity
High sc
hool
College
Employ
er0%
20%
40%
60%
80%
100%
K=50K=100K=200K=400
Lift
over
K=
20
Recall@1 Recall@3
![Page 22: Joint Inference of Multiple Label Types in Large Networks](https://reader035.fdocument.pub/reader035/viewer/2022062811/5681601c550346895dcf1a63/html5/thumbnails/22.jpg)
22
Results (versus Label Propagation)
Joint modeling helps most for employer Significant gains for high-school and college
as well
Hometo
wn
Curren
t city
High sc
hool
College
Emplo
yer-20%
0%20%40%60%80%
100%120%140%
K=20K=50K=100K=200K=400
Lift o
ver L
abel
Pro
paga
-tio
n
Hometo
wn
Curren
t city
High sc
hool
College
Emplo
yer-20%
0%
20%
40%
60%
80%
K=20K=50K=100K=200K=400
Lift o
ver L
abel
Pro
paga
-tio
n
Recall@1 Recall@3