Reporter: 謝凱旭 Advisor: 曾學文 教授 An Efficient Multicast Scheduling based on Social...
-
Upload
kristin-byrd -
Category
Documents
-
view
231 -
download
4
Transcript of Reporter: 謝凱旭 Advisor: 曾學文 教授 An Efficient Multicast Scheduling based on Social...
1
Reporter: 謝凱旭Advisor: 曾學文 教授
An Efficient Multicast Scheduling based on Social Network
in Data Center Networks
2
Outline
• Introduction• Related work• Multicast scheduling based on social network
(MSSN)• Mathematical analysis• Experiment results• Conclusions
3
Introduction
• Cloud services:– Amazon EC2, Facebook, Twitter, GFS and HDFS– Use multicasting widely
• Multicast benefits data center group – Avoid sending unnecessary duplicated packets– Reduce the task finish time of delay-sensitive applications
• But– Large and unbalanced multicast traffic in DCN
• Justin Bieber has 50,000,000 followers
– More social traffic in the world• Facebook: 22.36%
– Different types: words, voices, and videos • Complex
4
Introduction
• Twitter’s DCN– Uniform Resource Locator , URL
• Retweets• Favorites• Replies
– Engagement : Popular weights– Aggregation : Combined data by ID– Ingestion : Fetch features– Scorer: Scored by feature– Partitioner: Divide data– HDFS: Store data
Tweets
ImageURLs
VideosURLs
NewsURLs
BlogURLs
URLFetch
Schedule URLs
...
Aggregation IngestionEngagement
ScorerPartitonerHDFS
...
[1] https://blog.twitter.com/2014/building-a-complete-tweet-index[2] https://blog.twitter.com/2011/spiderduck-twitters-real-time-url-fetcher
5
Introduction
• For Twitter:– Open source
• Easy to fetch data
– Multicast tree = 1 user + many followers (members)• Members’ data in the same cluster • Many clusters in the same pod
• Twitter– 600 million users– Video traffic is more popular
• YouTube and Flicker• Tweet:0.2KB / Picture:0.7MB / Video:5MB • Video traffic in 2013: 1600 PB per month
Easily cause traffic congestion
6
Introduction
• Real-time response is critical in social DCNs– Packet latency in DCN: 200-500 μs– Congestion: >10ms (~200X) – Less congestion is better
– Problems in DCN• A lot of multicast trees in DCN• Overlapped paths Congested• Overlapped nodes High traffic• High video traffic Traffic congestion
User1 User2
Follower1 Follower2 Follower3 Follower4
Group1
Group2
Potential Hotspots
7
Related Work
• Related work– Traditional multicast scheduling in Network
• Only consider layer 3 routers• Hybrid in DCNs
– Layer 2 / Layer 3
– Multicast scheduling in DCNs– Multicast scheduling in Social DCNs
8
Related Work
• Multicast Scheduling in DCNs– Congestion occurs
• The core switches at the top manage the traffic– If nodes in low level is congested, they response to core switches– Core switches allocate traffic
• Not for social network– Centralized scheduler
» Not real-time
On-Line Multicast Scheduling with Bounded Congestion in Fat-Tree Data Center Networks, IEEE Journal on Selected Areas in Communications, vol. 32, no. 1, Jan. 2014
9
• Multicast Scheduling in Social DCNs
– Facebook’s 4-post DCN • In order to provide real time services for large delay-sensitive apps
– Disadvantages of 4-post architecture• The cluster size is dictated by the size of the CSW• Large switches are often oversubscribed internally
– Not all of the ports can be used simultaneously
Related Work
Facebook's data center network architecture, Optical Interconnects Conference, 2013 IEEE????
rack switch
cluster switch
aggregation switches
protection ring
10
Related Work
• The disadvantages of multicast scheduling– DCNs
• Not for social network• Can not support real time services of delay-sensitive apps
– Social DCNs• Too many multicast groups in DCN• Overlapped paths and nodes• Need multiple group scheduling
Our MSSN
Wongyai, W.; Charoenwatana, L., "Examining the network traffic of facebook homepage retrieval: An end user perspective," Computer Science and Software Engineering (JCSSE), 2012 International Joint Conference on , vol., no., pp.77,81, May 30 2012-June 1 2012
User1 User2
Follower1 Follower2 Follower3 Follower4
Group1
Group2
Potential Hotspots
11
Multicast scheduling based on Social Network (MSSN)
• MSSN– Traffic Injection– Issue Filter– Congestion Detection– Load Balance Policy
Traffic Injection
Congestion Detection
Load Balance Policy
If TrafficHotspots
> ThCongestion
Yes
No
Is Potential Hotspots?
Yes
No Traffic Congestion?
Yes No
Issue Filiter
User1 User2
Follower1 Follower2 Follower3 Follower4
Group1
Group2
Potential Hotspots
Network
m
iFlow BWSize
i
1
12
• Flow Manager (FM) manages traffic in the same Pod– Node locality in a tree is 84% in the same Pod [2]
• Overlapped routing paths and nodes– High social popularity
• High degree centrality[1]• High Traffic Load
Traffic congestion
Traffic Injection
[1] A social popularity aware scheduling algorithm for ad-hoc social networks, JCSSE, 2014[2] Delay Scheduling: A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling, EuroSys, 2010?????
13
Potential Hotspots List Safe node Potential Hotspots Source Destination Weight (MB) Weight Tag
A B C D E F G1 1 5 1 1 5 1 N 5 1 1 5
1 2 5 1 2 5
1 4 5
2 3 0.7
2 3 0.72 4 0.7
2 4 0.72 N 0.7
K N 0.0002 K N 0.0002
K 3 0.0002
K 4 0.0002
K 2 0.0002
ThCongestion
Potential Hotspots List: B E
Network
m
iFlow BWSize
i
1
AB
C
D
...
...
User1 User2 UserK
Friend1 Friend2 Friend3 Friend4 FriendN
EF
G...
SwitchFriend
Potential Hotspots List (PHL)
NetworkBWCongestionTh
//Traffic Burst
//Overlapped nodes
Cloud Analytics for Capacity Planning and Instant VM Provisioning, TNSM, IEEE Transactions on , 2013
14
(5min)
Safe node Potential Hotspots Source Destination Weight (MB) Weight Tag
A B C D E F G1 1 5 1 1 5 1 N 5 1 1 5
1 2 5 1 2 5
1 4 5
2 3 0.7
2 3 0.72 4 0.7
2 4 0.72 N 0.7
K N 0.0002 K N 0.0002
K 3 0.0002
K 4 0.0002
K 2 0.0002
ThCongestion
Potential Hotspots List: B E
15
Issue Filter & Congestion Detection
1Flow },...,,{ 11211 iMMM
2Flow },...,,{ 22221 jMMM
KFlow },...,,{ 21 KkKK MMM…
B
• Consider PHL• There are common issues (tag): in these flows (20%)• FM monitors the switches which contain flows with
iM1
iM1
AB
C
D
...
...
User1 User2 UserK
Friend1 Friend2 Friend3 Friend4 FriendN
EF
G...
SwitchFriend
• If ( > )– Start Load Balance
• Else – None
iSwitchDegree NetworkBW
16
Load Balance Policy
AB
C
D
...
...
User1 User2 UserK
Friend1 Friend2 Friend3 Friend4 FriendN
EF
G...
SwitchFriend
17
Experiment
For real world scenario:• Use Twitter API to fetch traffic between 3/3-3/10, 2015 (17:30~16:30)• Inject to NS3
18
Experiment
• For real world scenario– Multicast trees : 50~250 [1]– Members of a tree : 3~1000 [1]– Number of Pod : 10 [2]– Links between GS and AS : 10Gbps– Links between AS and ES : 1Gbps– Available link bandwidth : 30~100% [1]
• For real time response– Total throughput / total success delivery ratio– Each success delivery ratio of different types of tweets
[1]Reliable Multicast in Data Center Networks, TC, 2014[2]3D Beamforming for Wireless Data Centers, HotNets, 2011
Experiment
Compared with BCMS: 11.30%159.04GBCompared with Twitter: 7.20%101.34GB
BCMS:6.70%
Compared with BCMS: 6.70%52774.49 tweets (pic/video/text)Compared with Twitter: 2.70%21267.33 tweets (pic/video/text)
Throughput Success Delivery Ratio
20
ExperimentThroughput Success Delivery Ratio
Compared with BCMS: 4.75%27.22GBCompared with Twitter: 1.64%9.40GB
Compared with BCMS: 4.74%14824.21 tweets (pic/video/text)Compared with Twitter: 1.63%5097.78 tweets (pic/video/text)
21
Experiment
Video Picture Text
Compared with BCMS: 6.24%236.90 tweets Compared with Twitter: 2.39%90.75 tweets
Compared with BCMS: 5.41%19.79 tweets Compared with Twitter: 2.11%7.72 tweets
Compared with BCMS: 4.78%37440.6 tweets Compared with Twitter: 2.11%13415.1 tweets
22
Analytical Model
S17CS
AS
ES
Racks
HostSwtich
H0 H1 H3H2 H4 H5 H6 H7 H8 H10 H11 H12 H13 H14 H15
S0 S1 S2 S3 S4 S5 S6 S7
S9S8 S10 S11 S12 S13 S14 S15
S19S18S16
H9
Host
Swtich S6
S14
S0
S8
S16
S10 S12
S4 S5
H13 H1
H12
H14 H15
H10H9H8H4 H5
S17S7
S2T1
T2
Overlapped Path
• To simplify – Group number: 50 到 250– Member number: 3 到 1000– Pod number:4– Host number:40– CS,AS and AS,ES:10Gpbs– ES,ToR:1Gbps
Analytical Model-Throughput共用 Host 節點所收到的流量
共用 Switch 節點所收到的流量
每個 Multicast tree 所產生的流量
Multicast tree 的數目
Host
Swtich S6
S14
S0
S8
S16
S10 S12
S4 S5
H13 H1
H12
H14 H15
H10H9H8H4 H5
S17S7
S2T1
T2
Overlapped Path
25
Analytical Model-Throughput
整體網路的產量 整體網路經過所有 Switch 的流量
所有接收端所接收到的流量總和大小 所有的 host 節點數
Switch 所接收到的流量總和大小
所有的 Switch 數
Host
Swtich S6
S14
S0
S8
S16
S10 S12
S4 S5
H13 H1
H12
H14 H15
H10H9H8H4 H5
S17S7
S2T1
T2
Overlapped Path
26
Analytical Model-Throughput
Tweet 的大小 Picture 大小 Video 大小整體網路的產量
所有接收端所接收到的流量總和大小
Tweet 的大小 Picture 大小 Video 大小
Analytical Model-Standard Deviation
27
節點所收到的平均產量
Switch 所收到的平均流量
For Host=9397350=8=0.2MB=22710.84bpsSimulation: 21437.72bpsError ratio=5.94%