1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang...
-
Upload
rosalind-heath -
Category
Documents
-
view
224 -
download
0
Transcript of 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang...
![Page 1: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/1.jpg)
1April 21, 2023Network Appliance Confidential |
System EngineerClive [email protected]
CDN的規劃與建置
SimpleFastReliable
![Page 2: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/2.jpg)
2April 21, 2023Network Appliance Confidential |
CDN 的基本架構
Content DeliveryContent Storage
Content Reporting
Content Distribution
GSLB
![Page 3: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/3.jpg)
3April 21, 2023Network Appliance Confidential |
CDN 的基本元件
Content Storage 資料層 (Data Layer) Content Delivery 資料層 (Data Layer) Content Distribution 資料層 (Data Layer) Content Reporting 資料層 (Data Layer) Global Request Redirection 網路層 (Network Layer)
![Page 4: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/4.jpg)
4April 21, 2023Network Appliance Confidential |
Content Storage Architecture
Internet
Web Server Web Server Media Server Media Server
Load Balancer
Server Accelerator Farm
Content Storage
VLAN3
VLAN2
VLAN1
Data Center
Web Volume Media Volume
• A web volume shares to all web servers
• A media volume shares to all media servers
![Page 5: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/5.jpg)
5April 21, 2023Network Appliance Confidential |
Content Delivery – On demand Caching
Media Server
Internet
Content Delivery
Encoding Server
![Page 6: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/6.jpg)
6April 21, 2023Network Appliance Confidential |
Local DNS Srvr
(1)(5)
Trace of DNS queries for CDN sub-domain
CDN Provider
DNS Srvr(.x.com)
(4)
(3)(2)(6)
GRM DNS Srvr(.cdn.x.com)Top Level
DNS Srvrs(., .com)
(7)
InterNIC, etc.
Customer "x"ISP
![Page 7: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/7.jpg)
7April 21, 2023Network Appliance Confidential |
Local DNS Srvr
GRM DNS Server Redundancy
GRM DNS Srvr1
DNS ResponseNS: GRM DNS Srvr1NS: GRM DNS Srvr2
Local DNS Server will failover quickly to alternative GRM DNS Servers if there is no response
Local DNS Server will failover quickly to alternative GRM DNS Servers if there is no response
Response can contain multiple NS records
Response can contain multiple NS records
Top LevelDNS Srvrs
GRM DNS Srvr2
![Page 8: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/8.jpg)
8April 21, 2023Network Appliance Confidential |
Local DNS Srvr
GRM DNS Server Response
GRM DNS SrvrAgent3Agent2Agent1
DNS Responses1st: IP1, IP2, IP32nd: IP2, IP3, IP13rd: IP3, IP1, IP2
IP3IP2IP1
Failover– TTL is set low: changes are reflected quickly
Load-Balancing (crude, but effective)– The order is rotated for each response– Clients normally use 1st IP address– Note: Win2K uses "subnet prioritization"
and picks "closest" IP address`
Local DNS Server may cache result– Typically rotates addresses in responses
Failover– TTL is set low: changes are reflected quickly
Load-Balancing (crude, but effective)– The order is rotated for each response– Clients normally use 1st IP address– Note: Win2K uses "subnet prioritization"
and picks "closest" IP address`
Local DNS Server may cache result– Typically rotates addresses in responses
Response contains multiple IP addresses
Response contains multiple IP addresses
![Page 9: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/9.jpg)
9April 21, 2023Network Appliance Confidential |
Local DNS Srvr
Effect of Local DNS Server
GRM DNS Srvr
Agent1
Agent2
Assumption Client is close to its
local DNS server (LDNS)
Assumption Client is close to its
local DNS server (LDNS)
GRM DNS Server receives request from LDNS
– Proximity measurements are from agents to LDNS
GRM DNS Server receives request from LDNS
– Proximity measurements are from agents to LDNS
DNS Queries and Responses
Proximity Measurements
![Page 10: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/10.jpg)
10April 21, 2023Network Appliance Confidential |
Main Data Center
(7)
DNS Srvr
(1)(5) IP = DNS Agent2
DNS Based Routing:When it works
Remote Location
DNS Srvr
(4)
(3)
(2)
(6)
DNS Queries and Responses
Application Level Request
Proximity Measurements
DNS Agent1
DNS Agent2
DNS Srvr
Top LevelDNS Srvrs
![Page 11: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/11.jpg)
11April 21, 2023Network Appliance Confidential |
(3)
L7 Srvr
DNS Srvr
(1)
(2) IP = L7 Srvr
L7 Redirection
Remote Location
L7 Agent
Main Data Center
DNS Queries and Responses
Application Level Request
Redirection
![Page 12: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/12.jpg)
12April 21, 2023Network Appliance Confidential |
(7)
DNS Agent1
DNS Srvr
Top LevelDNS Srvrs
(1)
DNS + L7 Based Request Redirection
Remote Location
L7 Agent1DNS Agent 2
(6)
Main Data Center
DNS Queries and Responses
Application Level Request
Redirection
L7 Srvr
DNS Srvr
(5) IP = DNS Agent1
(4)
(3)
(2)
![Page 13: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/13.jpg)
13April 21, 2023Network Appliance Confidential |
Process:if (file type matches { "ppt", "pdf", ... }) {
redirect }else { serve request directly }
Process:if (file type matches { "ppt", "pdf", ... }) {
redirect }else { serve request directly }
Configurable List
Conditional Redirection Serving average sized files (~14kB) is quite fast Only redirect if file size is "likely" to be large
Conditional Redirection Serving average sized files (~14kB) is quite fast Only redirect if file size is "likely" to be large
HTTP Redirection
![Page 14: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/14.jpg)
14April 21, 2023Network Appliance Confidential |
RTSP Redirection
(1) Request
(2) Redirect
(3) Request
L7 Server
L7 Agent
(1) RequestGET rtsp://srvr/file.rm RTSP/1.0
(2) Response302 RedirectLocation: rtsp://cache/srvr/file.rm
(3) RequestGET rtsp://cache/srvr/file.rm RTSP/1.0
![Page 15: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/15.jpg)
15April 21, 2023Network Appliance Confidential |
MMS Redirection
Rewritten NetCache Response<ASX>
<ref href="mms://agent-ip/strm-srvr/strm.asf"/>
</ASX>
Browser Requesthttp://www.cdn.x.com/file.asx
Original Web Server Response<ASX>
<ref href="mms://strm-srvr/strm.asf"/>
</ASX>
The MMS protocol has no redirect capability ASX rewriting can be used to achieve a similar affect
The MMS protocol has no redirect capability ASX rewriting can be used to achieve a similar affect
![Page 16: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/16.jpg)
16April 21, 2023Network Appliance Confidential |
Load Balancing and Failover
Remote LocationMain Data Center
Primary location
Failover location
Clients A Clients B
Center CachesEdge Caches for Clients A
Edge Caches for Clients B
![Page 17: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/17.jpg)
17April 21, 2023Network Appliance Confidential |
Example Criteria for System Selection
System Selection
Criteria Notes
System HealthFailure or overload is a cause for removal from list of available systems.
Domain Based PoliciesAllow restriction based on location, supported protocols, etc.
Geographical LocationChoice can be narrowed down (at least to continent) by matching client IP address with IANA database
Client-Node ProximityDistance of client from each system can be determined dynamically.
Proximity information can also be preloaded.
![Page 18: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/18.jpg)
18April 21, 2023Network Appliance Confidential |
Performance Planning
HTTP Forward and Reverse Proxy Cache– Target web objects capacity– Total disk numbers– Total disk space– Network interface
Target web site– Total web site capacity
Testing Program– Ex, Polymix4
Measurement– URL/s– Response Time
![Page 19: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/19.jpg)
19April 21, 2023Network Appliance Confidential |
Performance Planning
WM Streaming Reverse Proxy Cache (non-transparent)– Total disk numbers ( 硬碟顆數要足夠 )– Total disk space– GbE Network interface ( 網路介面卡作 Trunking)– License ( 選用最高等級 )
Target Media Streaming– Bit rate (ex, 300Kbps,750Kbps,1Mbps,2Mbps,3Mbps)– Film length (ex, 60 min,90min)– Film numbers ( 數量越多越好 )
Testing Program– Windows Media Load Simulator
Measurement– ON-demand caching concurrent users
• Hit from Disk ( 所有影片是由 Disk 提供 )• Hit from Memory ( 所有影片是由 Memory 提供 )
– Live Splitting concurrent users
![Page 20: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/20.jpg)
20April 21, 2023Network Appliance Confidential |
Singapore MOE
7 NetCaches (2 x C6100, 5 x C3100) with MMS,Real, QuickTime license
2xC6100 for their Internet Zone with Smartfilter 3.0 (NOC1 & NOC2)
– Layer 4 (Foundry ServerIron 400) for Transparent Caching (GE Interface)
4xC3100 for the Schools WAN Routers with about 400 schools (North,South,East & West)
– WCCP Enabled (FE Interface)
1xC3100 for the MOE HQ LAN Staff– WCCP Enabled (GE Enabled)
![Page 21: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/21.jpg)
Network Appliance Confidential21
TANET 之 CDN 應用範例參考
![Page 22: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/22.jpg)
22April 21, 2023Network Appliance Confidential |
CDN System Architecture
Internet
Web / MediaServer
DB Server
NetAppPrimary Storage
NetCache
Web MediaReverse Proxy
NetCache
GRM Agent
NetCache
GRM Agent
NetCache
GRM Agent
Data CenterRemote Site
Encoding/Editing Servers
GRM System
![Page 23: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/23.jpg)
23April 21, 2023Network Appliance Confidential |
HTTP
DNS Server
Top LevelDNS Servers
GRM System Architecture
Remote Location
GRM L7 Agent1HTTP,MMS,Real
(6)
(1)
Data Center
DNS Queries and Responses
Application Level Request
(7) Redirection
GRM DNS Agent
GRM L7 Server2
(3)
HTTP
GRM DNS Agent
GRM L7 Server1
DFM CR
GRM L7 Agent2HTTP,MMS,Real
GRM L7 Agent1HTTP,MMS,Real
GRM L7 Agent2HTTP,MMS,Real
GRM Edge Group1
GRM Edge Group2
Streaming(8)
WMS
AuthoritativeDNS Server
(4)
(5)GRM DNSServer1
GRM DNSServer2
(2)
![Page 24: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/24.jpg)
24April 21, 2023Network Appliance Confidential |
GRM Domain
在 edu.tw 下建立一個新的 domain=cdn.edu.tw– 屬於 GRM 轉向管理的 domain
此 domain 下有兩個 name server– GRMDNS1 ip1– GRMDNS2 ip2
在 cdn.edu.tw 下建立兩個 host record– grm ip3– grm ip4
每個地理區域依 CIDR 的集合來建立 Edge group– 每個 Edge group 依需求放置一台或多台的 NetCache
![Page 25: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/25.jpg)
25April 21, 2023Network Appliance Confidential |
特色
不需使用 L4 switch 或 WCCP router 作透通式代理 不需使用 GSLB 設備來達到全球負載平衡 只需設計單一連結即可 . 任何地區的使用者的要求 ( 包括 http
和 streaming) 都會從當地的 Agent 直接提供服務 . Ex,– http://grm.cdn.edu.tw/page.html– http://grm.cdn.edu.tw/run.asp– http://grm.cdn.edu.tw/play.asx
可自行指定要轉向所需的副檔名 每個 NetCache 可指定成只擔任反向代理模式 (Reverse Prox
y) 的 Web 及 Streaming Accelerator ,避免被使用者任意指定為正向代理快取伺服器來使用
![Page 26: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/26.jpg)
26April 21, 2023Network Appliance Confidential |
特色 Name server 本身具有 DNS 的備援性 GRM DNS Server 會自動選擇合適的 GRM L7 server
– GRM L7 server 會自動負載平衡– 可隨時依負載狀況增加 GRM L7 server
GRM L7 Server 會自動選擇合適的 GRM Agent– 每個 Edge group 內的 Agent 會自動負載平衡– 可隨時依負載狀況增加 GRM Agent
任何一套系統的單一顆硬碟故障時– 不會造成服務中斷– 不會造成檔案流失– 不會造成日誌流失– 備援硬碟會自動進行重建 (Auto Rebuild)– 自動發出 email 和 SNMP 通知系統管理人員
![Page 27: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/27.jpg)
27April 21, 2023Network Appliance Confidential |
TANET 之 CDN 應用範例參考查榜系統架構說明
NetApp Primary Storage– 可加速資料庫的存取速度
NetApp NetCache– 可加速網路的存取速度– Data Center
• 保護 Web Server• 減輕防火牆負擔
– Remote Site• 加快網頁的回應速度
GRM– 不需使用 L4 Switch/WCCP router 作透通式代理– 不需針對不同區域撰寫不同的網頁連結– 系統會自動將 user request 轉向到最佳的 NetCache
![Page 28: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/28.jpg)
28April 21, 2023Network Appliance Confidential |
查榜系統方案一
使用 asp?no=12345 方式來查詢 設定 NetCache 對此可強迫快取,並設定 TTL 結果
– 第一次的查詢結果從 Server 讀出– 第二次的查詢結果從 NetCache 讀出– 可減輕 Server 的負載– 加快查詢的回應速度
![Page 29: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/29.jpg)
29April 21, 2023Network Appliance Confidential |
查榜系統方案二
將所有結果根據準考證號碼以程式產生對應的 html– 使用 no_12345.html 方式來給予查詢結果– 在 Web Server 上對這些 html 設定 TTL
利用 DFM 預先將這些 html 全部派送到 NetCache– 可另外設定 min-age / TTL / Lock time
結果– 所有的查詢結果都從 NetCache 讀出– 大幅減輕 Server 的負載– 加快查詢的回應速度
![Page 30: 1 Network Appliance Confidential |September 6, 2015 System Engineer Clive Chiang Clive.Chiang@netapp.com CDN 的規劃與建置 Simple Fast Reliable.](https://reader036.fdocument.pub/reader036/viewer/2022062322/56649e195503460f94b06c2e/html5/thumbnails/30.jpg)
Network Appliance Confidential30
Q & A
Thank You