Herokuアプリ スケールアウト構成、
性能測定・可視化、性能改善法
Marat Zhanikeev [email protected]
maratishe.github.io
2015/04/11
Cloudy@tenjin
Heroku Apps Scale-out, performance
monitoring・visualization・management
.
maratishe.github.io
Today we...
1. review what scale -up -down -OUT are
2. review heroku design and features3. discuss cloud and specifically heroku populations4. run a measurement experiment and find out that heroku lives in theVirginia DC
5. formulate the heroku bottle model to explain experimental results
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 2/37...
2/37
.
maratishe.github.io
Scale-?
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 3/37...
3/37
.
maratishe.github.io
Scalability, scale -up -down -OUT
.Scalability..
.
...ability of a system to increase or decrease scale drasticallywithout much change in complexity
• (poor but still an) example: multi-thread programming• scale-up: interprocess communication over the network• scale-down: multicore parallel processing• ... heroku needed a new term -- the scale-out concept
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 4/37...
4/37
.
maratishe.github.io
Scale-OUT
.scale-out feature..
.
... represents a design in which the same item (source code) is replicatedand forms a flat (1-level) population
• basically, zero or very low complexity
• complexity does not change because population is flat• several active replicas of the same code is a kind of parallel programming
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 5/37...
5/37
.
maratishe.github.io
heroku basics
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 6/37...
6/37
.
maratishe.github.io
heroku : your code is a webapp
• feature 1: heroku apps are already webapps
• written, built, and ran as web apps
• so, heroku environment is very web-like◦ even for non-web code in the background
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 7/37...
7/37
.
maratishe.github.io
heroku : scaling• feature 2: scaling is easy• both from web dashboard and heroku CLI
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 8/37...
8/37
.
maratishe.github.io
heroku : scaling (2)• for example, setting scale to 15 literaly means that there are 15 replicas of mycode running in parallel
• this is an academic account, one would normally get a huge amount in the bill
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 9/37...
9/37
.
maratishe.github.io
heroku : scale is money
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 10/37...
10/37
.
maratishe.github.io
heroku : scale-out basics
• when you increase the scale, you getparallel replicas of your code
• your code can be web or workerdynos◦ dyno = process = container
• which one gets requests? heroku usescedar stack, which selects webdynos randomly
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 11/37...
11/37
.
maratishe.github.io
heroku : front and back
• feature 3 (or a bug?): webs andworkers cannot communicate directly
• webs get an incoming port forrequests, workers have no ports
• the only solution is to use 3rd partyspace to communicate
• again, parallel processing withlocking, etc., is applicable
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 12/37...
12/37
.
maratishe.github.io
heroku deploy is by git (feature 4)
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 13/37...
13/37
.
maratishe.github.io
heroku performance monitoring
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 14/37...
14/37
.
maratishe.github.io
heroku evolution
• not much monitoring until the late 2014 version• in 2014 version, some 3rd party tools became part of core functionality =part of dashboard
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 15/37...
15/37
.
maratishe.github.io
heroku performance metrics
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 16/37...
16/37
.
maratishe.github.io
heroku performance metrics (2)
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 17/37...
17/37
.
maratishe.github.io
heroku performance metrics (3)
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 18/37...
18/37
.
maratishe.github.io
what's missing?
• nothing really, basically all metrics are covered
• ... maybe no visualization for your entire populations (of dynos)
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 19/37...
19/37
.
maratishe.github.io
heroku populations
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 20/37...
20/37
.
maratishe.github.io
heroku populations
• heroku requests are routed by heroku nodes (bottlenecks?)• your app to addons: inner? in-DC? depends on add-on?• your app to users: outer!• your app to 3rd party services: outer!
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 21/37...
21/37
.
maratishe.github.io
heroku : outer vs inner
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 22/37...
22/37
.
maratishe.github.io
populations : VM version, streaming)
VMpopulation
CurrentSources
ServiceProvider
(SP)
Tracker
ServiceProvider
(SP)
Parentpeers
P2Pstreaming
Cloudstreaming
ClientClient
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 23/37...
23/37
.
maratishe.github.io
populations : docker exampel
• Docker is a DiY platform, youhave to do everything by yourself
• proper population managementtechnology is in demand
• can include geographicaldistribution, topology control,etc.
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 24/37...
24/37
.
maratishe.github.io
are heroku dynos populations?
• any difference in performance across dynos?• where are the bottlenecks?• can one measure? visualize for entire dyno populations?
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 25/37...
25/37
.
maratishe.github.io
heroku probing experiment
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 26/37...
26/37
.
maratishe.github.io
heroku probing experiment
• need outside of heroku : EC2◦ might be poor choice because heroku itself is based entirely on EC2
• send probes from various EC2 regions to visualize regional diversity• need to try out various scales and gradual scaling
◦ gradual scaling is so that dynos would be mapped to different VMs
• the probing is when EC2 request and download some bulk from a heroku app
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 27/37...
27/37
.
maratishe.github.io
heroku probing : EC2 + heroku• we can only hope that dynos are mapped to different VMs/PMs
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 28/37...
28/37
.
maratishe.github.io
heroku probing (2) 1 run
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 29/37...
29/37
.
maratishe.github.io
results
0
10
20
30
40
Thru
1k50
0k 1M 2M 3M
california.1b/1 california.1b/2 california.1b/3 california.1b/4 california.1b/5 california.1b/6 california.1b/7 california.1b/8 california.1b/9 california.1b/10 california.1b/11 california.1b/12 california.1b/13 california.1b/14 california.1b/15
frankfurt.1a/1 frankfurt.1a/2 frankfurt.1a/3 frankfurt.1a/4 frankfurt.1a/5 frankfurt.1a/6 frankfurt.1a/7 frankfurt.1a/8 frankfurt.1a/9 frankfurt.1a/10 frankfurt.1a/11 frankfurt.1a/12 frankfurt.1a/13 frankfurt.1a/14 frankfurt.1a/15
ireland.1a/1 ireland.1a/2 ireland.1a/3 ireland.1a/4 ireland.1a/5 ireland.1a/6 ireland.1a/7 ireland.1a/8 ireland.1a/9 ireland.1a/10 ireland.1a/11 ireland.1a/12 ireland.1a/13 ireland.1a/14 ireland.1a/15
oregon.2b/1 oregon.2b/2 oregon.2b/3 oregon.2b/4 oregon.2b/5 oregon.2b/6 oregon.2b/7 oregon.2b/8 oregon.2b/9 oregon.2b/10 oregon.2b/11 oregon.2b/12 oregon.2b/13 oregon.2b/14 oregon.2b/15
saopaulo.1a/1 saopaulo.1a/2 saopaulo.1a/3 saopaulo.1a/4 saopaulo.1a/5 saopaulo.1a/6 saopaulo.1a/7 saopaulo.1a/8 saopaulo.1a/9 saopaulo.1a/10 saopaulo.1a/11 saopaulo.1a/12 saopaulo.1a/13 saopaulo.1a/14 saopaulo.1a/15
singapore.1a/1 singapore.1a/2 singapore.1a/3 singapore.1a/4 singapore.1a/5 singapore.1a/6 singapore.1a/7 singapore.1a/8 singapore.1a/9 singapore.1a/10 singapore.1a/11 singapore.1a/12 singapore.1a/13 singapore.1a/14 singapore.1a/15
sydney.2b/1 sydney.2b/2 sydney.2b/3 sydney.2b/4 sydney.2b/5 sydney.2b/6 sydney.2b/7 sydney.2b/8 sydney.2b/9 sydney.2b/10 sydney.2b/11 sydney.2b/12 sydney.2b/13 sydney.2b/14 sydney.2b/15
tokyo.1c/1 tokyo.1c/2 tokyo.1c/3 tokyo.1c/4 tokyo.1c/5 tokyo.1c/6 tokyo.1c/7 tokyo.1c/8 tokyo.1c/9 tokyo.1c/10 tokyo.1c/11 tokyo.1c/12 tokyo.1c/13 tokyo.1c/14 tokyo.1c/15
virginia.1e/1 virginia.1e/2 virginia.1e/3 virginia.1e/4 virginia.1e/5 virginia.1e/6 virginia.1e/7 virginia.1e/8 virginia.1e/9 virginia.1e/10 virginia.1e/11 virginia.1e/12 virginia.1e/13 virginia.1e/14 virginia.1e/15
• figure is vector, so,zoom-in
• rows: DCs, columns:dynos
• x: bulk size, y:throughput
• does any DCstand out?
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 30/37...
30/37
.
maratishe.github.io
results (without virginia)
0
4
8
12
Thru
1k50
0k 1M 2M 3Mcalifornia.1b/1 california.1b/2 california.1b/3 california.1b/4 california.1b/5 california.1b/6 california.1b/7 california.1b/8 california.1b/9 california.1b/10 california.1b/11 california.1b/12 california.1b/13 california.1b/14 california.1b/15
frankfurt.1a/1 frankfurt.1a/2 frankfurt.1a/3 frankfurt.1a/4 frankfurt.1a/5 frankfurt.1a/6 frankfurt.1a/7 frankfurt.1a/8 frankfurt.1a/9 frankfurt.1a/10 frankfurt.1a/11 frankfurt.1a/12 frankfurt.1a/13 frankfurt.1a/14 frankfurt.1a/15
ireland.1a/1 ireland.1a/2 ireland.1a/3 ireland.1a/4 ireland.1a/5 ireland.1a/6 ireland.1a/7 ireland.1a/8 ireland.1a/9 ireland.1a/10 ireland.1a/11 ireland.1a/12 ireland.1a/13 ireland.1a/14 ireland.1a/15
oregon.2b/1 oregon.2b/2 oregon.2b/3 oregon.2b/4 oregon.2b/5 oregon.2b/6 oregon.2b/7 oregon.2b/8 oregon.2b/9 oregon.2b/10 oregon.2b/11 oregon.2b/12 oregon.2b/13 oregon.2b/14 oregon.2b/15
saopaulo.1a/1 saopaulo.1a/2 saopaulo.1a/3 saopaulo.1a/4 saopaulo.1a/5 saopaulo.1a/6 saopaulo.1a/7 saopaulo.1a/8 saopaulo.1a/9 saopaulo.1a/10 saopaulo.1a/11 saopaulo.1a/12 saopaulo.1a/13 saopaulo.1a/14 saopaulo.1a/15
singapore.1a/1 singapore.1a/2 singapore.1a/3 singapore.1a/4 singapore.1a/5 singapore.1a/6 singapore.1a/7 singapore.1a/8 singapore.1a/9 singapore.1a/10 singapore.1a/11 singapore.1a/12 singapore.1a/13 singapore.1a/14 singapore.1a/15
sydney.2b/1 sydney.2b/2 sydney.2b/3 sydney.2b/4 sydney.2b/5 sydney.2b/6 sydney.2b/7 sydney.2b/8 sydney.2b/9 sydney.2b/10 sydney.2b/11 sydney.2b/12 sydney.2b/13 sydney.2b/14 sydney.2b/15
tokyo.1c/1 tokyo.1c/2 tokyo.1c/3 tokyo.1c/4 tokyo.1c/5 tokyo.1c/6 tokyo.1c/7 tokyo.1c/8 tokyo.1c/9 tokyo.1c/10 tokyo.1c/11 tokyo.1c/12 tokyo.1c/13 tokyo.1c/14 tokyo.1c/15
• 2 groups of DCs?
• absolutethroughput is 4times smaller thanwithin virginia
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 31/37...
31/37
.
maratishe.github.io
heroku : the bottle model
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 32/37...
32/37
.
maratishe.github.io
heroku : the bottle model
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 33/37...
33/37
.
maratishe.github.io
intRA-DC throughput (pressure)
california
ireland
oregon
saopaulo
singapore
sydneytokyo virginia
key (probe)sizes (2000 5000)parties (aa)
• visualization from earlier Cloudy oncloud probing
• in-DC throughput wildly differentacross DCs
• but heroku (in US) leaves in Virginia, soirrelevant?
• ... still, traffic inside DCs is under somepressure
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 34/37...
34/37
.
maratishe.github.io
intER-DC throughput (pressure)
california
ireland
oregon
saopaulo
singapore
sydney
tokyo
virginia
key (probe)sizes (2000 5000)parties (ab)
• average across DCs is mostly thesame for all DCs◦ all-to-all has smoothing effect
• pullees show the slowestremote party for a given DC, there issome difference here
• this is applicable to heroku because thistraffic goes through the bottleneck
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 35/37...
35/37
.
maratishe.github.io
The Big Questions
1. How strong the effect of bottlenck compared to:◦ in-DC and inter-DC traffic flows◦ note that user traffic always goes through the bottleneck
2. If one uses heroku addons, is the related traffic in-DC? -- yet tomeasure
3. When using addons, what is the reduction factor so that final replywould fit through the bottleneck?◦ experiment showed factor of 4◦ the 30s reply deadline in heroku (feature/bug 5) is related to this
4. Note the huge latency = smallest throughput for small bulk --
interactivity apps on heroku?
5. ...
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 36/37...
36/37
.
maratishe.github.io
That’s all, thank you ...
M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 37/37...
37/37