Post on 08-Sep-2014
description
How to build a High Performance PSGI/Plack Server
PSGI/Plack・Monocerosで学ぶハイパフォーマンス
Webアプリケーションサーバの作り方
YAPC::Asia 2013 TokyoMasahiro Nagano / @kazeburo
Me• 長野雅広 Masahiro Nagano
• @kazeburo
• PAUSE:KAZEBURO
• Operations Engineer, Site Reliability
• LINE Corp. Development support LINE Family, livedoor
livedoorBlogOne of the largest Blog Hosting Service in Japan
livedoorBlog uses
Perl (5.16 and 5.8)
Carton
Plack/PSGI and mod_perl
$ curl -I http://blog.livedoor.jp/staff/| grep Server
Server: Plack::Handler::Starlet
Starlet handles1 Billion(10億) reqs/day
To get over this burst traffic,
We need to improvePerformance
across all layersこの負荷を乗り切るために様々なレイヤーで最適化をしています
Layers
Hardware / Network
OS
App Server
Routing
Cache Logic
SQL
Template Engine
RDBMS
Cached Web Server
Hardware / Network
OS
Routing
Cache Logic
SQL
Template Engine
RDBMS
Cached Web Server
Today’s Topic
Hardware / Network
OS
Routing
Cache Logic
SQL
Template Engine
RDBMS
Cached Web Server
Today’s Topic
App Server
By the way..
“Open & Share”is our driver
for excellence.And LOVE CPAN/OSS
Open & Share は私たちの目指すところです。CPAN/OSSを多く使い、また貢献もしています
Improving Performance of livedoorBlog
directly linkedPerformance of Plack/Starlet
on CPAN
livedoorBlogのパフォーマンス改善で行った事はCPAN上のPlack/Starletにも当然影響してきます
0
3500
7000
10500
14000 13083
6241
“Hello World” Reqs/Sec
Plack 1.0016 1.0029Starlet 0.16 0.20
2013/02 2013/09
mod_perl era plack era
Monoceros
Monoceros isa yet another
Plack/PSGI Serverfor Performance
the Goal
Reduce TCP 3way hand shake
betweenProxy and PSGI Server
ReverseProxy
AppServer
GET / HTTP/1.1Host: example.com
SYNACK
SYN+ACK
HTTP/1.1 200 OKContent-Type: text/html
FINACK
GET /favicon.ico HTTP/1.1Host: example.com
SYNACK
SYN+ACK
HTTP/1.1 404 NOT FOUNDContent-Type: text/html
FINACK
HTTP/1.0-1.1 have KeepAlive
ReverseProxy
AppServer
GET / HTTP/1.0Host: example.comConnection: keep-alive
SYNACK
SYN+ACK
HTTP/1.0 200 OKContent-Type: text/htmlConnection: keep-alive
Content-Length: 941
GET /favicon.ico HTTP/1.1Host: example.com
HTTP/1.1 200 OKContent-Type: image/vnd.microsoft.icon
Transfer-Encoding: chunked
GET /site.css HTTP/1.1Host: example.com
HTTP/1.1 200 OKContent-Type: text/css
Content-Length: 1013
C10K problem
nginx
C10KReadyReverseProxy
nginx
nginx
StarletStarman
AppServer
KeepAlive Req
KeepAlive Req
KeepAlive Req
Starman, Starlet’sPreforking model requires1 connection per 1 process
By defaultStarman: 5 procs Starlet: 10 procs
Monoceros adoptsPreforking model,
But C10K ready
WorkerProcess
WorkerProcess
WorkerProcess
WorkerProcess
ManagerProcess
SOCK
Client
GET / HTTP/1.1Host: example.com
200 OKContent-Type: text/html
WorkerProcess
WorkerProcess
WorkerProcess
WorkerProcess
ManagerProcess
SOCK
Client
GET / HTTP/1.1Host: example.com
200 OKContent-Type: text/html
WorkerProcess
WorkerProcess
WorkerProcess
WorkerProcess
ManagerProcess
SOCK
Client
GET / HTTP/1.1Host: example.com
200 OKContent-Type: text/html
GET / HTTP/1.1Host: example.com
Event Driven
WorkerProcess
WorkerProcess
WorkerProcess
WorkerProcess
ManagerProcess
SOCK
Client
GET / HTTP/1.1Host: example.com
200 OKContent-Type: text/html
GET / HTTP/1.1Host: example.com
200 OKContent-Type: text/html
Event Driven
Monoceros Workersthat inherits “Starlet”
not C10K ready
Event DrivenManager Process
C10K readybuilt with AnyEvent
KeepAlive Benchmarklike a Browser
1) connect2) do requests certain number
3) leave alone a socket4) timeout and close
250 conn / 200 reqs
Starlet Monoceros
Total time (sec) 54.51 8.74
Failed reqs 971 0
Plack/PSGI Basics
PSGI = speci"cationPlack = implementation
PSGI Interface
my $app = sub { my $env = shift; ... return [200, [‘Content-Type’ => ‘text/html’], [‘Hello World’] ];};
PSGI environment hash
* CGI keys REQUEST_METHOD,SCRIPT_NAME, PATH_INFO,REQUEST_URI, QUERY_STRING,SERVER_PROTOCOL,HTTP_*
* PSGI-specific keys psgi.version, psgi.url_scheme, psgi.input, psgi.errors, psgi.multiprocess, psgi.streaming psgi.nonblocking
$env->{...}
PSGI Response (1) ArrayRef
[200, #status code [ ‘Content-Type’ => ‘text/html’, ‘Content-Length => 10 ], [ ‘Hello’, ‘World’ ]];
PSGI Response (1’) arrayref+IO::Handle
open my $fh, ‘<’, ‘/path/icon.jpg’;
[200, [ ‘Content-Type’ => ‘image/jpeg’, ‘Content-Length => 123456789 ], $fh];
PSGI Response (2)Delayed and Streamingsub { my $env = shift; return sub { my $responder = shift; ... $responder->([ 200, $headers, [$body] ]); }};
PSGI Response (2’)Delayed and Streamingreturn sub { my $responder = shift; my $writer = $responder->([200, $headers]); wait_for_events(sub { my $new_event = shift; if ($new_event) { $writer->write($new_event->as_json . "\n"); } else { $writer->close; } });};
Role of “PSGI Server”
PSGI Server“A PSGI Server is a Perl program
providing an environment for a PSGI application to run in”
PSGIServer
App$env
$res
Apache
Nginx
Apache
ProxyBrowser
CGI
mod_perl
FCGI
HTTP
Perl direct
PSGI Serveris called
“Plack Handler”
Plack HandlerAdaptor interface Plack and PSGI Server.
Make PSGI Server to runwith “plackup”
e.g. Starman
Starman::Server= PSGI Server
Plack::Handler::Starman= Plack Handler
Make
PSGI/PlackServer
a High Performance
Tiny StandalonePSGI Web Server
my $null_io = do { open my $io, "<", \""; $io };
my $app = sub { my $env = shift return [200,['Content-Type'=>'text/html'],['Hello','World',"\n"]];};
my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1,);
while ( my $conn = $listen->accept ) { my $env = { SERVER_PORT => '5000', SERVER_NAME => 'localhost', SCRIPT_NAME => '', REMOTE_ADDR => $conn->peerhost, 'psgi.version' => [ 1, 1 ], 'psgi.errors' => *STDERR, 'psgi.url_scheme' => 'http', 'psgi.run_once' => Plack::Util::FALSE, 'psgi.multithread' => Plack::Util::FALSE, 'psgi.multiprocess' => Plack::Util::FALSE, 'psgi.streaming' => Plack::Util::FALSE, 'psgi.nonblocking' => Plack::Util::FALSE, 'psgi.input' => $null_io, }; $conn->sysread( my $buf, 4096); my $reqlen = Plack::HTTPParser::parse_http_request($buf, $env);
my $res = Plack::Util::run_app $app, $env;
my @lines = ("HTTP/1.1 $res->[0] @{[ status_message($res->[0]) ]}\015\012"); for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]\015\012"; } push @lines, "Connection: close\015\12\015\12";
$conn->syswrite(join "",@lines); Plack::Util::foreach($res->[2], sub { $conn->syswrite(shift); }); $conn->close;}
60 lines
Listen and Accept
my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1,);
while ( my $conn = $listen->accept ) { ...}
Read a request
use Plack::HTTPParser qw/parse_http_request/;my $null_io = do { open my $io, "<", \""; $io };
while ( my $conn = $listen->accept ) {
my $env = { SERVER_PORT => '5000', SERVER_NAME => 'localhost', SCRIPT_NAME => '', REMOTE_ADDR => $conn->peerhost, 'psgi.version' => [ 1, 1 ], 'psgi.errors' => *STDERR, 'psgi.url_scheme' => 'http', 'psgi.multiprocess' => Plack::Util::FALSE, 'psgi.streaming' => Plack::Util::FALSE, 'psgi.nonblocking' => Plack::Util::FALSE, 'psgi.input' => $null_io, };
$conn->sysread(my $buf, 4096); my $reqlen = parse_http_request($buf, $env);
Run App
my $res = $app->($env);
or
my $res = Plack::Util::run_app $app, $env;
Write a response
use HTTP::Status qw/status_message/;
my $res = ..
my @lines = ("HTTP/1.1 $res->[0] \ @{[ status_message($res->[0]) ]}\015\012");
for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]\015\012";}push @lines, "Connection: close\015\12\015\12";
$conn->syswrite(join "",@lines);
foreach my $buf ( @{$res->[2]} ) { $conn->syswrite($buf);});
$conn->close;
This PSGI Serverhas some problem
* handle only one at once* no timeout
* may not fast
Increase concurrency
Multi ProcessIO Multiplexing
orBoth
Preforking modelSimple, Scaling
Manager
Manager
bind
listen
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Manager
bind
listen
fork fork fork fork
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Manager
bind
listen
fork fork fork fork
Client Client ClientClient
use Parallel::Prefork;
my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1,);
my $pm = Parallel::Prefork->new({ max_workers => 5, trap_signals => { TERM => 'TERM', HUP => 'TERM', }});
while ( $pm->signal_received ne 'TERM') { $pm->start(sub{ while ( my $conn = $listen->accept ) { my $env = {..}
NO Accept Serialization
os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Manager
bind
listen
Zzz.. Zzz.. Zzz.. Zzz..
os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Client
Manager
bind
listen
Zzz.. Zzz.. Zzz.. Zzz..
os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Client
Manager
bind
listen
Zzz.. Zzz.. Zzz.. Zzz..
os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Client
Manager
bind
listen
Thundering Herd突然の負荷
WakeUp WakeUp WakeUp WakeUP
os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Client
Manager
bind
listen
Thundering Herd突然の負荷
WakeUp WakeUp WakeUp WakeUP
Thundering Herdis an old story
Worker
accept
Worker
accept
Worker WorkerManager
bind
listen accept accept
Client
Zzz.. Zzz.. Zzz..Zzz..
Worker
accept
Worker
accept
Worker WorkerManager
bind
listen accept accept
Client
Zzz.. Zzz.. Zzz..Zzz..
Worker
accept
Worker
accept
Worker WorkerManager
bind
listen accept accept
Client
modern os/kernel
Zzz.. Zzz.. Zzz..Zzz..
Worker
accept
Worker
accept
Worker WorkerManager
bind
listen accept accept
Client
modern os/kernel
Zzz.. Zzz.. Zzz..Zzz..
Worker
accept
Worker
accept
Worker WorkerManager
bind
listen accept accept
Client
modern os/kernel
Zzz.. Zzz..Zzz..WakeUp
NO Accept Serialization(except for multiple interface)
TCP_DEFER_ACCEPT
Wake up a process when DATA arrived
not established
コネクションが完了したタイミングではなく、データが到着した段階でプロセスを起こします
clientA
clientB
GET / HTTP/1.0Host: example.comConnection: keep-alive
SYNACK
SYN+ACK
SYNACK
SYN+ACK
GET / HTTP/1.0Host: example.comConnection: keep-alive
default defer_accept
Accept
RunApp
blockto
read
RunApp
clientA
clientB
GET / HTTP/1.0Host: example.comConnection: keep-alive
SYNACK
SYN+ACK
SYNACK
SYN+ACK
GET / HTTP/1.0Host: example.comConnection: keep-alive
default defer_accept
Accept
RunApp
Accept
blockto
read
RunApp
clientA
clientB
GET / HTTP/1.0Host: example.comConnection: keep-alive
SYNACK
SYN+ACK
SYNACK
SYN+ACK
GET / HTTP/1.0Host: example.comConnection: keep-alive
default defer_accept
Accept
RunApp
Accept
blockto
read
idle
RunApp
clientA
clientB
GET / HTTP/1.0Host: example.comConnection: keep-alive
SYNACK
SYN+ACK
SYNACK
SYN+ACK
GET / HTTP/1.0Host: example.comConnection: keep-alive
default defer_accept
Accept
RunApp
Accept
RunApp
Accept
RunApp
Accept
blockto
read
idle
use Socket qw(IPPROTO_TCP);
my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1,);
if ($^O eq 'linux') { setsockopt($listen, IPPROTO_TCP, 9, 1);}
timeout to read header
alarm
my $READ_TIMEOUT = 5;
eval { local $SIG{ALRM} = sub { die "Timed out\n"; }; alarm( $READ_TIMEOUT ); $conn->sysread(my $buf, 4096);};alarm(0);
next if ( $@ && $@ =~ /Timed out/ );
my $reqlen = parse_http_request($buf, $env);
nonblocking + select
use IO::Select;my $READ_TIMEOUT = 5;
while( my $conn = $listen->accept ) {
$conn->blocking(0);
my $select = IO::Select->new($conn); my @ready = $select->can_read($READ_TIMEOUT); next unless @ready;
$conn->sysread($buf, 4096);
my $reqlen = parse_http_request($buf, $env);
alarmvs.
nonblocking + select
Fewer syscalls is good
Hardwares
User Application
OS/Kernel
system callslisten,fork, accept, read, write, select, alarm
Worker Worker Worker Worker Worker
alarm
rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0rt_sigaction(SIGALRM, {0x47e5b0, [], SA_RESTORER, 0x7ff7d6e0cba0}, {SIG_DFL, [], SA_RESTORER, 0x7ff7d6e0cba0}, 8) = 0rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0alarm(5) = 0read(7, "GET / HTTP/1.1\r\nUser-Agent: curl"..., 65536) = 155rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, 0x7ff7d6e0cba0}, {0x47e5b0, [], SA_RESTORER, 0x7ff7d6e0cba0}, 8) = 0rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0alarm(0) = 5 9 syscalls
non-blocking + select
fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0select(8, [5], NULL, [5], {300, 0}) = 1 (in [5], left {299, 999897})read(5, "GET / HTTP/1.1\r\nUser-Agent: curl"..., 131072) = 155
4 syscalls
Parse a request with “C”
$ cpanm HTTP::Parser::XS
Plack::HTTPParser uses H::P::XS if installed
TCP_NODELAY
When data was writtenTCP packets does not
immediately send
“TCP uses Nagle's algorithm to collect small packets
for send all at once by default”
write(“foo”)
write(“bar”)
os/kernel networkinterfaceApplication
buffering
“foobar”
write(“foo”)
write(“bar”)
os/kernel networkinterfaceApplication
“foo”
TCP_NODELAY
“bar”
Take care of excessive fragmentation of TCP
packets
Write in oncejoin content in Server
my @lines = ("HTTP/1.1 $res->[0] @{[ status_message($res->[0]) ]}\015\012");
for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]\015\012";}push @lines, "Connection: close\015\12\015\12";
$conn->syswrite(join "",@lines, @{$res->[2]});
accept4, writev
Choose PSGI/Plack
Server
CPAN has manyPSGI Server &
Plack::Hanlder:**
Standalone(HTTP::Server::PSGI)
Default server for plackupSingle process Web Server
For development
Starman
Preforking Web ServerHTTP/1.1, HTTPS,
Multiple interfaces, unix-domain socket,
hot deploy using Server::Starter
Starlet
Preforking Web ServerHTTP/1.1(0.20~)
hot deploy using Server::StarterSimple and Fast
Monoceros
C10K Ready Preforking Web ServerHTTP/1.1
hot deploy using Server::Starter
Twiggy
based on AnyEventnonblocking, streaming
Single Process
Twiggy::Prefork
based on Twiggy and Parallel::Prefork
nonblocking, streamingMulti Process
hot deploy using Server::Starter
Feersum
Web server based on EV/libevnonblocking, streaming
Single/Multi Process
How to choosePSGI Server
SingleProcess
MultiProcess
CPU Intensive -Starlet
StarmanMonoceros
RequiresEvent Driven
TwiggyFeersum
Twiggy::PreforkFeersumTy
pe o
f Web
App
licat
ion
Finding Bottlenecks of Performance
use Devel::NYTProf
Flame Graph is awesome
Pro"le nytprof.out.{PID}for preforking server
$ nytprofhtml -f nytprof.out.1210$ open nytprof/index.html
use strace or dtrusstrace syscalls
$ strace -tt -s 200 -p {pid} \ 2>&1 | tee /tmp/trace.txt
Process 30929 attached - interrupt to quit16:13:46.826828 accept(4, {sa_family=AF_INET, sin_port=htons(43783), sin_addr=inet_addr("127.0.0.1")}, [16]) = 516:13:48.916233 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid argument)16:13:48.916392 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)16:13:48.916493 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid argument)16:13:48.916573 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)16:13:48.916661 fcntl(5, F_SETFD, FD_CLOEXEC) = 016:13:48.916873 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)16:13:48.916959 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 016:13:48.917095 setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 016:13:48.917362 read(5, "GET / HTTP/1.0\r\nHost: 127.0.0.1:5005\r\nUser-Agent: ApacheBench/2.3\r\nAccept: */*\r\n\r\n", 131072) = 8216:13:48.917613 brk(0x1e8e000) = 0x1e8e00016:13:48.917746 gettimeofday({1379402028, 917802}, NULL) = 016:13:48.917953 write(5, "HTTP/1.1 200 OK\r\nDate: Tue, 17 Sep 2013 07:13:48 GMT\r\nServer: Plack::Handler::Starlet\r\nContent-Type: html\r\nConnection: close\r\n\r\nhello", 133) = 13316:13:48.918187 close(5) = 016:13:48.918428 accept(4, {sa_family=AF_INET, sin_port=htons(43793), sin_addr=inet_addr("127.0.0.1")}, [16]) = 516:13:48.923736 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid argument)16:13:48.923843 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)16:13:48.923924 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid argument)16:13:48.924461 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)16:13:48.924600 fcntl(5, F_SETFD, FD_CLOEXEC) = 016:13:48.924762 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)16:13:48.924853 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 016:13:48.924939 setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 016:13:48.925162 read(5, "GET / HTTP/1.0\r\nHost: 127.0.0.1:5005\r\nUser-Agent: ApacheBench/2.3\r\nAccept: */*\r\n\r\n", 131072) = 8216:13:48.925445 gettimeofday({1379402028, 925494}, NULL) = 016:13:48.925629 write(5, "HTTP/1.1 200 OK\r\nDate: Tue, 17 Sep 2013 07:13:48 GMT\r\nServer: Plack::Handler::Starlet\r\nContent-Type: html\r\nConnection: close\r\n\r\nhello", 133) = 13316:13:48.925854 close(5) = 016:13:48.926084 accept(4, {sa_family=AF_INET, sin_port=htons(43803), sin_addr=inet_addr("127.0.0.1")}, [16]) = 516:13:48.930480 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid argument)16:13:48.930626 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)16:13:48.930744 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid argument)16:13:48.930838 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)16:13:48.930915 fcntl(5, F_SETFD, FD_CLOEXEC) = 016:13:48.931070 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)16:13:48.931170 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 016:13:48.931383 setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 016:13:48.931536 read(5, "GET / HTTP/1.0\r\nHost: 127.0.0.1:5005\r\nUser-Agent: ApacheBench/2.3\r\nAccept: */*\r\n\r\n", 131072) = 8216:13:48.931748 gettimeofday({1379402028, 931791}, NULL) = 016:13:48.931869 write(5, "HTTP/1.1 200 OK\r\nDate: Tue, 17 Sep 2013 07:13:48 GMT\r\nServer: Plack::Handler::Starlet\r\nContent-Type: html\r\nConnection: close\r\n\r\nhello", 133) = 13316:13:48.932078 close(5) = 016:13:48.932256 accept(4, {sa_family=AF_INET, sin_port=htons(43813), sin_addr=inet_addr("127.0.0.1")}, [16]) = 5
in conclusion
PSGI Server get Faster.
PSGI/Plack RocksStable, Fast
Found problems?RT, GitHub Issue, PullReqs
IRC #perl @kazeburo
#"n. Thank you!