How to build a High Performance PSGI/Plack Server

How to build a High Performance PSGI/Plack Server

PSGI/Plack・Monocerosで学ぶハイパフォーマンス

Webアプリケーションサーバの作り方

YAPC::Asia 2013 TokyoMasahiro Nagano / @kazeburo

Me• 長野雅広 Masahiro Nagano

• @kazeburo

• PAUSE:KAZEBURO

• Operations Engineer, Site Reliability

• LINE Corp. Development support LINE Family, livedoor

livedoorBlogOne of the largest Blog Hosting Service in Japan

livedoorBlog uses

Perl (5.16 and 5.8)

Carton

Plack/PSGI and mod_perl

$ curl -I http://blog.livedoor.jp/staff/| grep Server

Server: Plack::Handler::Starlet

Starlet handles1 Billion(10億) reqs/day

To get over this burst traffic,

We need to improvePerformance

across all layersこの負荷を乗り切るために様々なレイヤーで最適化をしています

Layers

Hardware / Network

OS

App Server

Routing

Cache Logic

SQL

Template Engine

RDBMS

Cached Web Server

Hardware / Network

OS

Routing

Cache Logic

SQL

Template Engine

RDBMS

Cached Web Server

Today’s Topic

Hardware / Network

OS

Routing

Cache Logic

SQL

Template Engine

RDBMS

Cached Web Server

Today’s Topic

App Server

By the way..

“Open & Share”is our driver

for excellence.And LOVE CPAN/OSS

Open & Share は私たちの目指すところです。CPAN/OSSを多く使い、また貢献もしています

Improving Performance of livedoorBlog

directly linkedPerformance of Plack/Starlet

on CPAN

livedoorBlogのパフォーマンス改善で行った事はCPAN上のPlack/Starletにも当然影響してきます

0

3500

7000

10500

14000 13083

6241

“Hello World” Reqs/Sec

Plack 1.0016 1.0029Starlet 0.16 0.20

2013/02 2013/09

mod_perl era plack era

Monoceros

Monoceros isa yet another

Plack/PSGI Serverfor Performance

the Goal

Reduce TCP 3way hand shake

betweenProxy and PSGI Server

ReverseProxy

AppServer

GET / HTTP/1.1Host: example.com

SYNACK

SYN+ACK

HTTP/1.1 200 OKContent-Type: text/html

FINACK

GET /favicon.ico HTTP/1.1Host: example.com

SYNACK

SYN+ACK

HTTP/1.1 404 NOT FOUNDContent-Type: text/html

FINACK

HTTP/1.0-1.1 have KeepAlive

ReverseProxy

AppServer

GET / HTTP/1.0Host: example.comConnection: keep-alive

SYNACK

SYN+ACK

HTTP/1.0 200 OKContent-Type: text/htmlConnection: keep-alive

Content-Length: 941

GET /favicon.ico HTTP/1.1Host: example.com

HTTP/1.1 200 OKContent-Type: image/vnd.microsoft.icon

Transfer-Encoding: chunked

GET /site.css HTTP/1.1Host: example.com

HTTP/1.1 200 OKContent-Type: text/css

Content-Length: 1013

C10K problem

nginx

C10KReadyReverseProxy

nginx

nginx

StarletStarman

AppServer

KeepAlive Req

KeepAlive Req

KeepAlive Req

Starman, Starlet’sPreforking model requires1 connection per 1 process

By defaultStarman: 5 procs Starlet: 10 procs

Monoceros adoptsPreforking model,

But C10K ready

WorkerProcess

WorkerProcess

WorkerProcess

WorkerProcess

ManagerProcess

SOCK

Client


200 OKContent-Type: text/html

WorkerProcess

WorkerProcess

WorkerProcess

WorkerProcess

ManagerProcess

SOCK

Client




Event Driven

WorkerProcess

WorkerProcess

WorkerProcess

WorkerProcess

ManagerProcess

SOCK

Client





Event Driven

Monoceros Workersthat inherits “Starlet”

not C10K ready

Event DrivenManager Process

C10K readybuilt with AnyEvent

KeepAlive Benchmarklike a Browser

1) connect2) do requests certain number

3) leave alone a socket4) timeout and close

250 conn / 200 reqs

Starlet Monoceros

Total time (sec) 54.51 8.74

Failed reqs 971 0

Plack/PSGI Basics

PSGI = speci"cationPlack = implementation

PSGI Interface

my $app = sub { my $env = shift; ... return [200, [‘Content-Type’ => ‘text/html’], [‘Hello World’] ];};

PSGI environment hash

* CGI keys REQUEST_METHOD,SCRIPT_NAME, PATH_INFO,REQUEST_URI, QUERY_STRING,SERVER_PROTOCOL,HTTP_*

* PSGI-specific keys psgi.version, psgi.url_scheme, psgi.input, psgi.errors, psgi.multiprocess, psgi.streaming psgi.nonblocking

$env->{...}

PSGI Response (1) ArrayRef

[200, #status code [ ‘Content-Type’ => ‘text/html’, ‘Content-Length => 10 ], [ ‘Hello’, ‘World’ ]];

PSGI Response (1’) arrayref+IO::Handle

open my $fh, ‘<’, ‘/path/icon.jpg’;

[200, [ ‘Content-Type’ => ‘image/jpeg’, ‘Content-Length => 123456789 ], $fh];

PSGI Response (2)Delayed and Streamingsub { my $env = shift; return sub { my $responder = shift; ... $responder->([ 200, $headers, [$body] ]); }};

PSGI Response (2’)Delayed and Streamingreturn sub { my $responder = shift; my $writer = $responder->([200, $headers]); wait_for_events(sub { my $new_event = shift; if ($new_event) { $writer->write($new_event->as_json . "\n"); } else { $writer->close; } });};

Role of “PSGI Server”

PSGI Server“A PSGI Server is a Perl program

providing an environment for a PSGI application to run in”

PSGIServer

App$env

$res

Apache

Nginx

Apache

ProxyBrowser

CGI

mod_perl

FCGI

HTTP

Perl direct

PSGI Serveris called

“Plack Handler”

Plack HandlerAdaptor interface Plack and PSGI Server.

Make PSGI Server to runwith “plackup”

e.g. Starman

Starman::Server= PSGI Server

Plack::Handler::Starman= Plack Handler

Make

PSGI/PlackServer

a High Performance

Tiny StandalonePSGI Web Server

my $null_io = do { open my $io, "<", \""; $io };

my $app = sub { my $env = shift return [200,['Content-Type'=>'text/html'],['Hello','World',"\n"]];};

my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1,);

while ( my $conn = $listen->accept ) { my $env = { SERVER_PORT => '5000', SERVER_NAME => 'localhost', SCRIPT_NAME => '', REMOTE_ADDR => $conn->peerhost, 'psgi.version' => [ 1, 1 ], 'psgi.errors' => *STDERR, 'psgi.url_scheme' => 'http', 'psgi.run_once' => Plack::Util::FALSE, 'psgi.multithread' => Plack::Util::FALSE, 'psgi.multiprocess' => Plack::Util::FALSE, 'psgi.streaming' => Plack::Util::FALSE, 'psgi.nonblocking' => Plack::Util::FALSE, 'psgi.input' => $null_io, }; $conn->sysread( my $buf, 4096); my $reqlen = Plack::HTTPParser::parse_http_request($buf, $env);

my $res = Plack::Util::run_app $app, $env;

my @lines = ("HTTP/1.1 $res->[0] @{[ status_message($res->[0]) ]}\015\012"); for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]\015\012"; } push @lines, "Connection: close\015\12\015\12";

$conn->syswrite(join "",@lines); Plack::Util::foreach($res->[2], sub { $conn->syswrite(shift); }); $conn->close;}

60 lines

Listen and Accept


while ( my $conn = $listen->accept ) { ...}

Read a request

use Plack::HTTPParser qw/parse_http_request/;my $null_io = do { open my $io, "<", \""; $io };

while ( my $conn = $listen->accept ) {

my $env = { SERVER_PORT => '5000', SERVER_NAME => 'localhost', SCRIPT_NAME => '', REMOTE_ADDR => $conn->peerhost, 'psgi.version' => [ 1, 1 ], 'psgi.errors' => *STDERR, 'psgi.url_scheme' => 'http', 'psgi.multiprocess' => Plack::Util::FALSE, 'psgi.streaming' => Plack::Util::FALSE, 'psgi.nonblocking' => Plack::Util::FALSE, 'psgi.input' => $null_io, };

$conn->sysread(my $buf, 4096); my $reqlen = parse_http_request($buf, $env);

Run App

my $res = $app->($env);

or

my $res = Plack::Util::run_app $app, $env;

Write a response

use HTTP::Status qw/status_message/;

my $res = ..

my @lines = ("HTTP/1.1 $res->[0] \ @{[ status_message($res->[0]) ]}\015\012");

for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]\015\012";}push @lines, "Connection: close\015\12\015\12";

$conn->syswrite(join "",@lines);

foreach my $buf ( @{$res->[2]} ) { $conn->syswrite($buf);});

$conn->close;

This PSGI Serverhas some problem

* handle only one at once* no timeout

* may not fast

Increase concurrency

Multi ProcessIO Multiplexing

orBoth

Preforking modelSimple, Scaling

Manager

Manager

bind

listen

Worker

accept

Worker

accept

Worker

accept

Worker

accept

Manager

bind

listen

fork fork fork fork

Worker

accept

Worker

accept

Worker

accept

Worker

accept

Manager

bind

listen

fork fork fork fork

Client Client ClientClient

use Parallel::Prefork;


my $pm = Parallel::Prefork->new({ max_workers => 5, trap_signals => { TERM => 'TERM', HUP => 'TERM', }});

while ( $pm->signal_received ne 'TERM') { $pm->start(sub{ while ( my $conn = $listen->accept ) { my $env = {..}

NO Accept Serialization

os/kernel

Worker

accept

Worker

accept

Worker

accept

Worker

accept

Manager

bind

listen

Zzz.. Zzz.. Zzz.. Zzz..

os/kernel

Worker

accept

Worker

accept

Worker

accept

Worker

accept

Client

Manager

bind

listen

Zzz.. Zzz.. Zzz.. Zzz..

os/kernel

Worker

accept

Worker

accept

Worker

accept

Worker

accept

Client

Manager

bind

listen

Thundering Herd突然の負荷

WakeUp WakeUp WakeUp WakeUP

Thundering Herdis an old story

Worker

accept

Worker

accept

Worker WorkerManager

bind

listen accept accept

Client

Zzz.. Zzz.. Zzz..Zzz..

Worker

accept

Worker

accept


bind


Client

modern os/kernel

Zzz.. Zzz.. Zzz..Zzz..

Worker

accept

Worker

accept


bind


Client

modern os/kernel

Zzz.. Zzz..Zzz..WakeUp

NO Accept Serialization(except for multiple interface)

TCP_DEFER_ACCEPT

Wake up a process when DATA arrived

not established

コネクションが完了したタイミングではなく、データが到着した段階でプロセスを起こします

clientA

clientB


SYNACK

SYN+ACK

SYNACK

SYN+ACK


default defer_accept

Accept

RunApp

blockto

read

RunApp

clientA

clientB


SYNACK

SYN+ACK

SYNACK

SYN+ACK



Accept

RunApp

Accept

blockto

read

RunApp

clientA

clientB


SYNACK

SYN+ACK

SYNACK

SYN+ACK



Accept

RunApp

Accept

blockto

read

idle

RunApp

clientA

clientB


SYNACK

SYN+ACK

SYNACK

SYN+ACK



Accept

RunApp

Accept

RunApp

Accept

RunApp

Accept

blockto

read

idle

use Socket qw(IPPROTO_TCP);


if ($^O eq 'linux') { setsockopt($listen, IPPROTO_TCP, 9, 1);}

timeout to read header

my $READ_TIMEOUT = 5;

eval { local $SIG{ALRM} = sub { die "Timed out\n"; }; alarm( $READ_TIMEOUT ); $conn->sysread(my $buf, 4096);};alarm(0);

next if ( $@ && $@ =~ /Timed out/ );

my $reqlen = parse_http_request($buf, $env);

nonblocking + select

use IO::Select;my $READ_TIMEOUT = 5;

while( my $conn = $listen->accept ) {

$conn->blocking(0);

my $select = IO::Select->new($conn); my @ready = $select->can_read($READ_TIMEOUT); next unless @ready;

$conn->sysread($buf, 4096);

my $reqlen = parse_http_request($buf, $env);

alarmvs.

nonblocking + select

Fewer syscalls is good

Hardwares

User Application

OS/Kernel

system callslisten,fork, accept, read, write, select, alarm

Worker Worker Worker Worker Worker

rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0rt_sigaction(SIGALRM, {0x47e5b0, [], SA_RESTORER, 0x7ff7d6e0cba0}, {SIG_DFL, [], SA_RESTORER, 0x7ff7d6e0cba0}, 8) = 0rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0alarm(5) = 0read(7, "GET / HTTP/1.1\r\nUser-Agent: curl"..., 65536) = 155rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, 0x7ff7d6e0cba0}, {0x47e5b0, [], SA_RESTORER, 0x7ff7d6e0cba0}, 8) = 0rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0alarm(0) = 5 9 syscalls

non-blocking + select

fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0select(8, [5], NULL, [5], {300, 0}) = 1 (in [5], left {299, 999897})read(5, "GET / HTTP/1.1\r\nUser-Agent: curl"..., 131072) = 155

4 syscalls

Parse a request with “C”

$ cpanm HTTP::Parser::XS

Plack::HTTPParser uses H::P::XS if installed

TCP_NODELAY

When data was writtenTCP packets does not

immediately send

“TCP uses Nagle's algorithm to collect small packets

for send all at once by default”

write(“foo”)

write(“bar”)

os/kernel networkinterfaceApplication

buffering

“foobar”

write(“foo”)

write(“bar”)

os/kernel networkinterfaceApplication

“foo”

TCP_NODELAY

“bar”

Take care of excessive fragmentation of TCP

packets

Write in oncejoin content in Server

my @lines = ("HTTP/1.1 $res->[0] @{[ status_message($res->[0]) ]}\015\012");

for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]\015\012";}push @lines, "Connection: close\015\12\015\12";

$conn->syswrite(join "",@lines, @{$res->[2]});

accept4, writev

Choose PSGI/Plack

Server

CPAN has manyPSGI Server &

Plack::Hanlder:**

Standalone(HTTP::Server::PSGI)

Default server for plackupSingle process Web Server

For development

Starman

Preforking Web ServerHTTP/1.1, HTTPS,

Multiple interfaces, unix-domain socket,

hot deploy using Server::Starter

Starlet

Preforking Web ServerHTTP/1.1(0.20~)

hot deploy using Server::StarterSimple and Fast

Monoceros

C10K Ready Preforking Web ServerHTTP/1.1


Twiggy

based on AnyEventnonblocking, streaming

Single Process

Twiggy::Prefork

based on Twiggy and Parallel::Prefork

nonblocking, streamingMulti Process


Feersum

Web server based on EV/libevnonblocking, streaming

Single/Multi Process

How to choosePSGI Server

SingleProcess

MultiProcess

CPU Intensive -Starlet

StarmanMonoceros

RequiresEvent Driven

TwiggyFeersum

Twiggy::PreforkFeersumTy

pe o

f Web

App

licat

ion

Finding Bottlenecks of Performance

use Devel::NYTProf

Flame Graph is awesome

Pro"le nytprof.out.{PID}for preforking server

$ nytprofhtml -f nytprof.out.1210$ open nytprof/index.html

use strace or dtrusstrace syscalls

$ strace -tt -s 200 -p {pid} \ 2>&1 | tee /tmp/trace.txt

Process 30929 attached - interrupt to quit16:13:46.826828 accept(4, {sa_family=AF_INET, sin_port=htons(43783), sin_addr=inet_addr("127.0.0.1")}, [16]) = 516:13:48.916233 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid argument)16:13:48.916392 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)16:13:48.916493 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid argument)16:13:48.916573 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)16:13:48.916661 fcntl(5, F_SETFD, FD_CLOEXEC) = 016:13:48.916873 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)16:13:48.916959 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 016:13:48.917095 setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 016:13:48.917362 read(5, "GET / HTTP/1.0\r\nHost: 127.0.0.1:5005\r\nUser-Agent: ApacheBench/2.3\r\nAccept: */*\r\n\r\n", 131072) = 8216:13:48.917613 brk(0x1e8e000) = 0x1e8e00016:13:48.917746 gettimeofday({1379402028, 917802}, NULL) = 016:13:48.917953 write(5, "HTTP/1.1 200 OK\r\nDate: Tue, 17 Sep 2013 07:13:48 GMT\r\nServer: Plack::Handler::Starlet\r\nContent-Type: html\r\nConnection: close\r\n\r\nhello", 133) = 13316:13:48.918187 close(5) = 016:13:48.918428 accept(4, {sa_family=AF_INET, sin_port=htons(43793), sin_addr=inet_addr("127.0.0.1")}, [16]) = 516:13:48.923736 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid argument)16:13:48.923843 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)16:13:48.923924 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid argument)16:13:48.924461 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)16:13:48.924600 fcntl(5, F_SETFD, FD_CLOEXEC) = 016:13:48.924762 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)16:13:48.924853 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 016:13:48.924939 setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 016:13:48.925162 read(5, "GET / HTTP/1.0\r\nHost: 127.0.0.1:5005\r\nUser-Agent: ApacheBench/2.3\r\nAccept: */*\r\n\r\n", 131072) = 8216:13:48.925445 gettimeofday({1379402028, 925494}, NULL) = 016:13:48.925629 write(5, "HTTP/1.1 200 OK\r\nDate: Tue, 17 Sep 2013 07:13:48 GMT\r\nServer: Plack::Handler::Starlet\r\nContent-Type: html\r\nConnection: close\r\n\r\nhello", 133) = 13316:13:48.925854 close(5) = 016:13:48.926084 accept(4, {sa_family=AF_INET, sin_port=htons(43803), sin_addr=inet_addr("127.0.0.1")}, [16]) = 516:13:48.930480 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid argument)16:13:48.930626 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)16:13:48.930744 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid argument)16:13:48.930838 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)16:13:48.930915 fcntl(5, F_SETFD, FD_CLOEXEC) = 016:13:48.931070 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)16:13:48.931170 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 016:13:48.931383 setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 016:13:48.931536 read(5, "GET / HTTP/1.0\r\nHost: 127.0.0.1:5005\r\nUser-Agent: ApacheBench/2.3\r\nAccept: */*\r\n\r\n", 131072) = 8216:13:48.931748 gettimeofday({1379402028, 931791}, NULL) = 016:13:48.931869 write(5, "HTTP/1.1 200 OK\r\nDate: Tue, 17 Sep 2013 07:13:48 GMT\r\nServer: Plack::Handler::Starlet\r\nContent-Type: html\r\nConnection: close\r\n\r\nhello", 133) = 13316:13:48.932078 close(5) = 016:13:48.932256 accept(4, {sa_family=AF_INET, sin_port=htons(43813), sin_addr=inet_addr("127.0.0.1")}, [16]) = 5

in conclusion

PSGI Server get Faster.

PSGI/Plack RocksStable, Fast

Found problems?RT, GitHub Issue, PullReqs

IRC #perl @kazeburo

#"n. Thank you!

How to build a High Performance PSGI/Plack Server

Technology

Transcript of How to build a High Performance PSGI/Plack Server