IAsyncResult Pattern ou Asynchronous Programming Model (APM)
How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins
-
Upload
gosuke-miyashita -
Category
Technology
-
view
2.642 -
download
2
Transcript of How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins
![Page 1: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/1.jpg)
YAPC::Asia 2009 Tokyo
2009/9/10
Gosuke Miyashita
![Page 2: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/2.jpg)
Speaker
Gosuke Miyashita ( mizzy ) Technical manager at paperboy&co. Have most kids in Japan Perl world?
3 kids.Even with Yappo.4th kid will be born in November.
![Page 3: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/3.jpg)
![Page 4: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/4.jpg)
![Page 5: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/5.jpg)
Agenda Asynchronous processing with event
driven programming Asynchronous processing with
Danga::Socket Overview of Perlbal asynchronous
processing How to write asynchronous Perlbal
plugins Summary
![Page 6: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/6.jpg)
![Page 7: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/7.jpg)
Event driven programming A programming paradigm in which the
flow of the program is determined by events
Counter paradigm of flow-based programming
![Page 8: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/8.jpg)
Events
I/OOn I/O read ready or write ready
TimersOn time passed
SignalOn getting a signal
Child processOn child process exit
![Page 9: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/9.jpg)
Main loop
Also called “event loop” On each loop, check that events are
occurred or not, and process the events. After process the events, return to the
loop and repeat the loop till next events occur
This is the flow of asynchronous processing
![Page 10: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/10.jpg)
Eaxmple of main loop
Process timers
Wait I/O events
Process I/O events
Post main loop process
![Page 11: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/11.jpg)
![Page 12: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/12.jpg)
Events supported by Danga::Socket
I/O Timers
![Page 13: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/13.jpg)
Process I/O events
Danga::Socket supports 3 I/O event notification facilitykqueueepollpoll
Select adequate facility automatically
![Page 14: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/14.jpg)
Add I/O watcher with Danga::Socketuse base ‘Danga::Socket’;
sub new { # pass file descripter $self->SUPER::new($fd); }
sub event_read { # process when $fd is read ready}
sub event_write { # procdess when $fd is write ready}
![Page 15: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/15.jpg)
Add I/O wathcer (pattern 2)
Danga::Socket->AddOtherFds( $fd => sub { # process when $fd is read ready
},);
![Page 16: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/16.jpg)
Add I/O watcher (pattern 3)
Danga::Socket::Callback->new( handle => $fd, on_read_ready => sub { # process on read ready }, on_write_ready => sub { # process on write ready },);
![Page 17: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/17.jpg)
Add timer watcher with Danga::Socket
Danga::Socket->AddTimer( 10, sub { # process after 10 seconds
},}
![Page 18: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/18.jpg)
Start main loop with Danga::Socket
Danga::Socket->EventLoop();
![Page 19: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/19.jpg)
Main loop(again)
Process Timers
Wait I/O events
Process I/O events
Post main loop process
![Page 20: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/20.jpg)
![Page 21: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/21.jpg)
Perlbal mechanism(as reverse proxy)
BackendHTTP
ClientProxy
TCPListenerClient
Server
ClientProxy
Client
BackendHTTP
Danga::Socket based objects
![Page 22: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/22.jpg)
Perlbal with a plugin (on start_proxy_request hook)
BackendHTTP
ClientProxy
TCPListenerClient
Server
Plugin::Hoge
Target of this session
![Page 23: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/23.jpg)
![Page 24: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/24.jpg)
Hooks
Perlbal has many plugin hooks Explain with start_proxy_request hook in
this session
![Page 25: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/25.jpg)
In case of a synchronous plugin
ClientProxy
TCPListenerClient
Server
Plugin::SyncClient
![Page 26: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/26.jpg)
In case of an asynchronous plugin
ClientProxy
TCPListenerClient
Server
Plugin::AsyncClient
ClientProxy
![Page 27: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/27.jpg)
How to write asynchronous plugins? Write the main process based on
Danga::Socketprocess within the main loop of
Danga::SocketCreate Danga::Socket based classOr use Danga::Socket::CallbackOr use Danga::Socket::AddTimerAlso other libraries need to be non-blocking
○ You can use Gearman::Client::Async with blocking processes
![Page 28: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/28.jpg)
How to write asynchronous plugins? When finish asynchronous process, go
to next phaseCall back functionAlso need to fix Perlbal itself
![Page 29: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/29.jpg)
Main process of a pluginpackage My::Drizzle;
use base ‘Danga::Socket’;use Net::Drizzle ':constants';
sub new { $self->SUPER::new($fh); $self->watch_read(1);}
sub event_read { # Check db request status and call back
when finished}
![Page 30: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/30.jpg)
Attention You can also use
Danga::Socket::Callback You cannot use AddOtherFds()
File descriptors added by Add AddOtherFds() are only evaluated at the beginning of the main loop
So if already in the main loop, AddOtherFds() are meaningless.
You can call epoll_ctl, EV_SET and so on directly, but no portability
![Page 31: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/31.jpg)
plugin registerpackage Perlbal::Plugin::AsyncDb;
sub register { my ( $class, $svc ) = @_;
$svc->register_hook( 'Async' => 'start_proxy_request', \&request_db, );
return 1;}
![Page 32: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/32.jpg)
Call asynchronous processsub request_db { my Perlbal::ClientProxy $client =
shift;
My::Drizzle->new( callback => sub { # call back }, );
return 1; # very important!}
![Page 33: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/33.jpg)
In case of “return 0”
BackendHTTP
ClientProxy
TCPListenerClient
Server
Plugin::Async
Go to next step even if Plugin::Async is not
finished
![Page 34: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/34.jpg)
BackendHTTP
“return 1” & “call back”
ClientProxy
TCPListenerClient
Plugin::Async
Stop the process and go to main loop call back
Server
return 1
![Page 35: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/35.jpg)
Call back function If plugin process finished, ClientProxy must
go to the next process(call BackendHTTP) Stopped at the following code of
handle_request() :
return if $svc->run_hook( 'start_proxy_request', $self ); ClientProxy must start processing from the
next of this code
![Page 36: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/36.jpg)
Call back function
my Perlbal::ClientProxy $client = shift;
My::Drizzle->new( callback => sub { $client->{async_complete} = 1;
$client->handle_request; },);
![Page 37: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/37.jpg)
Patch of ClientProxy::handle_request- return if $svc->run_hook(- 'start_proxy_request', $self- );+ unless ( $self->{async_complete} ) {
+ return if $svc->run_hook(+ 'start_proxy_request', $self
+ );+ }+ $self->{async_complete} = 0;
![Page 38: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/38.jpg)
![Page 39: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/39.jpg)
Points of asynchronous Perlbal plugins Process within the Danga::Socket main
loop return 1 when plugin called Restart ClientProxy process by a call
back function when plugin finished Also need to fix Perlbal itself Be careful with order of plugins
Following plugins on a same hook are ignored when after a plugin returns 1
![Page 40: How Danga::Socket handles asynchronous processing and how to write asynchronous Perlbal plugins](https://reader036.fdocument.pub/reader036/viewer/2022062704/55627a67d8b42ae73c8b4d88/html5/thumbnails/40.jpg)