Introducing wsgi_lineprof / PyCon JP 2017 LT

17
Introducing wsgi_lineprof Yusuke Miyazaki 2017-09-08 PyCon JP 2017

Transcript of Introducing wsgi_lineprof / PyCon JP 2017 LT

Page 1: Introducing wsgi_lineprof / PyCon JP 2017 LT

Introducing wsgi_lineprof

Yusuke Miyazaki 2017-09-08

PyCon JP 2017

Page 2: Introducing wsgi_lineprof / PyCon JP 2017 LT

Yusuke Miyazaki @ymyzk• Master’s course student of Kyoto University

• Programming languages / Type system

• Member of CAMPHOR- in Kyoto

• Community for student engineers and designers

• Intern of a start-up in Tokyo (Working Remotely)

• Uses Node.js (TypeScript & JavaScript) mainly

• We’re hiring!

Visit ymyzk.com for more details

Page 3: Introducing wsgi_lineprof / PyCon JP 2017 LT

Do you like building web applications?

🙋🙋🙋

Page 4: Introducing wsgi_lineprof / PyCon JP 2017 LT

Do you like building"fast"

web applications?"""

Page 5: Introducing wsgi_lineprof / PyCon JP 2017 LT

Do you know ISUCON?🙋"🙋"

Page 6: Introducing wsgi_lineprof / PyCon JP 2017 LT

ISUCON = Iikanjini Speed Up Contest

• Team: 2 or 3 people

• Objective: Make the given web application faster

• Modify configuration of OS or middleware

• Add or replace middleware

• Rewrite applications…

• Rule: If a benchmark score (≈ number of requests) is high, then you win!!

Page 7: Introducing wsgi_lineprof / PyCon JP 2017 LT

ISUCON = Iikanjini Speed Up Contest

• I participated in ISUCON in 2015 and 2016…

• But I lost in qualification rounds

• It is not easy to speed up existing applications

Page 8: Introducing wsgi_lineprof / PyCon JP 2017 LT

How can we speed up applications?

🤔🤔🤔

Page 9: Introducing wsgi_lineprof / PyCon JP 2017 LT

— Robert C. Pike "Notes on Programming in C"

“Measure.  Don't tune for speed until you've measured, and even then don't unless one

part of the code overwhelms the rest.”

Page 10: Introducing wsgi_lineprof / PyCon JP 2017 LT

How can we find bottlenecks? 🤔🤔🤔

Page 11: Introducing wsgi_lineprof / PyCon JP 2017 LT

Use Profilers• Python

• profile / cProfile

• timeit

• line_profiler

• memory_profiler

• WSGI

• Werkzurg

• linesman

They works well. But I want more simple one for webapp

(especially for ISUCON)

Page 12: Introducing wsgi_lineprof / PyCon JP 2017 LT

I create a new profiler for web applications

💪💪💪

Page 13: Introducing wsgi_lineprof / PyCon JP 2017 LT

🎉 wsgi_lineprof 🎉

Page 14: Introducing wsgi_lineprof / PyCon JP 2017 LT

wsgi_lineprof• WSGI middleware

• Works well with WAFs: Django, Flask, Bottle…

• Easy to integrate with existing applications without changing existing code of app.

• Provides line-by-line profiling

• Records every function call during a request

• Records the followings for every lines:

• Number of times the line was executed

• Amount of time spent to execute the line

Page 15: Introducing wsgi_lineprof / PyCon JP 2017 LT

How to use# Assume that WSGI app is defined as "app"

server = make_server('127.0.0.1', 8000, app) server.serve_forever()

# wsgi_lineprof filters = [ FilenameFilter("main.py"), TotalTimeSorter(), ] app = LineProfilerMiddleware(app, filters=filters)

Page 16: Introducing wsgi_lineprof / PyCon JP 2017 LT

Demo

Page 17: Introducing wsgi_lineprof / PyCon JP 2017 LT

Conclusion• wsgi_lineprof is a simple WSGI middleware for line-

by-line profiling

github.com/ymyzk/wsgi_lineprof

• Proposals, feedback, and PRs are welcome!

• Let’s participate in ISUCON using Python and win a victory! 🏆🏆🏆

Thank you for Listening!!