Introducing wsgi_lineprof

Yusuke Miyazaki 2017-09-08

PyCon JP 2017

Yusuke Miyazaki @ymyzk• Master’s course student of Kyoto University

• Programming languages / Type system

• Member of CAMPHOR- in Kyoto

• Community for student engineers and designers

• Intern of a start-up in Tokyo (Working Remotely)

• Uses Node.js (TypeScript & JavaScript) mainly

• We’re hiring!

Visit ymyzk.com for more details

Do you like building web applications?

🙋🙋🙋

Do you like building"fast"

web applications?"""

Do you know ISUCON?🙋"🙋"

ISUCON = Iikanjini Speed Up Contest

• Team: 2 or 3 people

• Objective: Make the given web application faster

• Modify configuration of OS or middleware

• Add or replace middleware

• Rewrite applications…

• Rule: If a benchmark score (≈ number of requests) is high, then you win!!

ISUCON = Iikanjini Speed Up Contest

• I participated in ISUCON in 2015 and 2016…

• But I lost in qualification rounds

• It is not easy to speed up existing applications

How can we speed up applications?

🤔🤔🤔

— Robert C. Pike "Notes on Programming in C"

“Measure.  Don't tune for speed until you've measured, and even then don't unless one

part of the code overwhelms the rest.”

How can we find bottlenecks? 🤔🤔🤔

Use Profilers• Python

• profile / cProfile

• timeit

• line_profiler

• memory_profiler

• WSGI

• Werkzurg

• linesman

They works well. But I want more simple one for webapp

(especially for ISUCON)

I create a new profiler for web applications

💪💪💪

🎉 wsgi_lineprof 🎉

wsgi_lineprof• WSGI middleware

• Works well with WAFs: Django, Flask, Bottle…

• Easy to integrate with existing applications without changing existing code of app.

• Provides line-by-line profiling

• Records every function call during a request

• Records the followings for every lines:

• Number of times the line was executed

• Amount of time spent to execute the line

How to use# Assume that WSGI app is defined as "app"

server = make_server('127.0.0.1', 8000, app) server.serve_forever()

# wsgi_lineprof filters = [ FilenameFilter("main.py"), TotalTimeSorter(), ] app = LineProfilerMiddleware(app, filters=filters)

Demo

Conclusion• wsgi_lineprof is a simple WSGI middleware for line-

by-line profiling

github.com/ymyzk/wsgi_lineprof

• Proposals, feedback, and PRs are welcome!

• Let’s participate in ISUCON using Python and win a victory! 🏆🏆🏆

Thank you for Listening!!