Post on 07-Apr-2018
8/6/2019 Djangocon 2011 Talk Jesper Noehr
1/25
Bitbucket - Lessons learned@jespern
1Tuesday, June 7, 2011
8/6/2019 Djangocon 2011 Talk Jesper Noehr
2/25
8/6/2019 Djangocon 2011 Talk Jesper Noehr
3/25
Who am I?
Founder of Bitbucket, now the CTO Started programming back in 1998 Been using Django for 4 years (magic
removal branch, anyone?) @jespern on Twitter
3Tuesday, June 7, 2011
Opening .exe les -> Basic -> PHP (.php3) -> C -> Perl -> Python
8/6/2019 Djangocon 2011 Talk Jesper Noehr
4/25
Bitbucket in summary
First commit was May 2008, launch ~1
month after that. 2 people, part time, for 2 years. Ran entirely on AWS. Acquired in August 2010.
4Tuesday, June 7, 2011
Eirik left after 2 years, had a baby, just me after that.
Started from my blog.
8/6/2019 Djangocon 2011 Talk Jesper Noehr
5/25
Bitbucket in summary
Now 9 people, 6 engineers, one designer. 2 in Sydney, 7 in San Francisco. Only 2 people knew Django, only 3 Python! 1 is a Mercurial core developer. Incubated at Atlassian.
5Tuesday, June 7, 2011
Mehmet and I knew Django, Brodie knew Python.
8/6/2019 Djangocon 2011 Talk Jesper Noehr
6/25
Behind the scenes
Its all Python. 73934 lines of code, 86% is Django. We use Mercurial (surprisingly.) PostgreSQL, virtualenv/pip, etc.
6Tuesday, June 7, 2011
The entire website is 100% django, only parts that arent are background stu f like scripts,various SSH stu f , hooks.
Deploys semi-contained in virtualenvs, individual Pythons, most system-wide packagesinside the env as well (statically linked.)
8/6/2019 Djangocon 2011 Talk Jesper Noehr
7/25
Behind the scenes
On production: HAProxy, nginx, gunicorn, puppet. Roll out once a week, hotxes go out
instantly.
7Tuesday, June 7, 2011
Didnt use to be like this, but more details on that later.
8/6/2019 Djangocon 2011 Talk Jesper Noehr
8/25
Behind the scenes
Testing: Selenium for functional tests. Kong for headless tests (@ericholscher) Bamboo for CI (just getting there.)
8Tuesday, June 7, 2011
8/6/2019 Djangocon 2011 Talk Jesper Noehr
9/25
Lessons learned
9Tuesday, June 7, 2011
Experiences over the years, things that helped us and things that didnt.
8/6/2019 Djangocon 2011 Talk Jesper Noehr
10/25
Stay idiomatic
10Tuesday, June 7, 2011
Probably the most important point Im gonna make.What I mean is to follow Django idioms, models, forms, dont step too much outside thecommon way of doing things.
Doing this has helped us *so* much. Its not always the most intuitive thing to do, but itcomes in very handy in so many situations.
8/6/2019 Djangocon 2011 Talk Jesper Noehr
11/25
From our codebase (actual code):
11Tuesday, June 7, 2011
Basically everythings like this, from the core concept of Repositories to everything else wedo.
8/6/2019 Djangocon 2011 Talk Jesper Noehr
12/25
Everythings an app
Repositories
Issue tracker User accounts (SSH, etc.) Compare view ...
12Tuesday, June 7, 2011
They all have models.py, forms.py, views.py, etc.Can perhaps run standalone but wouldnt make sense.
8/6/2019 Djangocon 2011 Talk Jesper Noehr
13/25
Benets & Disadvantages
13Tuesday, June 7, 2011
8/6/2019 Djangocon 2011 Talk Jesper Noehr
14/25
Free upgrades, little backporting
Getting/nding help is usually easy --awesome community
Easy for new people to pick up...
14Tuesday, June 7, 2011
...lead into where you can point them for the documentation (next slide.)
8/6/2019 Djangocon 2011 Talk Jesper Noehr
15/25
https://docs.djangoproject.com/Wherere the docs?
15Tuesday, June 7, 2011
Really nice to be able to just point people there. They can go there and learn all the Djangoidioms.
http://docs.djangoproject.com/http://docs.djangoproject.com/8/6/2019 Djangocon 2011 Talk Jesper Noehr
16/25
Kinda feels like youre shoe-horningsometimes
Unusual application requirements (werequite transactional)
..
16Tuesday, June 7, 2011
CNAME is one thing we had to really twist Django into doing for us.
Transactional = involving the disk (we obviously do this a lot), handling errors, rolling back,etc. The Django ORM isnt *that* great. Certainly su ces, but does dumb things like
redundant joins and the transactional stu f (we use Postgres -- mention isolation levels.)
8/6/2019 Djangocon 2011 Talk Jesper Noehr
17/25
For better or worse.. Usually better.We stick to idioms
17Tuesday, June 7, 2011
Next up, things change.
8/6/2019 Djangocon 2011 Talk Jesper Noehr
18/25
Things change
18Tuesday, June 7, 2011
Dont do vendor lock-in! We did (AWS - S3/cloudfront, but mostly EBS), and it hurt is quite abit down the line. The places in which we didnt, came in *really* handy.
We took the pain of making sane technology choices and decisions early -- often a good
idea. AWS great for bootstrap, but we didnt make use of anything we couldnt replace later(EBS looks like a disk).
8/6/2019 Djangocon 2011 Talk Jesper Noehr
19/25
Embrace it!
19Tuesday, June 7, 2011
Design your app in such a way that youre not bound to any specic technology or vendor.
Changes *will* happen, whether its to replace a bad technology, or more often, to makesomething scale better.
8/6/2019 Djangocon 2011 Talk Jesper Noehr
20/25
Good choices we made
Linux
Django
WSGI
No need to pick a DB Everything else we left
open-ended
20Tuesday, June 7, 2011
WSGI was awesome, as theres a ton of WSGI hosts out there -- weve been throughmod_wsgi, uwsgi, and now gunicorn. Hell, we even ran FastCGI.We could rip out postgres for mysql or whatever any day.
8/6/2019 Djangocon 2011 Talk Jesper Noehr
21/25
Open source
21Tuesday, June 7, 2011
Piggyback on existing e f orts, stand on the shoulders of giants. Dont fear NIH.
8/6/2019 Djangocon 2011 Talk Jesper Noehr
22/25
We use stuff like... Celery, South, etc.
-(social)registration,-compressor, ...
A lot of really well-donestuff out there!
22Tuesday, June 7, 2011
8/6/2019 Djangocon 2011 Talk Jesper Noehr
23/25
And try to give back django-piston
django-dogslow
tipper
23Tuesday, June 7, 2011
Explain what dogslow is, how it relates to tipper.
8/6/2019 Djangocon 2011 Talk Jesper Noehr
24/25
Be transparent& humble
24Tuesday, June 7, 2011
Explain outages, dont think youre better than your users.
8/6/2019 Djangocon 2011 Talk Jesper Noehr
25/25
Thanks!
25Tuesday, June 7, 2011