We open sourced our trade secrets, and so can you!
-
Upload
james-meickle -
Category
Technology
-
view
546 -
download
0
description
Transcript of We open sourced our trade secrets, and so can you!
We open sourced our trade secrets, and so can you!
James MeickleDeveloper evangelist, AppNeta
@jmeickle
Boston Rails MeetupAugust 26, 2014
AppNeta
• Application performance management (APM)– Client– Network– Server
• Software as a service• Cloud hosted
A brief history of APM
• Closed source monitoring for closed source applications
• Oriented around mainframes and J2EE
• Incredibly high cost, not production-ready
• Today: rapid value, easy deployment, low cost
AppNeta TraceView
• Distributed tracing (Zipkin, Dapper)
• Bytecode instrumentation
• Java, .NET, PHP, Ruby, Python, node.js
• Definitely not Tracelytics!
THE GEMImplementing TraceView’s functionality in Ruby
Our maintainer
• Peter Lombardo came on a few years ago
• Exclusively focused on the oboe gem
• Works remote from Italy, or else he’d be giving this talk!
oboe
• Installed like a normal gem (with C extension)
• No configuration required• Provides:
– Application performance management
– Software as a service– Distributed tracing– Bytecode instrumentation– Trace API
Application performance management
• Collect performance data:– Database queries– Cache requests– Remote calls– Templating time
• Without causing problems:– Identical behavior– Overhead below 1%– Stable
Software as a service
• Don’t do any processing locally
• Report data to the agent (UDP)
• Report data to collector (SSL, on Heroku)
Distributed tracing
• Follow calls from one application to another– Different servers– Multiple languages– Asynchronous
• Common request/response protocols:– HTTP– Thrift– EJB (for JRuby)
Bytecode instrumentation
• Support common components:– Frameworks– Libraries– Drivers/clients
• Without code modification– Ruby, Python: monkey
patching– Java: classloader– PHP:
Trace API
• Same API that we use when adding layers
• Our backend is built around supporting it
• Duck typing for events• Report anything!*
*please don’t report credit card numbers, thanks
Example 1: memcache
Example 2: rack
Example 2: Net::HTTP
Example 4: resque
THE PROCESSOpening the oboe gem to the world
Why did we open source oboe?
• We <3 open source• Larger customers want
to audit our code• Increased community
contributions• Better capture of
reported issues• Good publicity
Why couldn’t we “flip the switch”?
• The code was not appropriately documented
• The test suite needed work• There was no contribution
guide• It wasn’t licensed• Not branded properly!• Issues contained
customer-specific information
Documentation
• Added
Tests• “Before opensourcing, we had to
move away from manual testing via nosetests (30 something stack variations, manual boot and manually run nosetests, rinse and repeat for the next hour).
We went with Minitest. Today with Minitest, we now test 7 Ruby versions: each with 202 tests, 1979 assertions - biggest win beyond coverage is that it’s automated with Travis for each git commit.”
– Peter Lombardo
Contribution guide
Branding
• We were already AppNeta TraceView
• But we still were Tracelytics in our code
• Not as simple as a s/foo/bar/; lots of unexpected places (support URLs!)
Licensing
• Open source != free software
• Competitive advantage• Still an unfathomable
change from where we were 10 years ago
Issues with issues
• We didn’t want to lose historical issues
• New issues would also need to reference specific customers and post about their data
The solution
• “The key was to use two repos (1 private and 1 public) using two git remotes. Now our default is that all issues are filed in the public repository unless it contains customer specific data. If a public issue, turns out to need a pointer to some internal data or resource, we file a corresponding internal issue that points to the public issue. We now always prefer the public repo over the private and only use the private when absolutely necessary.”
– Peter Lombardo
The final countdown
Marketing helped too!
• Pre-launch– Implications of licensing– How to reward and
publicize contributors– Info page for repo
• Post-launch– Launch blog post– Social media– Existing customers!
Launch day
• Set up the public repo as a private one
• Switch our build process (rubygems) to use the right repo
• Post the new build as a GitHub Release too
• Publish the blog post!
THE RESULTWhat we’ve seen after open sourcing
appneta / oboe-ruby
• Not many visitors • But they were incredibly
engaged: more than 25% of them cloned the repo.
• And about a third of them contributed code!
More contributions
• 80%: post an issue• 15%: post a fix/snippet• 5%: an entire feature!
– Grape API: Todd Lunter at Swipely
– EventMachine: Diogo Benica at Abril Midia
Happier maintainer• “Not sure if this part is going to make
sense but working on an opensource project is like having a weight lifted off of your shoulders. When it’s private, proprietary code, the burden is completely on you to assure quality and to not screw up. When the code is opensource, there is a slightly calming feel because you know that your code is public and is available to be reviewed and improved upon by anyone. We love the contributions we’ve gotten so far and are always looking for more.”
– Peter Lombardo
GitHub issues
• Work being done lives in GitHub issues on the public repo
• Anyone can post there!
Supporting contributors
• Most people use the trace API incorrectly
• We only notice when they file support tickets
• Post custom code on GH, and we can help while you write it!
Thank you!Come work for us! We’re a mostly Python shop
You can meet us at:
• Sep 15-17: Velocity New York• Sep 18: WebPerfDays New York• Oct 21: TechBreakfast New York• October 24: Surge (DC)• Nov 11-14: AWS re:Invent
(Vegas!)
Or just try us out:
http://www.appneta.com/products/traceview/