Low Ceremony Processes for Short Lifecycle Projects

Tony  Wasserman  Carnegie  Mellon  Silicon  Valley  

 Int’l  Conf.  on  So9ware  and  System  Process  

19  May  2013  

Bay to Breakers: a Complex System

Bay to Breakers Processes 50000 people traveling 12K over 4 hours

•  Security  •  Traffic  control  •  RegistraEon  •  Finances  •  Publicity  •  Awards  •  CollecEbles  •  Refreshments  

•  Sponsor  management  •  RecruiEng  •  Exhibitor  management  •  Entertainment  •  Medical  •  Warmup  and  start  •  Individual  Eming  

Individual Runner

Individual runner processes 1 person running 12K in 1+ hour

•  Travel    •  Refreshments  

•  Individual  Eming  

Facebook’s early days Mark  Zuckerberg  at  Harvard  in  2004  

How  much  process  did  he  use?    Where’s  the  spec?  

“Process Ceremony”

•  The  extent  to  which  a  well-­‐defined  and  carefully  followed  process  is  needed  

•  More  process  steps  needed  for  –  Life-­‐criEcal  and  business-­‐criEcal  systems  

–  Hard-­‐to-­‐modify  systems  

–  Systems  with  high  financial  risk  

–  Systems  with  high  public  visibility  

Different types of systems need different types of processes

High  ceremony  processes  may  follow  a  defined,  or  even  an  op7mized,  process,  as  put  forth  by  the  SEI.  

CMMI 1.2 Key Practices

•  ConfiguraEon  Mgmt.  •  Measurement  and  Analysis  •  Project  Monitoring  and  

Control  •  Project  Planning  •  Product  and  Process  QA  •  Requirements  Mgmt.  •  Supplier  Agreement  Mgmt.  •  Decision  Analysis  and  

ResoluEon  •  Integrated  Project  Mgmt.  •  OrganizaEonal  Process  Def.  •  OrganizaEonal  Process  Focus  

•  OrganizaEonal  Training  •  Product  IntegraEon  •  Requirements  Development  •  Risk  Management  •  Technical  SoluEon  •  ValidaEon  •  VerificaEon  •  OrganizaEonal  Process  

Performance  •  QuanEtaEve  Project  Mgmt.  •  Causal  Analysis  and  

ResoluEon  •  OrganizaEonal  InnovaEon  

Too  much  process?  

 

How  many  organizaEons  truly  address  all  of  these  areas,  even  when  building  

complex  systems?  

Modern Startups

•  Mobile  apps  

•  Consumer-­‐oriented  web  and  cloud  apps  

•  Add-­‐ons  to  exisEng  applicaEons  •  Development  and  deployment  tools  and  pla[orms  

•  ApplicaEon  monitoring  

•  Web  analyEcs  

•  AcEvity  trackers  •  “Smart”  sensor-­‐based  devices  

Startups  ini7ally  have  few,  if  any  processes.    Some  individual  developers  have  good  habits;  some  hack.  

What are the most important activities for a tiny startup? •  Requirements:  addressing  customer  pain  points  •  Delivering  funcEoning  code:  minimizing  Eme  to  market  •  Product  architecture;  building  on  proven  components  •  Product  usability  •  Product  performance,  robustness,  scalability  •  Development  and  deployment  environments  •  Building  a  team  •  Business  development  with  partners  •  GeneraEng  favorable  market  awareness  •  Funding,  burn  rate  and  revenue  sources  

How many of these activities address the key needs of the tiny startup?

•  ConfiguraEon  Mgmt.  •  Measurement  and  Analysis  •  Project  Monitoring  and  

Control  •  Project  Planning  •  Product  and  Process  QA  •  Requirements  Mgmt.  •  Supplier  Agreement  Mgmt.  •  Decision  Analysis  and  

ResoluEon  •  Integrated  Project  Mgmt.  •  OrganizaEonal  Process  Def.  •  OrganizaEonal  Process  Focus  

•  OrganizaEonal  Training  •  Product  IntegraEon  •  Requirements  Devel.  •  Risk  Management  •  Technical  SoluEon  •  ValidaEon  •  VerificaEon  •  OrganizaEonal  Process  

Performance  •  QuanEtaEve  Project  Mgmt.  •  Causal  Analysis  and  

ResoluEon  •  OrganizaEonal  InnovaEon  

Process  Mismatch  

Needed: process to fit

Enter  agile  processes  and  lean  soFware  development  

Main Principles of Agile Software Development

•  Capture  and  define  requirements  at  a  high  level  •  User  involvement  is  essenEal  •  The  team  must  be  allowed  to  make  decisions  •  Develop  small  incremental  releases,  and  iterate  •  Focus  on  frequent  delivery  of  products  •  Requirements  evolve  but  the  Emescale  is  fixed  •  Complete  each  feature  before  moving  on  to  the  next  •  TesEng  is  integrated  throughout  the  product  lifecycle  •  A  collaboraEve  and  cooperaEve  approach  between  all  

stakeholders  is  essenEal  From  managedmayhem.com/2009/05/06  

Lean Software Development

•  Pursues  perfecEon  through  the  systemaEc  idenEficaEon  and  eliminaEon  of  wasteful  acEviEes  –  ReducEon  of  non-­‐value-­‐added  acEvity  –  Smoothing  of  flow  

–  EliminaEon  of  overburdening  

•  “Agile  methods”  contained  less  “waste”  and  thus  produced  a  beger  outcome  

Software Engineering Evolves

•  New  processes  •  New  tools  and  technologies  •  Extensive  use  of  FOSS  •  Dependence  on  third-­‐party  services    •  Extremely  rapid  changes  

•  Difficulty  of  replicaEng  problems  

Common Thread: Interactive access to the Internet

Modern software engineering is different

•  SimilariEes  –  Quality/reliability  issues  –  Performance  (download  Emes,  database  access)  –  ConfiguraEon  management  –  Architecture  issues  –  Version  management  

•  Key  Differences  –  User  interface  –  Release  frequency  – MulEple  programming  notaEons  (HTML,  Perl,  Java,  JavaScript,  etc.)  –  Reliance  on  APIs  and  reuse  –  Diversity  of  development  teams  –  Development  methodology  

Where are the process changes?

•  Frequent  feature-­‐driven  releases  (need  for  DevOps)  •  Short,  agile  development  cycle  

•  Frequent  builds  •  Daily  standup  meeEngs    

•  Strong  UX  end  user  focus  •  Extensive  use  of  exisEng  infrastructure,  frameworks,  

components,  i.e.,  write  less  code  

•  Heavy  tool  emphasis  

•  Expanded  tesEng  and  QA  

Stringent customer expectations

•  ApplicaEon  always  up  –  How  do  you  upgrade  hardware  and  so9ware?  –  How  do  you  maintain  control  over  third  parEes?  

•  Easy  to  use  and  navigate  •  Secure  and  reliable  end-­‐to-­‐end  transacEons  •  Acceptable  performance  at  normal  connecEon  speeds  

•  Suitable  privacy  policy  

Application must be fault-tolerant, reliable, scalable, usable

Quality Assurance Is Critical

•  Impact  of  consumer  reviews,  especially  in  app  stores  

•  Front-­‐end  and  usability  tesEng    –  FuncEonal  tesEng  for  common  tasks  

–  Link  tesEng  (can  be  largely  automated)  

–  A/B  tesEng  •  Back-­‐end  tesEng  

–  White-­‐box  tesEng  

–  FuncEonal  tesEng  •  Load  tesEng  

–  Match  site  infrastructure  to  esEmated  load  

Mobile and Web Applications •  Involve  mulEple  design  dimensions  

–  Front  end  •  InteracEon  design,  e.g.,  navigaEon  •  Image  and  Graphics  •  ScripEng  (JavaScript)  

–  Back  end  •  Architecture  and  component  integraEon  •  Performance  (response  Eme)  •  Database  design  •  Scalability  (number  of  users)  •  Fault  tolerance  and  reliability  •  Logical  structure  of  code  (programming  and/or  scripEng  language)  

–  IntegraEon  architectures  and  frameworks  

Modern applications depend on third-party services

•  Content  caching  •  Credit-­‐card  processing  •  Order  fulfillment  

•  Visitor  tracking  •  Email  processing  

•  Ad  servers  •  Discussion  and  chat  forums  

Many “web site” functions are outsourced – a site is not a single entity

Needed team skills for successful development and deployment

•  Planning  of  feature-­‐based  releases    •  System  architecture  (hardware  and  so9ware)  

•  Security  •  Deployment  environments  (devops)  

•  Programming  in  various  languages  

•  UX  and  UI  •  FuncEonal  and  stress  tesEng;  mulEple  client  tesEng  

•  Scalability  •  Web  infrastructure  (firewalls,  proxy  servers,  load  balancers)  

Processes driven by tools

•  Programming  environments  

•  CollaboraEon  tools  •  Mockups  and  prototyping  

•  Device  emulators  

GitHub project forge

CollabNet TeamForge

Appcelerator’s Titanium Studio

Xcode for iOS App

Eclipse for Android development

Telerik Load Testing

Startups grow up The Week, January 15, 2013

When Startups Grow Up – Process Accumulates

•  More  people,  more  process  

•  More  customers  dependent  on  the  system  

•  Products  evolve  more  slowly  

•  Developers  spend  less  Eme  developing  

•  Key  founders  depart  to  create  another  startup  (or  to  become  angel  investors  and  advisors)  

Increased  business  and  technical  risks  drive  process  change  

Contact information

Anthony  I.  (Tony)  Wasserman    post:      Carnegie  Mellon  Silicon  Valley  

     MoffeO  Field,  CA  94035  USA  email:    tonyw@acm.org  Skype,  G+:  tony.wasserman  TwiOer:    twasserman