TSP SYMPOSIUM Schneider, Henry.pptx

TSPSymposium

MexicoCity

February23–24,2016

®2016AllRightsReserved-PPQC 24/18/16

PPQCPROCESS AND PRODUCT

Q UA L I T Y C O N S U LT I N G

•  CoreConcepts•  Iden-fyingSta-s-calStability•  Selec-ngProcessesandPerformanceMeasures•  Ques-onsandAnswers•  SomeCommonMeasures(referenceonly)




•  PerformedProcesso  Aprocessthataccomplishestheneededworktoproducework

products•  ManagedProcess

o  Aperformedprocessthatisplannedandexecutedinaccordancewithpolicy;employsskilledpeoplehavingadequateresourcestoproducecontrolledoutputs;involvesrelevantstakeholders;ismonitored,controlled,andreviewed;andisevaluatedforadherencetoitsprocessdescripLon

•  DefinedProcess

o  AmanagedprocessthatistailoredfromtheorganizaLon’ssetofstandardprocessesaccordingtotheorganizaLon’stailoringguidelines;hasamaintainedprocessdescripLon;andcontributesprocessrelatedexperiencestotheorganizaLonalprocessassets




•  SpecialCauseofVaria-ono  Acauseofadefectthatisspecifictosometransientcircumstanceand

isnotaninherentpartofaprocess

•  CommonCauseofVaria-on

o  ThevariaLonofaprocessthatexistsbecauseofnormalandexpectedinteracLonsamongcomponentsofaprocess

•  StableProcess

o  ThestateinwhichallspecialcausesofprocessvariaLonhavebeenremovedandpreventedfromrecurringsothatonlycommoncausesofprocessvariaLonoftheprocessremain




•  QualityandProcessPerformanceObjec-ves(QPPOs)o  QuanLtaLveobjecLvesandrequirementsforproductquality,service

quality,andprocessperformance

•  CapableProcess

o  AprocessthatcansaLsfyitsspecifiedproductquality,servicequality,andprocess-performanceobjecLves

•  ProcessPerformance

o  Ameasureofactualresultsachievedbyfollowingaprocess.Processperformanceischaracterizedbybothprocessmeasures(e.g.,effort,cycleLme,anddefectremovalefficiency)andproductorservicemeasures(e.g.,reliability,defectdensity,andresponseLme)




•  ProcessPerformanceBaseline(PPB)o  AdocumentedcharacterizaLonofprocessperformance,whichcan

includecentraltendencyandvariaLon.Aprocessperformancebaselinecanbeusedasabenchmarkforcomparingactualprocessperformanceagainstexpectedprocessperformance




•  ProcessPerformanceModel(PPM)o  AdescripLonoftherelaLonshipsamongthemeasurableaZributesof

oneormoreprocessesorworkproductsthatisdevelopedfromhistoricalprocessperformancedataandisusedtopredictfutureperformance

o  OneormoreofthemeasureableaZributesrepresentcontrollableinputsLedtoasub-processtoenableperformanceof“what-if”analysesforplanningdynamicre-planning,andproblemresoluLon

o  PPMsincludestaLsLcal,probabilisLc,andsimulaLonbasedmodelsthatpredictinterimorfinalresultsbyconnecLngpastperformancewithfutureoutcomes

o  PPMsmodelthevariaLonofthefactorsandprovideinsightintotheexpectedrangeandvariaLonofpredictedresults




•  ComposetheDefinedProcesso  ComposingtheProject’sDefinedProcess(PDP)involvesidenLfying

alternaLvestooneormoreprocessesorsub-processes,performingquanLtaLveanalysisofperformanceandselecLngthealternaLvesthatarebestabletohelptheprojectachieveitsQPPOs

•  Sta-s-calProcessControl(SPC)

o  StaLsLcallybasedanalysisofaprocessandmeasuresofprocessperformance,whichwillidenLfycommonandspecialcausesofvariaLoninprocessperformanceandmaintainprocessperformancewithinlimits




•  Sta-s-calTechniqueso  TechniquesadaptedfromthefieldofmathemaLcalstaLsLcsusedfor

acLviLessuchascharacterizingprocessperformance,understandingprocessvariaLon,andpredicLngoutcomes.

o  ExamplesofstaLsLcaltechniquesincludesamplingtechniques,analysisofvariance(ANOVA),chi-squaredtests,andprocesscontrolcharts

•  CausalAnalysis

o  Theanalysisofoutcomestodeterminetheircauses

o  Usedtoaddressselectedissuesanddiagnosedeficienciesinachievingtheproject’sQPPOs




Beforeanorganiza-oncanperformHighMaturityac-vi-es,itmusthavethefollowinginplace:

o  CapabilitytogatherandusedataatallorganizaLonallevels,i.e.,projectmemberswho

•  Gatherdataontheirownwork

•  Understandandusethedatainplanningandperformingtheirwork

o  Processesthatspecifyhowandwhendataarecollected

o  ConsistentexecuLonofthedefinedprocess,wheretailoringishandledinacontrolledanddisciplinedfashion




•  Everythingwedoinlife,includingproductdevelopmentandservicedelivery,isaseriesofinterconnectedprocesses

•  Allprocessesarevariable•  Peopleperformingtheprocessisthelargestsourceof

varia-on•  Understandingvaria-onisthebasisformanagementbyfact

andsystema-cimprovement:

o  Understandthepast–quan-ta-vely

o  Controlthepresent–quan-ta-vely

o  Predictthefuture–quan-ta-vely




•  ScaOerDiagrams•  RunCharts•  Cause-and-EffectDiagrams•  Histograms•  BarCharts•  ParetoCharts•  ControlCharts




•  Realprocessbehaviormustbeunderstoodbeforedrawingreasonableconclusionsaboutwhereperformanceneedstobeimproved

•  Asktheseques-onstofindoutaboutrealprocessbehavior:

o  WhatisthenormalorinherentprocessvariaLon?

o  WhatdifferenLatesinherentfromanomalousvariaLon?

o  WhatiscausingtheanomalousvariaLon?

•  Sta-s-calProcessControlprovidesthemethodsandtoolsneededtomeasureandanalyzeprocessbehavior,drawconclusions,anddriveimprovement




•  Asummaryorcharacteriza-onofadistribu-on(i.e.,asetofnumbers)

•  Acharacteriza-onofacentraltendency(e.g.,mean,median,andmode)

•  Acharacteriza-onofdispersion(e.g.,variance,standarddevia-on,interquar-lerange,andrange)




Statistics (specifically hypothesis testing) enable you to place a confidence interval on the Central Tendency and Dispersion of the population and on future samples

A data population consists of the total possible observations with which you are concerned but to which you do not necessarily have access (X1 thru X15).

X5 X15

X14 X13

X12

X11

X6

X10 X8 X7 X9

X3

X2

X1 X4

A sample is a set of observations selected from a data population that you can access.




•  Hypothesistes-ngusessta-s-calanalysisasaformalwayofmakingacomparisonanddecidingwhetherornotthedifferenceissta-s-callysignificant

•  Hypothesistes-ngconsistsofcomparinganullandanalterna-vehypothesis:

o  ThenullhypothesisstatesthatthemembersofthecomparisonarestaLsLcallyequal;thereisnodifference(aconcrete,defaultposiLon)

o  ThealternaLvehypothesisstatesthatthereisastaLsLcallysignificantdifference;itissupportedwhenthenullhypothesisisrejected

•  Theconclusioneitherrejectsorfailstorejectthenullhypothesis.




Whenthepislow,thenullmustgo.

Whenthepishigh,thenullmustfly.

Note:Thep-valueisthekeyoutputinstaLsLcalanalysisthatstudentsaretaughttoidenLfyandusetodrawaconclusionregardingthehypothesistestcomparisonorregardingthesignificanceofastaLsLcalmodel




•  “Whileeveryprocessdisplaysvaria1on,someprocessesdisplaycontrolledvaria1on,whileothersdisplayuncontrolledvaria1on”

•  Rou-nevaria-on:o  AstablepaZernofrandomvariaLonoverLme

o  TypicalobservaLonwithinacommoncausesystem•  Aprocessisoutofsta-s-calcontrolifitisopera-nginthe

presenceofassignablecauseso  AfactorthatisnotinherentintheprocessisaffecLngitsperformance

o  VariaLonintheprocesscanbe“assigned”toaspecificcause•  Aprocessundersta-s-calcontrolispredictable




•  Wheeler’sEmpiricalRules:o  60%≤±1σ≤75%

o  90%≤±2σ≤98%

o  99%≤±3σ≤100%

Wheeler’sEmpiricalRulesarerobustwithregardtoaremarkablenumberofdistribu-ons




•  SPCisatechniquethatcanbeusedtotellyouhowmuchvariabilitythereisinaprocess

o  HowmuchvariabilityisduetorouLnevariaLon

o  Howmuchvariabilityisduetosomeuniqueeventorfactornotinherentintheprocess(assignablecauses)

o  AcontrolchartisatoolusedinSPCtohelpdothis

•  Controlchartsaregeneratedfromtwodatasets

o  PPBdata–usedtostaLsLcallyderiveaprocessaverage,UpperControlLimit(UCL),andLowerControlLimit(LCL)(±3σ)

o  Actualperformancedata–ploZedlikeaRunChartandcomparedagainstbaselinedata




•  Whichofthese20projectsshouldbeactedon?

0

1

2

3

4

5

6

7

8

9

10

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Num

bero

fNon

-com

plia

nces

ProjectID




•  Differenttechniquescanbeusedtounderstandprocessvaria-on

o  Simplegraphs,suchasfrequencyhistogramsandbarcharts

o  StaLsLcalformulae,suchasstandarddeviaLonandrange

o  StaLsLcalprocesscontroltools,suchasXmRchartsandu-charts

o  Analysisofvariancetechniques,suchasANOVAandhypothesistest




ExcepLonal(unexpected)

variaLon

RouLne(expected)variaLon ExcepLonal(unexpected)variaLon




0

1

2

3

4

5

6

7

8

9

10

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Num

bero

fNon

-com

plia

nces

ProjectID

UpperControlLimit=11.7

Mean=5

LowerControlLimit=0




UpperControlLimit

LowerControlLimit

ProcessAverage

ExcepLonalprocessvariaLon

ExcepLonalprocessvariaLon

ActualPerformanceDatafromProjectXYZ




1σ

2σ

3σ

3σ

2σ

1σ

ZONEA

ZONEA

ZONEB

ZONEB

ZONEC

ZONEC

UCL

LCL

ProcessAverage




•  1singlepointoutside3σ

•  2of3successivevaluesonsamesideofcenterlineandoutsideof2σ

•  8successivevaluesonsamesideofcenterline




“…inso:waredevelopmentitisdifficulttousecontrolchartsintheformalSPCmanner.Itisaformidabletask,ifnotimpossible,toaDempttodefinetheprocesscapabilityofaso:waredevelopmentprocess.” -StephenH.Kan(1995,p.143)•  Cri-calassump-onforinterpre-ngcontrolcharts–dataare

affectedbyhomogeneoussourcesofvaria-on

o  SamepopulaLon

o  SamebehavioralimplementaLonoftheprocess

o  Samecommoncausesystem




“Despitetheseissues,controlchartsareusefulforso:warequalityimprovement–whentheyareusedinarelaxedmanner…controlchartsuseinso:wareisnotintermsofformalsta1s1calprocesscontrolandprocesscapability…usedasatoolforimprovingconsistencyandstability…moreappropriatelycalledpseudo-controlcharts.”

-StephenH.Kan(1995,p.144)




•  Systemdevelopmentprocessesrou-nelyoperatewithinthepresenceofmul-plecommoncausesystems–PPBsmixnaturalvaria-onwith:o  VariaLonindeveloperskillso  VariaLoninaZributesofworkproducts(e.g.complexity)o  VariaLoninsituaLons

•  Thevoiceoftheprocessisacacophonyofmanyvoicesfrommanybehavioralimplementa-onsoftheprocess:o  ConfoundedbyvariaLonfrommanydifferentsourceso  Controllimitsareexcessivelywide–occasionally10Xrangeo  Outofcontroleventsaredifficulttodetecto  ‘StaLsLcalstability’maynotoffercontrolorpredictability

o  AsinglePPBmayprovidepoorcharacterizaLon




Experienced Inexperienced

Method&SupportTools

NoMethod&SupportTools

Eachcellrepresentsauniquecommoncausesystemwithdifferentmeanresults




•  Intheexample,methodsandexperiencelevelsare:o  SourcesofprocessvariaLonthatguidetheidenLficaLonofcommon

causesystems

o  FactorswhosevariaLonmustbecontrolledinestablishingseparatecommoncausesystems

o  LikelytoappearasassignablecausesofprocessvariaLonifdatafromseveralcommoncausesystemsarecombinedintoasinglePPB

•  Eachcellisadifferentcommoncausesystemo  DifferentPPBs

o  DifferentbehavioralimplementaLonsoftheprocess•  Thetypicalusesofcontrolchartsinsystemdevelopment

regularlymixdatafrommul-plecommoncausesystems




•  Controlchartsareappropriatewhen:o  TheploZedresultscharacterizeprocessperformancewithinasingle

commoncausesystem

o  DatadescribeasinglebehavioralimplementaLonoftheprocess

o  Processfidelityishigh(theprocessiswell-definedandperformedincompliancewithhowitisdefined)

•  Whencondi-onsareappropriateforcontrolcharts;

o  Processstabilityisameaningfulconcept

o  AggregaLngPPBsprovidesagoodbasisforunderstandingperformance




Individual

Aggregate

OR




•  TypicalML4prac-cesresultinplokngaggregatedinspec-onresults

o  Acrossdifferentdevelopers

o  Acrossdifferentreviewers

o  AcrosscomponentsorworkproductswithdifferentaZributes




•  Sincepeoplearethegreatestsourceofvaria-oninsystemdevelopmentamorePersonalSolwareProcess(PSP)-likeapproach:

o  Treatsindividualsasseparatecommoncausesystemsratherthanasassignablecauses

o  Decreasesthewidthofcontrollimits

o  Makesunlikelyorunexpectedeventseasiertodetect




“A:erproductsize,peoplefactorshavethestrongestinfluenceindeterminingtheamountofeffortrequiredtodevelopaso:wareproduct.”

-BarryBoehm,et.al.(2000)




•  Establish and maintain the organization’s QPPOs, which are traceable to business objectives o  TheQPPOsarederivedfromtheorganizaLon’sbusinessobjecLves

andwilltypicallybespecifictotheorganizaLon,group,orfuncLon

o  TheQPPOstakeintoaccountwhatisrealisLcallyachievablebaseduponaquanLtaLveunderstanding(knowledgeofvariaLon)oftheorganizaLon’shistoricqualityandprocessperformance




•  Asimpletaskthatmany-mesdoesnotspecifyenoughinforma-onforuseatMaturityLevel4or5

•  QPPOsneedtobeSMARTo Specifico Measurable

o Achievable

o RealisLc

o Time-based




•  Selectmeasuresandprocessesthatprovideinsightintotheorganiza-on’sabilitytomeetitsQPPOs

o  Create/updateclearunambiguousoperaLonaldefiniLonsfortheselectedmeasures

o  Reviseandupdatethesetofmeasures,analyses,andproceduresaswarranted

o  Inusage,besensiLvetomeasurementerror

o  ThesetofmeasuresmayprovidecoverageoftheenLrelifecycleandbecontrollable




•  Eachprojectandorganiza-onhasuniqueinforma-onneeds•  Wheniden-fyinggoalsandobjec-ves,considersourcessuchas

o  businessandcustomerneeds

o  constraintsandassumpLons

o  projectororganizaLoncharacterisLcs

•  domain,technology

•  peopleandskills•  productfeaturesandacceptancecriteria•  cost,schedule,andmilestones

o  risks

o  pastexperienceoftheteam




Enterprise

Howdoweperform,comparedtoothers?

Whichimprovements

shouldwefund?

Inwhattrainingortoolsmustwe

invest?

Areourcustomerssa-sfied?

Process

Howconsistentisourperformance?

Whichprocessesrequire

improvement?

WhatistheROIofimprovement

efforts?

Howareimprovementefforts

progressing?

Project

Howbigisthisproject?

Istheprojectplanfeasible?

Howistheprojectprogressing?

Whenwillwebereadytodeliver?

Product

Howbigistheapplica-on?

Whichmodulesarehavethemost

defects?

Whatisthecosttomaintain?

Whatarethequalityandgrowthtrends?

Goals

Data




Term Defini-on

Data AcollecLonofactualvaluesassignedtoameasure(example:dataforasourcelinesofcodebasemeasuremightbe100,115,121).

Measurement AsetofoperaLonshavingtheobjectofdeterminingavalueofameasure.

Measurementprocess

Theprocessforestablishing,planning,performing,andevaluaLngmeasurementwithinaprojectororganizaLon.

Measure(noun) TheruleforassigningaquanLtaLveorcategoricalvaluetooneormoreenLty/object/aZribute(e.g.process,project,resource).

Informa-onNeed

Reasonforwhichdataormeasuresareneeded;generallyphrasedas“<who>needstoknow<what>,inorderto<dowhat>”




• ProcessCompliance•  ProcessEfficiency• ProcessEffec-veness

ProcessPerform-

ance

• MilestoneCompleLon• CriLcalPathPerformance• WorkUnitProgress•  IncrementalCapability

Scheduleand

Progress

• PhysicalSizeandStability• FuncLonalSizeandStability

ProductSizeandStability

Source:Prac=calSo>wareandSystemsMeasurement

• FuncLonalCorrectness• Supportability–Maintainability• Efficiency• Portability• Usability• Dependability-Reliability

ProductQuality

• PersonnelEffort• FinancialPerformance• EnvironmentandSupportResources

ResourcesandCost

• TechnologySuitability• TechnologyVolaLlity

TechnologyEffec-ve-

ness

• CustomerFeedback• CustomerSupport

CustomerSa-sfac-on




•  BalancedScorecardsprovideamechanismtobalanceo  internalgoalsandexternalgoals

o  pastperformanceanddriversforfutureperformance

o  shorttermandlongtermperformance

o  People,Process,andTechnologyinvestment

•  Categoriesgenerallyused

FinancialOutcomes CustomerSa-sfac-on

InternalBusinessProcesses

LearningandGrowth

BalancedScorecard




•  First,establishandcommunicateastrategy

o  setfinancialobjecLves

o  idenLfykeyneedsofcustomersandmarketsegments;linkthosetofinancialgoals

o  focusoninternalbusinessprocessesandefficienciesinmeeLngcustomerneeds

o  investinlearningandgrowthtoexcelwiththeprocesses

•  Withinthescorecard

o  idenLfyhowtoseefinancialgoalsaremet

o  settargetsformeeLngcustomerneeds

o  idenLfyprocesschangesorinnovaLonsneeded

o  matchlearningandimprovementobjecLvestothese




Financial

Increaseprofitability

Increaserevenue

Increasemargins

Customer

Servemorelevelsofbusiness

Usesocialnetworkingservices

InternalProcesses

Increaserangeofservices

Providesocialnetworkingservices

Maintainsocialnetworking

site

LearningandGrowth

Buildcapability

Increasesocialnetworking

skills

BusinessGoals QPPOs




Area Measure Target Financial Profit

RevenueGrowthMargin

25%beforetax+30%overlastyear+5%overlastyear

Customer Mean#businesslevelsserved%usingsocialnetworkingservices

+10%overlastyear15%thisyear

InternalProcesses(andefficiency)

increaserangeofservicesMinimizesocialnetworkingdownLme

newPMOsalesbyJulywebsiteup99.9%

LearningandGrowth

#ofPMPs#ofmonthsoftotalsocialnetworkingexperience

+3overlastyear+24overlastyear




•  Ingeneral,processimprovementdirectlyaffectsthelowerhalfoftheBalancedScorecardo  processdevelopmentand/ormaintenance

o  processperformance

o  learningandgrowthofprocessusers

o  awayofidenLfyingandtracingQPPOs•  Changesintheseareascanlinkto

o  increasedcustomersaLsfacLon

o  increaseincustomercountortransacLonvalue

o  reducLonincosttomeetcustomerneeds•  These,inturn,translatetofinancialresults




Financial

Developnewbusiness

Reducecosts

Customer

IncreaseCustomerContact

Increasecustomervalue

InternalProcesses

Increaseprocess

compliance

Improveprocessquality

Improveproductquality

LearningandGrowth

Increaseskills

Retainmoreemployees

RaiseemployeesaLsfacLon

OverallGoal:ControlledGrowthandSa-sfiedReturnBusinessGoals QPPOs




Areas Objec-ves Measures Financial • DevelopNewBusiness

• ReduceCost

TotalrevenueRevenueinnewbusinesssegmentProducLvity(LOC/projecthr.)

Customer • IncreaseCustomerContact

• IncreaseCustomerValue

%customerswithannualinterview%employeesvisiLngcustomerscustomersaLsfacLon%withmaintenancecontracts

InternalProcesses

• IncreaseProcessCompliance• ImproveProcessQuality

• ImproveProductQuality

1-5onkeypracLces%rework%milestonesonschedule#warrantyhoursneeded

LearningandGrowth

• IncreaseSkills• RetainMoreEmployees• RaiseEmployeeSaLsfacLon

#cerLfiedinkeyareas%retenLonemployeesaLsfacLonscore



Q UA L I T Y C O N S U LT I N GSt

rate

gic

Goa

l

supporLnggoal

supporLnggoal

supporLnggoal

supporLnggoal

supporLnggoal measure

measure

measure

measure

measure performance

performance measure

measure

measure

measure

CorporateBusinessGoalsandObjec-ves

QualityandProcessPerformanceObjec-ves



Q UA L I T Y C O N S U LT I N GIm

prov

edP

rofit

abili

ty OpLmizeROI

ImprovecustomersaLsfacLon

ImproveposiLoningincurrentmarket

Minimizecostofbusiness

ImproveoperaLonsefficiency

Overheadrates

Product-ivity

CustomerSaLsfacLon

MarketShare

Revenue

CorporateBusinessGoalsandObjec-ves

MinimizedevelopmentLme

Improvereviewprocesstoremovedefectsearly

COQ

#CRs

#defects

CycleLme

QualityandProcessPerformanceObjec-ves




ImprovementGoals QPPO

Forsmallfeaturesrequestedbycustomers,providearesponsenextyearinhalftheLmewedotodaybyusinganincrementalreleasemodel

•  ReducecycleLmeforareleaseby50%

ReducenegaLveimpactofchangemanagementoncustomersaLsfacLon

•  CutresponseLmeondefecthandlingby25%

•  CutresponseLmeonchangerequestsby10%eachquarter

Usepeerreviewsforkeypartsofrequirementsworkproductsonallprojects

•  Reducecostofreworkfromrequirementsdefectsby50%nextyear




•  CoreBaseandDerivedMeasureso  RequirementsMeasures

o  So{wareSizeMeasures

o  CostandEffortMeasures

o  ScheduleandProgressMeasures

o  DefectMeasures

•  OtherKeyDerivedMeasures

o  EarnedValue

o  ProducLvity/UnitCosts

•  SummaryMeasuresforaProgramorOrganiza-on




•  Howused:o  requirementsbaselinedandtrackedthroughoutproject

o  providesinsightintogrowthandstability;alsousedtostatusprogress•  Usefulmeasures:

o  numberofrequirements(growthoverLme)

o  numberofnewrequirements,changedordeletedrequirements(stability/volaLlity)

o  requirementsstatus(approved,tested,implemented)•  Typicalcoun-ngissues:

o  howtocounta“requirement”consistently,sincesizeanddifficultyvary

o  statedvs.derivedrequirements

o  traceability/lackofrequirementstooluse

o  inmaintenance,morelikelytoseechangerequests




0

100

200

300

400

500

600

700

800

Num

bero

fReq

uire

men

ts

ProjectStatusDate

RequirementsStability

Plan

Actual

Project:OLDMS




0

10

20

30

40

50

60

Feb Mar Apr May Jun Jul Aug

Pers

onH

ours

ofW

ork

orR

ewor

k

AddRequirements

ChangeRequirements




70% 73% 73% 76% 79% 80% 81% 82% 82% 83%

8%10% 10%

13%13% 15% 14% 13% 13% 12%

4%6% 6%

6%6% 5% 4% 4% 4% 2%

18%11% 11%

5% 2% 0% 1% 1% 1% 2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

6/3/2009 6/10/2009 6/17/2009 6/24/2009 7/1/2009 7/8/2009 7/15/2009 7/22/2009 7/29/2009 8/5/2009

Passed Failed Blocked Unverified




•  Sizeisacri-calcomponentofmeasurementandmanagemento  basisforesLmaLngeffort,andquanLfyingandmanagingscope

o  basisforproducLvitymeasurement(efforttobuildaunit/size)

o  allowsnormalizaLonandcomparisonacrosssystemsandprojects(defects/size)

•  SizeMeasureso  #ofmodules,screens/reports

o  #ofcalls

o  #ofboards

o  #ofrequirements




0

50

100

150

200

250

Plan1 Plan2 Plan3 Actual

Num

bero

fCom

pone

nts

Reused

New/Modified




0.1

1

10

100

1000

10000

0.1 1 10 100 1000 10000

Staff

Mon

ths

SourceLinesofCode Thousands

Size–EffortEs-ma-ngRela-onship

ProjectUnderConsideraLon

Average

-1StandardDeviaLon

+1StandardDeviaLon




•  Effortises-mated,staffingplanned,andactualefforttrackedtoplano  overallprojectlevel

o  byLmeperiod,byworkunit/task,build,phase•  Variancemonitoredforconsistentoverorunder-run

o  underesLmatedbysizeoreffort

o  unexpectedlycomplexordefect-prone•  Costusuallyderivedfromeffort(plusothercosts)•  Usefulmeasures:

o  #oflaborhrs/FTEs(byevent,Lmeperiod)•  Typicalcoun-ngissues:

o  noLmereporLngtool;toolnotusedorenforced

o  contractorsvs.employees;mgt,admin,othersnotcaptured

o  lackofgranularityofplanandactualeffortdata

051015202530

Q1 Q2 Q3 Q4

FinancialSummary

Plan

Actual




0

2

4

6

8

10

12

14

1 2 3 4 5 6 7 8 9 10 11 12

Staff

Hou

rs(t

hous

ands

)

Repor-ngPeriods

Plan

Actual




0

1

2

3

4

5

6

7

8

9

10

1-Aug 1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May 1-Jun 1-Jul 1-Aug 1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May

EffortAlloca-on

Subcontractor Actual Internal Actual




Ac-vity % PlanHours

ProjectManagement 12% 9654

ConfiguraLonManagement 8% 6436

QualityAssurance 3% 2414

Training 4% 3218

Requirements 9% 7241

Design 12% 9654

CodeandUnitTest 20% 16090

IntegraLonTest 16% 12872

UserAcceptanceTest 9% 7241

Maintenance 5% 4023

CustomerSupport 2% 1609

100% 80450




0

10

20

30

40

50

60

70

80

90

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Tim

eto

Clo

se(d

ays)

Ticket#




•  Howused:o  examineplannedscheduleandmilestonesandactualaZainmentofeach

o  monitortaskcompleLonatalevelofdetailtobeabletorecoverfromproblems•  Usefulmeasures:

o  milestonedates(schedule)

o  dayslate

o  workunitprogress(requirements,design,code,test,etc.)•  #ofunitsplannedtobecomplete,#completed

o  scheduleaccuracy/performance•  TypicalCoun-ngIssues:

o  plandatanotbaselined;noCMcontrol

o  worknotbrokendownintomanageableworkunits

o  workunitsofdifferentsize,complexity,andimportance




Milestone Original Revised Actual

ProposalComplete 14January 17January 20January

ApprovaltoProceed1 03February 03February 03February

ReqmtsSignedOff 08April 15April 15April

ApprovaltoProceed2 02May 02May 05May

DesignSignedOff 10June 14July 15July

SystemTestComplete 17July 10August

BetaTestComplete 03October 07November

ApprovaltoProceed3 14October 10November

ReleasetoCustomer 10November 16December




0

20

40

60

80

100

120

140

#of

Uni

tsC

ompl

e-ng

Uni

tTes

t

ProjectStatusDate

UnitTestStatus

Plan

Actual




•  Howused:

o  idenLfiesthenumberofdefects(problemreports)intheso{wareproduct

o  providesinsightintotestprogressandbothpre-andpost-producLonquality

o  providesinsightintoreworkremaining•  Usefulmeasures:

o  defecttrends(open,closed,severity,ageofopen,types)

o  defectdensiLes(bysystem,subsystem,module)

o  defectorigins,detecLoneffecLveness,andleakage•  TypicalCoun-ngIssues:

o  defectsversuseffort;reworknotalways1:1

o  definiLonofadefect/classificaLoninconsistencies

o  cultural/behavioralissuesassociatedwithcollecLng




5 5

14

48

18

10 104

18

84

9 9 10

7266

74

67

60 6057 57

4852

4540 38 39 37

116

14 1518

13

5

12 11 10 10 116 8

0

10

20

30

40

50

60

70

80

Num

bero

fDef

ects

StatusofSeverity1Defects

New Open(IncludingNew) WorkedoffDuringtheWeek

OpenLastWeek=ClosedThisWeek=Downgraded=NewThisWeek=Upgraded=OpenThisWeek=

3984

------27100

------37




1017 1028 1034 1039 1048 1063 1077 1078 1096 1097 1111 1118 1129 1143 1147 1165 1179 1191

71 72 108 124 125

379454 454 484 509 510 522

605653

726776 751 782

0

200

400

600

800

1000

1200

1400

Num

bero

fDef

ects

Severity2Defects

Severity2DRsTotalDiscovered Severity2DRsTotalClosed




0102030405060708090

<1 1-2 3-4 5-8 9-18 >18

Num

bero

fPro

blem

Rep

orts

WeeksOpen

ProblemReportAgingOpenProblemReports

Dataasof15June2009

Average=5.7weeksTarget<8weeks




4

98

10

6

0

2

4

6

8

10

12

#of

Sev

erity

1D

efec

ts

Statusof37OpenSeverity1Defects

PendingValida-on

DefectUnderAnalysis

FixStarted–WorkinProgress

FixMade–PendingDelivery

FixDelivered–PendingTest




0510152025303540

A B C D E F

#of

Pro

blem

Rep

orts

ModuleID

ProblemReportStatusOpenSeverity1and2byModule

Dataasof15June2009




Module Size(KSLOC) Defects DefectDensity

(Defects/KSLOC) A 44 48 1.1

B 32 60 1.9

C 36 36 1.0

D 28 33 1.2

E 34 42 1.2

F 15 46 3.1

G 9 30 3.3

Total 198 295 1.5

Dataasof15June2009




0

20

40

60

80

100

120

140

160

180

Reqmts Design Code UnitTest IntegTest SysTest Del+90days

Defe

cts/

KSLO

C

LifeCyclePhase

DefectDetec-onProfile

AllProjects

NewProcess




PhaseDiscovered

PhaseInsertedReqmts Analysis Design Code Integ.Test SysTest Accept.

Test Ops

Requirements 50 Analysis 65 65 Design 70 80 65 Code&UnitTest 74 83 85 70 Integra-onTest 76 89 94 88 55 SystemTest 85 93 95 93 70 50 AcceptanceTest 94 95 96 94 75 70 701yearOpera-ons 98 97 98 96 88 90 90 76

Columnspresentcumula=ve%ofdefectsremoved








o  DefectMeasures


o  EarnedValue


•  SummaryMeasuresforaProgramorOrganiza-on




0

20

40

60

80

100

120

140

160

3 6 9 12 15 18 21

Cost

($K)

Months

ACWP

BCWP

Now

CostVariance

ActualCostofWorkPerformed(ACWP)

BudgetedCostofWorkPerformed(BCWP)

CostVariance(CV)isthedifferencebetweenactualandbudgetedcosts(BCWP-ACWP)

CPI=.63(<1overbudget)




0

20

40

60

80

100

120

140

160

3 6 9 12 15 18 21

Cost

($K)

Months

BCWS

BCWP

Now

ScheduleVarianc BudgetedCostofWorkScheduled(BCWS)

BudgetedCostofWorkPerformed(BCWP)

ScheduleVariance(SV)isthedifferencebetweenbudgetedcostforworkperformedandworkscheduled(BCWP-BCWS)

SPI=.52(<1behindschedule)




0

20

40

60

80

100

120

140

160

180

Proposal ActualstoDate Re-Plan1(2builds) Re-Plan2(3builds)

SLO

Cs/P

erso

nM

onth

BuildProduc-vity

Build1

Build2

Build3

15June2009








o  DefectMeasures


o  EarnedValue


•  SummaryMeasuresforaProgramoranOrganiza-on




PerformancetoSchedulemilestonebaselineforecastactualxxx 1/03 1/03 1/02yyy 6/16 6/30zzz 7/04 8/05

Risks• Descrip-ons• Ac-onPlans

OverallStatus(Red/Yellow/Green):ProjectAlphaasofApril

0

10

20

30

40

1stQtr 2ndQtr 3rdQtr 4thQtr

FinancialSummary

PlanActual

0 200 400 600

1stQtr

2ndQtr

3rdQtr

4thQtr

Func-onalityDelivered

ActualPlan




05

1015202530354045

Mon

thly

Effo

rt(F

TEH

ours

)

ActualProjectedBudget

0.720.72

0.170.000.100.200.300.400.500.600.700.80

Cum

ula-

veP

roje

ctC

osts

($10

0K) Projected

BudgetActual

CostSitua-onCurrentForecastMonthTotalMonthTotal

DeviaLonmorethan30%DeviaLonmorethan15%MeeLngbudget

MilestoneEnvisionStartVision/ScopePlanningEndsDevelCompleteV&VCompeteBetaStartBetaLiveGAProjectClosedDelaymorethan60daysDelaymorethan30daysOnTargetorEarly

05

1015202530354045

Mon

thly

Effo

rt(F

TEH

ours

)

ActualProjectedBudget

0.720.72

0.170.000.100.200.300.400.500.600.700.80

Cum

ula-

veP

roje

ctC

osts

($10

0K) Projected

BudgetActual

ScheduleSitua-onMilestone CurrLastEnvisionStartVision/ScopePlanningEndsDevelCompleteV&VCompeteBetaStartBetaLiveGAProjectClosedDelaymorethan60daysDelaymorethan30daysOnTargetorEarly




Size CostandEffort Schedule Quality

ProcessPerfor-mance

CustomerSa-sfac-

-onRisk Quick

Overview

Suppor-ngComments

<ProjectName>MonthlyStatusReport

TechnologyEffec-veness

Size

CostandEffort

Schedule

QualityandProcessPerformance

RiskandCustomerSa-sfac-on

Tech.Effec-ve-

ness




SuiteofindicatorsforeachareaofreporLngforreviewbyprojectmanagers,teammembers,andbusinessmanagers

TotalPhysicalSize

-100102030405060

08-ju

n22-ju

n06-ju

l

20-ju

l03-aug

17-aug

31-aug

KLO

C

EsLmated Actual Milestone Trend ModuleSize

0 5 10 15 20ModuleAModuleBModuleC

KLOC

CurrentesLmate Actual

MilestoneTracking

19-may 29-may 08-jun 18-jun 28-jun 08-julMST1PMST1AMST2PMST2A

Mile

ston

esa

ctua

l/pl

anne

d

MilestoneStatus

CompletedonLme20%

CompletedLate10%

Notyetcompleted

70%

CostandSchedulePerformance

-40-35-30-25-20-15-10-505

10

Apr-08

Jul-08

Oct-08

Jan-09

Apr-09

Jul-09

Oct-09

Jan-10

Apr-10

Jul-10

Oct-10

Perc

entv

aria

nce SchedulePerformanceVariance

CostPerformanceVariance

Cri-calRa-o

0,400,600,801,001,201,40

Apr-08

Jul-08

Oct-08

Jan-09

Apr-09

Jul-09

Oct-09

Jan-10

Apr-10

Jul-10

Oct-10

CriLcalRaLoThreshold(red)Threshold(yellow)




•  Issuesinsolwareprojectsgenerallyrequireuseofcommoncoremeasurestohandle

o  requirementsgrowthandstability

o  monitoringofproductsize

o  managementofcostandefforttobudget

o  scheduleandotherprogressindicators

o  monitoringofdefectlevels,toseetrends

•  Derivedmeasuresprovideinsightwithcombina-onsofbasemeasures




Address: HenrySchneiderPresident/SeniorPrincipalConsultant

ProcessandProductQualityConsul-ng,LLC2009SimondAvenue,SuiteBAus-n,TX78723

Phone: 832-628-2486E-mail: [email protected]: www.ppqc.net

TSP SYMPOSIUM Schneider, Henry.pptx

Documents

Transcript of TSP SYMPOSIUM Schneider, Henry.pptx