® IBM Software Group © 2007 IBM Corporation Measured Capability Improvement Framework: A...

®

IBM Software Group

© 2007 IBM Corporation

Measured Capability Improvement Framework:A Systematic Approach to Software Excellence

Per KrollChief ArchitectIBM Rational Expertise Development & Innovation (REDI)[email protected]

IBM Software Group | Rational software

2

Discussion Topics

What is Measured Capability Improvement Framework

Value-Based Selling and Health Assessment

Ongoing self assessment with Self-Check

Practices and IBM Rational Method Composer

Performance Measurements and

Value Traceability Trees


3

The Challenge

How do you improve productivity to reduce cost?

How do you reduce time-to-value?

How do you increase quality?

How do you increase likelihood of compliance?

How, even, should Software and Systems Delivery teams be measured?

How do you optimize business value from your software and system delivery organization?


4

What Is Measured Capability Improvement Framework (MCIF)

A systematic approach for improving your business using four phases: Elicit desired business results

Determine the right set of products/services/usage models

Accelerate and monitor the adoption of the technology

Measure the business results realized

Captures +10 years of Rational experiences in incremental adoption Key aspects have been used in agile transformation’s of +80 IBM internal

projects

Process independent – used in conjunction with RUP, XP, Scrum and other processes


5

Going to the doctor…

What you want to know

How long will I live?

What the doctor does

A health assessment

What the doctor will measure

Heart rate, cholesterol, blood pressure, body fat, …

There is a correlation between the values of what a doctor measure and your life expectancy

But you can outlive somebody with lower cholesterol than you have…

Experiences shows that there is a similar correlationbetween software best practices and attainment of desired business results


6

Going to a personal trainer

Business value is maximized through “personalized” software process improvement plans that are continually adapted based on measurable results

What you want to do Run a marathon on a personal best time

What the trainer will do A fitness assessment to understand your

strengths and weaknesses

Personalized plan is produced basedon where you are and where you want to be Run sprints / intervals, run medium distances, strength training, eat healthy, …

Plan will be adjusted based on progress in each area Ongoing monitoring

Your plan will be personalized and continuously adapted, but based on known patterns of success…


7

Example: Financial Service Company

Satisfy compliance mandate

Enable flexible/global resourcing

Improve oversight

Improve consistency/predictability

Increase innovation

Improve productivity

Reduce time-to-market

Operational Objectives

Recent SOX audit failure

Inconsistencies with integrated financial reporting

Functionality of customer web falling behind competition

Create financial products more quickly

Customer Business Challenges

Measured capability improvement frameworkMap business value to software delivery practices

Outside scope

Target: Phase 1

Already implemented

Software Delivery Best Practices

Shared vision

Use-case driven development

Whole team

Continuous integration

Iterative development

SOA modeling

…Structured testing

Test management

Functional testing

Security testing

Architecture modeling

SOA governance

Asset-based development

Test-driven development

2-level project planning

Risk-value lifecycle

Asset governance

Enterprise SOAAdopt Practice

80.46E

100.39D

61.618C

41.414B

72.322A

Innovation(Cust. Sat. )

Quality (Defect

Density)

Time to Market (M)Project

Business Metrics Ongoing Adoption Assessment


8

MCIF: Cohesive service offerings, assets, and products

Phase 1: Elicit desired business results Executive Business Value Workshop: Service offering - explore objectives and strategies

Phase 2: Determine the right products/services/usage models Health Assessment for Software Delivery: Service offering - Understand where you are at

today, and establish a roadmap forward

Business Value Assessment Tool: ROI analysis

Phase 3: Accelerate and monitor the adoption of the technology IBM Rational Self Check for Software Teams: Service offering - Facilitated self

assessment

Rational Method Composer 7.5: Guidance on practices and effective usage of products

Tiered Performance Measurement using Rational Insight: Business, operational, and practice-level metrics (Rational Insight)

Phase 4: Measure the business results realized Tiered Performance Measurement using Rational Insight: Business, operational, and

practice-level metrics (Rational Insight, point products, …)

In beta!


9

Discussion Topics








10

MCIF: Value Selling and Incremental Value RealizationChanging the Way We Interact with Customers

UnderstandingNeeds

Delivery

Products and Services

Business Value

Traceability

Realization

Health

Assessm

ent

Self

Check

Metrics

RoadmapPracticesToolsServices

Practices

Busines

s M

etrics Business drivers

Metrics

ROI KPIs

Establish R

OI

Business drivers

RoadmapPracticesToolsServices

Value delivered

Are we on track?


11

Service Offering: Health Assessment for Software Delivery

Identify drivers for delivery improvements

Current state: Organization & Collaboration, Oversight, Software Delivery Practices, Skills, Development Environment, and Metrics

Future state: An initial roadmap outlining how you can evolve your delivery approach to address your key business drivers

List of inhibitors that may prevent successful implementation of roadmap

Identify metrics to track progress on the implementation of your roadmap


12

Discussion Topics








13

Service Offering: IBM Rational Self Check for Software Teams

Systemic approach for teams to assess their adoption of desired practices

Enables teams to learn, improve their effectiveness, and share their learnings

I’m a developer.

I want to learn and remember practices.

I want new ideas to stick.

I don’t want to be beaten by a stick.

I’m a developer.

I want to learn and remember practices.

I want new ideas to stick.

I don’t want to be beaten by a stick. I’m a coach. I want efficient continuous improvement. I want to hear from quiet people on the team.

I want to learn from teams like mine.

I’m a coach. I want efficient continuous improvement. I want to hear from quiet people on the team.

I want to learn from teams like mine.

I’m an executive.

I want to understand if my teams are improving.

I want to involve team members in the change effort

I’m an executive.

I want to understand if my teams are improving.

I want to involve team members in the change effort


14

0 2 4 6 8 10

Reflections

Non Solo

Iterative

Scrum meeting

Automated Unit Tests

Use Cases

Vision

Evol. Architecture

Big Picture of Practice Adoption

Deep Dive on Practice

IBM Rational Self Check for Software Teams

Practice Comments Average Deviation

Use Cases Use Stories instead 7.8 0.50

Unit Test Not enough, not in build 6.8 1.89

Stakeholder Feedback Want more 8.5 0.58

Time-Boxed Iterations Not well specified 8.3 0.50

Daily Scrum Not Fridays 9.3 0.50

Test Early Not enough TDD 4.5 1.91

Reflections Need clear actions 8.8 0.50

Static Analysis Not automated yet. 3.3 0.96

The Assessment Tool

Experience Report


15

Quick example: The iterative questions

Target QuestionResults Section

Average Deviation Talk?

Time-boxed Iterations

Do you hold your iteration end dates fixed, and adjust content of that iteration if needed? 3.0 2.000 Talk

Working Increment

Does each iteration (except perhaps the earliest ones) result in a stable executable release (internal or external), with code that you can demonstrate?

1.3 0.577 Talk

Feedback Used

Do you use feedback from key stakeholders such as sponsors, partners, users to adjust the content of the next iterations? 3.3 0.577

Estimation /Velocity

Do you involve the entire team in estimation. Do you re-plan your work for each iteration based on your previous "Velocity" (how much work got done in previous iterations)? Do you update overall plan and stakeholder expetations based upon actual progress.

4.0 1.000

Prioritized Backlog

Do you select content for your next iteration from prioritize set of work items (including functionality and defects)? 4.7 0.577

Just-In-Time Detailed Plan

Do you detail the plan for the next iteration at the end of the current iteration? 3.1 1.153

Retrospective

During iteration assessment, do you calibrate progress made with project goals by discussing what worked well, what didn't work well, and how to improve? Do you improve planning for next iteration and update long-range plan accordingly? Do you use feedback, including test results to improve your process?

2.0 1.732 Talk

Micro-Increments

For each iteration, do you define measurable tasks for iteration objectives, and are these tasks small enough to be performed by one or a few people?

3.5 0.503


16

0 2 4 6 8 10

Custom

Non Solo

Reflections

Scrum meetings

Iterative

Automated Unit Tests

Use Cases

Vision

0

3

7

10

Time boxediterations

WorkingSoftware

Feedback Used

Estimating

PrioritizedBacklog

Scrum meetings

Example: Are we iterative?

235 people. Multi-shore. Java8 week “iterations”. 24 Month project. New to agile.

Iterations have enabled them to realize the plan was severely overbookedand prioritization of content began early.

The Big Picture We’re iterative, right?


17

Discussion Topics








18

Practices included as part of Rational Method Composer 7.5


19

Understand how Rational tools supports the practice, and how to measure your adoption. Have

you done a great job yet implementing the practice?

Understand how Rational tools supports the practice, and how to measure your adoption. Have

you done a great job yet implementing the practice?

Each practice is clearly called out, with information Why should you use this practice?How do you adopt this practice?

How do learn about this practice? (Courses, books, etc)

Each practice is clearly called out, with information Why should you use this practice?How do you adopt this practice?

How do learn about this practice? (Courses, books, etc)

Find enablement material and sign up for courses, etcFind enablement material and sign up for courses, etc


20

Discussion Topics








21

3-Tiered Performance Measurement

Business Level Metrics Usage: Provide specific information that has impact on the business mission

Audience: CxO

Example: Revenue, Time-to-ROI, KPIs (specific to LOB)

Operational Level Metrics Usage: Measure and manage improvements to development performance

Audience: VP of development, Development Mgmt., Operation Mgmt.

Example: Productivity measurements, Quality measurements, Time-to-Value measurements

Practice Level Metrics Usage: 1) Demonstrate the level of adoption, 2) Track and monitor progress

Audience: Project manager, team lead and team members

Example: Iteration velocity, project backlog, defect density, growth chart, SPI, CPI


22

MCIF: A Performance Improvement System

Operational Level

Practice Level

Business Level

Process Enactment / Governance Enforcement / Process Awareness(Jazz Platform)

<Or other non-Jazz-based products>

Performance Measurement

Control Metrics(Subjective)(Self-Check)

Control Metrics(Objective)

(Rational Insight)

Value Metrics(Objective)

(Rational Insight / Cognos)

Operational Effectiveness Metrics(Objective)

(Rational Insight)

Business Objectives

Process Definition / Practices(RMC)

Operational Objectives

Feedback

Feedback

Feedback

Aut

omat

ic C

olle

ctio

n

Other Data Sources


23

Quality Value Traceability Tree: Decision Criteria

Increase Defect Prevention

Increase Defect Detection

Deliver on Customer Requirements

Improve Non-functional Quality Attributes

Development Quality Goals

High # of defects (pre/post-ship)

High maintenance costs (devt) of fixing defects Customer downtime

High support or maintenance costs without

high defect #s Low pipeline conversion

Low customer satisfaction High Service Level

Agreement

High defects both pre- and

post-ship Growing

defect backlog

High post-ship or

customer-reported defects

High fixes in error

High Requirements

Churn High Post

Delivery Support Aging

Enhancement Backlog

Occurring of Non-functional Req. Issues *

High # of Help Desk Calls

High Incidence Resolution

Times

Reduce Post-delivery Defects Deliver What Stakeholders Need


24

Example: Quality Measurements and Practices

Increase Defect Prevention

Increase Defect Detection

Deliver on Customer Requirements

Improve Non-functional Quality Attributes

Measures: Defect density Defect arrival/closure rates Defect backlog Fixes failing verification Rework effort

Practices:Test-driven Dev.Design-driven Implem.C&C managementSystem Component Arch.Whole teamPair ProgrammingReview/Inspection

Measures: Defect density, distribution Defect arrival/closure rates Defect removal effectiv. Fixes failing verification Test coverage Test execution status

Practices:Test managementContinuous integrationEvolutionary ArchitectureComponent ArchitectureTest-driven dev.Test practices Iterative Dev.Risk Value LifecycleC&C ManagementReview/Inspection

Measures: Post-ship problem reports Customer satisfaction Pipeline conversion? Support / maint. costs Requirem. test coverage Requirements delivery Survey of feature usage

Practices:Shared VisionUse-case Driven DevRequirements Mgnt. Whole TeamIterative Dev.Functional TestingC&C ManagementReview/Inspection

Measures: Post-ship problem reports Customer satisfaction Support / maint. costs Requirement test coverage Test execution results

Practices:Application Vulnerability Assessment Performance Testing Requirements Mgnt.Shared Vision Risk-Value LifecycleEvolutionary ArchitectureTest-Driven DevelopmentIterative DevelopmentEvolutionary DesignComponent ArchitectureContinuous IntegrationConcurrent TestingWhole TeamReview/Inspection


25

Objectives RTC Reports

Tracking iteration health and iteration progress

Iteration burndown / Sprint burndown

Story points / use case point

Blocking work items

Iteration planned items (iteration objective) report byPriority (high, medium, low)

Status (new, in-progress, resolved, etc..)

Deferred work itemsShows number of work items which are deferred

from previous iteration

Estimation accuracy Team velocity

Incrementally producing executable and testable software

System Growth

Iteration code coverage

Iteration code health (number of test failures)

Amount of test passing per iteration

Measuring Practice Adoption objectively:Example: Iterative and RTC


26

Development Quality Goals

Reduce post-delivery defects

Deliver What Stakeholders Need

Summary View of Value Traceability Trees

Development Productivity

DeliverFaster

Deliver Smarter

OptimizeResources

Align Operations& Development

Development TTV

Effective Project Management

Smart Reuse

Development Innovation

Collaborate Effectively

Smart Reuse Risk Acceptance

Trade-off

Early Issues Prevention / Detection

Collaborate Effectively

Identical

Identical

Scope Management

Scope ManagementReduce Rework

Ens

ure

Cor

rect

ness

Ensure Correctness

Deliver the right system


27

Where to Go for More Information

IBM.COM http://www.ibm.com/software/rational/announce/mcif/

MCIF Whitepaperhttp://www.ibm.com/common/ssi/fcgi-bin/ssialias?infotype=SA&subtype=WH&appname=SWGE_RA_RA_USEN&htmlfid=RAW14075USEN&attachment=RAW14075USEN.PDF

Self-Check article on developerWorks http://www.ibm.com/developerworks/rational/library/edge/08/may08/kroll_krebs/index.html


28

© Copyright IBM Corporation 2008. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, the on-demand business logo, Rational, the Rational logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.

Learn more at: IBM Rational software IBM Rational Software Delivery Platform Process and portfolio management Change and release management Quality management Architecture management

Rational trial downloads Leading Innovation Web site developerWorks Rational IBM Rational TV IBM Rational Business Partners

THANKYOU

® IBM Software Group © 2007 IBM Corporation Measured Capability Improvement Framework: A...

Documents

Transcript of ® IBM Software Group © 2007 IBM Corporation Measured Capability Improvement Framework: A...