Software Engineering Practice - Configuration management

McGill University ECSE 428 © 2003 Radu NegulescuSoftware Engineering Practice Configuration management—Slide 1

Configuration management

McGill ECSE 428Software Engineering Practice

Radu Negulescu

Winter 2003


About this course module

A typical software project may generate thousands of source files and pieces of documentation, and there are many opportunities for introducing inconsistencies between these artifacts:

• Teamwork, parallel development and QA

• Change requests

• Bug fixes

• Schedule pressures

Here we discuss how to effectively organize the artifacts of a software project.

Recommended reading:

• Jalote ch. 10

• Bruegge & Dutoit 10.2,10.3,10.4.1,10.4.3,10.4.4

• McConnell Rapid Dev. pp. 338-341, 403-414


What is configuration management

B&D: The disciplines and techniques of initiating, evaluating and controlling change to software products during and after development.

• In other words, staying on top of change

Known by many names:

• CM

• SCM - sometimes pronounced “scum”

• Version control

• Source control (especially in relation to code)

• Document control


Development problems addressed

Typical problems of software projects due to lack of tracking:

• Communication blocks

• InconsistenciesAccidental deletesWrong releasesOverlapping updates

• Lost issues (and rework)Lost bug reportsLost reasons for change

• Poor predictabilityAbsence of baseline data

• Awkward integration

• High cost of human errorDifficult “undo”


Challenges of configuration management

Challenges:

• Number of change requests (100s, 1000s, ...)

• Rapid growth of configurations: builds components variants

• Superlinear growth of team coordination overheadNumber of communication paths required

• Dangers of bureaucracy

• ...


Configurations

Item: artifact under configuration management

• Possibly nested aggregate

• Possibly overlapping with other aggregates

Configuration: state of an item

Variants: configurations of the same item that are intended to coexist

Beware:

• Terminology varies, although needs and solutions are similar

• Focus on the concepts


Configurations

Version: augmented specification

• Additional features

• Major or minor, depending on the amount of features

Release: same spec, enhanced implementation

• Bug fixes

• Refactored architecture

Build: instance of a system, integrated

• For testing

• Release candidatesShown to customer for evaluationBecome a release once accepted


Typical CM functions

Retrieve a configuration / undo a change

• E.g. what was our best build?

• … before we messed up the event model?

Life cycle control

• Static access restrictionsE.g. a code module can only be signed out by the “owner” (Dev.)E.g. a bug report can only be put in “closed” state by QAE.g. each new feature must be impact analyzed before it is scheduled for developmentE.g. only PL and CC create releases

• Dynamic access restrictionsE.g. no simultaneous updates


Typical CM functions

Automatic builds

• Invoke the compiler tools on the updated files only

Concurrent updates

• Branching then merging

• Reconciliation of changes

Tracing rationale

• E.g. why did we need a new event model anyway?

• Who issued a request for change?

Propagating change

• E.g. bug fix for deployed version then new version

• E.g. multiple platforms, languages, user privileges

Statistics

• E.g. determine percentage of testing vs. development delays


Retrieving a configuration

What we want from a mechanism for retrieving configurations:

• Label configurations in a way that orients the users

• Store configurations efficientlyThere can be many of them...

• Be tailored to the actual flow of configurations


How configurations are generated

Three main sources of change

• Shifting requirementsRework previous code

• Bug fixesBatches from QA sessionsContinuous flow from debugging

• Internal issues, such as: New vendor/new technologyPerformance tuningRefactoring

Not synchronized to the main development process

• Need for change may occur at any time

• Thus, you need some buffering

Reqflow

Issueflow

Bugflow

Develop.process

Cust

om

er

QA


Labeling of configurations

File names are vital

• Orient the developers and customers

• Gets interesting when dealing with 100s of files… or even 10s of files

Releases

• 2.5: version 2, release 5

• 2.3.5: version 2.3, release 5

Builds

• RN-2002-03-08-16:01:53

• Internal use onlyNot in the release file nameCC keeps record of release number


Example: JAR files

Distinguish release candidates sent to customer

• E.g.: do not send multiple System.jar files to customerBetter: FlatCart-Release4.3-…

Include the purpose of the file in an envelope nameE.g.: FlatCart-Release4.3-ForAcceptanceCandidate2.zip

Can use normal release names in the contained filesE.g.: FlatCart4.3.jar (JAR file)E.g.: FlatCart4.3.exe (self-extracting archive)

In Java 2, JAR files can be annotated with version info

• Version (“specification-version” attribute)

• Release or build (“package-version” attribute)

What if the system includes multiple components with different version & release numbers?

• The system can have yet another version scheme


Example: branch labeling

[After Bruegge & Dutoit]

revised

revised

Main

revised

Branch derived

merged

released

MUE.1.1:Releas

MUE.1.2:Releas

MUE.1.3:Releas MUE.1.2.1.1:Releas

MUE.1.2.1.2:Releas

MUE.2.0:Releas


Storage of configurations

Basic operations supported

• Get latest configuration: most frequent

• Insert a new configuration: frequent

• Get previous configurations: on a need-to basisRollback (undo changes)Locate insertion of a defect


Storage of configurations

Reverse delta

• Complete “baseline” version: latest

• DeltasSee diff tool

• Retrieval: compute previous versions from deltas

• E.g. VSS

Storing branches and variants

• A new base can be created for each branch on splitting from trunkNot many branches sharing won’t save much space

Storing aggregates

• Store cross-references only consistency, space (preferred)

• Store copies of all included components ready access


Item life cycles

Enforcing item life cycles

• Proper state transitions

• E.g. program modulesThe system is ready for acceptance testing when all modules are OK(Next page)

• E.g. triage of new feature requestsImpact analysis: which modules will be affected?Prioritization, cost estimationDevelopment, testingAcceptance


Item life cycles

Passed Module 1 Private

Ready for unit test

Baselined / released

Done

Failed

Passed Module 2 Private

Ready for unit test

Done

Failed

...

System/acceptance testing

Passed

Failed


Life cycle control

Objectives

• Facilitate access by developers

• Permit collection of management stats

• Allow easy traceability

Main mechanisms

• Directories

• Item database

• Document logs


Directories

Storage

• One directory for each module state

• State transitions: move among directories

• May use read/write permissions for directories

• Use CM tools for access restrictions in common areas

• Example: Infosys’ WAR project

Enforcement

• Directory read/write privileges


Item database

Storage

• State attributes for each item

• Some tools restrict which users may set which attributes

Enforcement

• Attribute settings


Document logs

Storage

• Use module headers

• Insert date of change, author

• Specify distribution list

Enforcement

• Self-discipline

• Configuration auditsA reason to maintain redundant info...


Dynamic rights

Mutual exclusion of updates

• Check-in/check-out

• Except for deliberate branching/merging


Example: WinCVS

www.wincvs.org

Work areas

• Repository

• Working directory

Check out

• CVS admin → Checkout module

• Specify module and working directory

Check in

• Update the working directory

• Commit to the repository

Related tools

• On-line CVS: www.freepository.org

• CVS

Workspace 1Workspace 2

Master directory

Software repository

Check-in Check-out

Baseline


Example: WinCVS


Example: document control

[After Jalote]

States = directories

• Under development← Initial stateAuthor: Done → Under review

• Under reviewReviewer: Defects found → Under developmentReviewer: No defects → Baselined

• BaselinedCCB: CR authorized → Under development

Under dev Underreview Baselined

[author]

[reviewer]

[reviewer]


Example: document control

Distribution listControlled

ProprietaryLimitedUnlimited

UncontrolledTo be destroyed


Automatic builds

Builds are frequent

• Some processes support daily buildsE.g. McConnell ch. 18Highly effective schedule reduction

• Other processes support continuous refactoringXP compensates de-emphasis of global design

Compilation can take a lot of time

• Idea: recompile only the updates

• Careful not to compile the wrong ones

Redundancy is hard to maintain

• Executables are easily created⇒ executables are not normally controlled

• Hence the term “source control”


Example: the make utility

Make

• A tool that controls generation of executables

• Handles multiple source files

• Recompiles if executable is older than a source

• Looks up change dates

• Not tied to a particular language

• Can even be used to format documents

Makefile

• Tells make how to recompile

• A text file

• Two sections: variables and rules

• Variables come first, but are optional

More info (GNU make)http://www.gnu.org/software/make/make.htmlhttp://www.gnu.org/manual/make/html_mono/make.html


Makefile rules

E.g. [from GNU]edith : main.o kbd.o command.o display.o \

insert.o search.o files.o utils.o

cc -o edith main.o kbd.o command.o display.o \


main.o : main.c defs.h

cc -c main.c

kbd.o : kbd.c defs.h command.h

cc -c kbd.c

command.o : command.c defs.h command.h

cc -c command.c

display.o : display.c defs.h buffer.h

cc -c display.c

insert.o : insert.c defs.h buffer.h

cc -c insert.c

search.o : search.c defs.h buffer.h

cc -c search.c

files.o : files.c defs.h buffer.h command.h

cc -c files.c

utils.o : utils.c defs.h

cc -c utils.c

clean :

rm edith main.o kbd.o command.o display.o \



Makefile rules

Callmake Program

make

make clean

General formattarget: prerequisite1 prerequisite2 ...

<tab> command1

<tab> command2

...

And here's a very useful one:clean:rm *.o core ./tmp/*

Deletes all object files in the current directoryany core dumpall files in ./tmp

// Looks up a file named “makefile”


Makefile variables

Exampleobjects = main.o kbd.o command.o display.o \


edith : $(objects)

cc -o edith $(objects)

main.o : main.c defs.h

cc -c main.c

kbd.o : kbd.c defs.h command.h

cc -c kbd.c

command.o : command.c defs.h command.h

cc -c command.c

display.o : display.c defs.h buffer.h

cc -c display.c

insert.o : insert.c defs.h buffer.h

cc -c insert.c

search.o : search.c defs.h buffer.h

cc -c search.c

files.o : files.c defs.h buffer.h command.h

cc -c files.c

utils.o : utils.c defs.h

cc -c utils.c

clean : rm edith $(objects)


Concurrent updates

Branch: concurrent development path requiring independent configuration management.

Merge: reconcile different branches.

merging point

UI branch

main trunk

comms branch


Concurrent updates

Support for change reconciliation

• View changes side-by-sideDecide which changes to adoptMight be tedious for divergent versionsSupported for text files, not binary (e.g. VSS)

• Merge the most recent changesExample: Word for windows “Merge Documents”

Tools → Merge Documents → open documentAccept or reject changes(Tools→Track changes→ ...)


Example: feature interaction

Functionality partitioning

• First release: F1, F2

• Second release: F3, F4

F3 is easy

• M4, M5 can be a separate increment

• Versioning of M4, M5 independent of M1, M2, M3

F4 needs a separate CM branch

• Create branch: copy M2, M4, M5; create M6

Do not check out M2, M4, M5 at this timeOther developers keep working on M2, M4, M5

• Implement F4

• Reintegrate branch to main build (trunk)

Check out M2, M4, M5Use delegated permission

Merge changesRegression tests of F4

Check in M2, M4, M5Regression tests of F1, F2, F3(and F4 to be sure)

• If reintegration fails, one version is dumped…

M1 M2 M3 M4 M5 M6 F1 X X X F2 X X F3 X X F4 X X X X


Change propagation & traceability

Forward change requests to all related modules

Retrieve reason for each modification

• So the modification can be undone!

Traceability databases

• Propagate changes along dependency relationsE.g. among variantsE.g. from SRS to DD to code to TP

• Dependency matrix

• Dependency tree


Change propagation & traceability

Example: derived modules

• After the current version is deployedThe current version is in a maintenance phase.Work may begin on a new version.

• If multiple versions are supportedBug fixes may occur for any supported versionBug reports, changes, etc. must propagate to the other versions

Keep module derivation relationships across multiple supported system versions.


The CM plan

CM item structure

• Naming scheme

• Versioning scheme

Release

Baseline

Physical setting

Configuration control process

• Staff responsibilities

Configuration audits


CM item structure

Can be more or less elaborate

• Depends on size of project, usage of artifacts, and change flow

Same idea as in modular design

• Things that are likely to change together should be kept together

• Things that are unlikely to change together should be separate

• The hierarchy of storage should reflect the likelihood of change

Modularity

• Subsystems

Nesting

• E.g. per increment

Overlapping

• E.g. per increment & per artifact type


Example: client-server system

[From B&D]


Responsibilities

Example:

• Configuration controllerCreate executablesBaseline code and docs...

• DevelopersCheck out, check in at specific times


Configuration auditing

Danger: state related mistakes

• E.g. move a non-tested module into the release directory

Use several mechanism for life-cycle support

• E.g.: directory structure + “master table”

• E.g.: directory structure + revision lists in module headers

Status monitoring

• Consistency of the redundant state mechanismsE.g. directory placement same as master table entry

• Consistency of the revision list with the artifact life cycle

Audits

• Check whether proper process is being followed

• Check the baseline and release states


Change control board

The change control board (CCB)

• Analyze changes

• “Triage” changes

• Bundle changes

Representatives from each party

• Development, QA, user documentationArtifacts owners

• Customer support, marketing, mgmt

Danger: go overboard...


CASE example: the Rational Suite

The full Rational Enterprise Suite is installed in our lab.

Team integration tools

• RequisitePro: organize, prioritize, track, control requirements

• ClearQuest: manage every type of changeImplements artifact lifecycleTalks to a database

• SoDA: automated documentationExtracts info and puts it in a Word template


Development

• Rose: draw UML diagramsChecks consistencyImplements code frameworks from the diagram: it writes class and method declarations and some docs, and you fill in the body of the methods

• Purify: run-time errors and memory leaksNeeded big time in C/C++Theoretically, not needed in Java (JVM-managed garbage collection)Practically, Java programs can exhibit object pool overflows, JVM malfunctions, heap reallocations, etc. -> need memory leak tools

• Visual PureCoverage: expose testing gapsIt instruments the code to register execution

• Visual Quantify: Profiles the application to check performance bottlenecks



Quality assurance

• Robot: record and playback test scriptsPoint-and-click operations are saved into a scriptGraphical object captureVerification points: test only certain properties of certain objects. This allows changes in the UI design without changing all the test casesThe scripting language is called SQA-Basic, and it is very similar to VB except for a couple of statements. See if you can run SQA-Basic scripts in VB

• TestFactoryAutomatic test generation:“Best test”: a random run through the code that works out as much of it as possibleSource code coverage analysis: what code was exercised by the test cases



Integration among these tools

• Rational Administrator

Integration with other tools

• MS Project: activities can be linked to requirements in the RequisitePro database

• MS Office: many of the Rational tools use Access for a RDBMS, and Word for their documentation interface

• MS VisualStudio: VB, VC++, ...

• Java


Discussion

Thank you!

Any questions?

Software Engineering Practice - Configuration management

Technology

Transcript of Software Engineering Practice - Configuration management