Why don’t we practice what we

teach?

Andre Oboler, David McG. Squire and Kevin B. Korb

School of Computer Science and Software Engineering,

Monash University, Melbourne, Australia

Engineering Software for Computer Science Research in Academia.

6-25-03

Outline1. “Computer Science Research” and “Software Engineering”

– Is this research relevant to me?

2. Do researchers use software engineering?1. Current practice

2. Impact of current practice

3. Getting the right mix– RAISER / RESET a new SDLC to support the creation

and development of research software.

Use SE in CS Research?

6-25-03

• What is “Computer Science”?

• What is “Research Software”?

• What is “Software Engineering”?

Is Software Engineering used when creating Research Software in Computer Science (or similar) university departments?

Approach

• Triangulation of:– statistical analysis of survey results– Interviews and E-mail discussion with experts– observations from case studies

These methods investigated the use, costs and benefits of using Software Engineering in

Computer Science Research

Past work

• no prior work on the costs/benefits of software engineering for research software

• This trend was started by Royce (1970) when he suggested that small projects used only by the developer need only use a 2 step analysis / coding approach (rather than his waterfall SDLC)

• This view that research is too small to warrant software engineering is still prevalent.

• Research is still approached this way.

Application of SDLCs in research

Results

SDLC used in Australian Research

0

2

4

6

8

10

12

14

SDLC

Re

sp

on

se

s

Problems with a lack of Software Engineering

• A lack leads to a waste of new research students’ time

• Follow on research is harder to achieve (some valuable research may be shelved)

• Authenticity of results is harder to verify

• Shortens the useful life of the project

• Shorter projects have fewer benefits

Why don’t we practice what we

teach?

6-25-03

• The needs of research and researcher are different– Documentation:

• Main work is by current researchers• Main benefit is for future researchers

– Design / Code quality:• Main benefit is for future spin off research• Time pressures mean getting publications is more important

Because research has different needs to

industry

6-25-03

Why Software Engineering is not used

Reason Percent that agree with this reason

Never thought about it 14%

Don't know about them 11%

Cost of learning them is too high 17%

Not appropriate for my work 83%

Cost of use is higher than pay off 46%

Organisational Policy against spending time on them

3%

Considerations for an SDLC to meets the needs of research

“A system built as part of a Ph.D. project is intended to prove

feasibility, and it would almost always be a mistake to spend the

time and effort during initial development to build it to product-quality standards”

(Brooks, 2002).

“The primary aim [of research] is to get a flaky prototype working sufficiently to get a few statistics

out. There is absolutely zero [incentive] for producing a robust,

flexible, extendable piece of software” (Allison, 2002).

“The major problem is that research projects tend to be

opportunistic rather than planned” (Waite, 2002).

“The implication is that any SE approach for research software

would have to be agile and evolutionary in nature”

(Pressman, 2002).

On User Documentation for the CDMS Case Study

“We're not sure how this will happen. We were sort of hoping it

would happen by magic or be delivered by a stork”

(Allison, 2002).

The RAISER / RESET idea

• Separate research activities from stabilization• Limit the negative impact during research phases• “Clean” up code so it is ready for the next

researcher to continue working on• While many researchers do not use software

engineering, those that do use it predict they will use more in the future, this is potentially as harmful as the current lack of application.

The SDLC Model

RAISER (for Research)

R eactive

A ssisted

I nformation

S cience

E nabled

R esearch

Minimum overhead, maximum benefit now

• High level design – before coding

• Use header blocks• Configuration Management• Paired Programming

– with other researchers

RESET(between Research)

R esearch

E nabled

S oftware

E ngineering

T echniques

Clean up and restructure for later

• Design and Code reviews

• Restructure for: – improved modularity

– ease of reuse

• Review API / User interface– Improve and document

• Create design documents

• Record current and future functionality

Implementation

The in-house software development lab

Questions?

NB: Future work will be undertaken in this area over the next three years, feedback is most welcome!