10-stramaim-Estratégias de Teste Baseadas em Mutação
-
Upload
vinicius-vieira-pessoni -
Category
Documents
-
view
220 -
download
0
Transcript of 10-stramaim-Estratégias de Teste Baseadas em Mutação
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
1/33
Unit and Integration Testing
Strategies for C Programs UsingMutation-Based Criteria
Auri Marcelo Rizzo Vincenzi
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
2/33
Mutation 2000 Symposium
San Jose CA October, 2000 2
Agenda
Introduction
Mutation testing
Experiment escription !esting Strategies
Score"Cost Incrementa# !esting Strategy
Su$$icient Incrementa# !esting Strategy Conc#usion and %uture &or'
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
3/33
Mutation 2000 Symposium
San Jose CA October, 2000 (
Introduction
!est Acti)ities
*#anning
!est Case esign
!est Case Execution
+esu#t Ana#ysis
!esting Strategies
Incrementa# strategy
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
4/33
Mutation 2000 Symposium
San Jose CA October, 2000
Introduction
!est esign !ec-ni.ues
%unctiona# /b#ac' box testing
Structura# /1-ite box testing
Error based
Comp#ementary tec-ni.ues
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
5/33
Mutation 2000 Symposium
San Jose CA October, 2000
Introduction
!esting *-ases
3nit testing
Integration testing
System testing
How we should combine different testing criteria,
covering unit and integration testing phases, using
a low cost and effective strategy?
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
6/33
Mutation 2000 Symposium
San Jose CA October, 2000 4
Mutation !esting
Mutation Ana#ysis /MA
3nit testing
Proteumtoo# $or C programs
56 mutant operators
Statement /6
Operator /4
7ariab#e /5
Constant /(
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
7/33
Mutation 2000 Symposium
San Jose CA October, 2000 5
Mutation !esting
Inter$ace Mutation /IM
App#y t-e same concepts o$ mutation testing at t-e
interprocedura# #e)e#
Integration Errors
SI(B)
incorrect
A
Incorrect
outputB
SI(B)
incorreto
SO(B)incorrect
A Incorrect
output
B
SO(B)incorrect
f Incorrect
output
g
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
8/33
Mutation 2000 Symposium
San Jose CA October, 2000 8
Mutation !esting
Inter$ace Mutation
PROTEUM/IMtoo# $or C programs
(( mutant operators
9roup"I /2
: Mutations inside t-e body o$ $unctionB /inter$ace
commands, g#oba# and inter$ace )ariab#es
9roup"II /;
: Mutations 1-ereAca##sB
B
A Group-II
Group-I
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
9/33
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
10/33
Mutation 2000 Symposium
San Jose CA October, 2000 60
Mutation !esting
*rob#ems 1it- Mutation !esting
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
11/33
Mutation 2000 Symposium
San Jose CA October, 2000 66
Experiment escription
Steps
*rogram Se#ection
!oo# Se#ection
!est Set 9eneration
Strategies
+esu#ts and ata Ana#ysis
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
12/33
Mutation 2000 Symposium
San Jose CA October, 2000 62
Experiment escription
*rogram Se#ection
!oo# Se#ectionProteum/Mutation Ana#ysis unit testing
PROTEUM/IM/Inter$ace Mutation integration testing
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
13/33
Mutation 2000 Symposium
San Jose CA October, 2000 6(
Experiment escription
!est Set 9eneration
!1o poo#s o$ test cases $or eac- program
3nit testing
Integration testing Eac- poo# is composed by
ad hoc test cases based on t-e program speci$ication
+andom test cases ta'e $rom &ong>s experiments
!est cases generated by -and
Ana#y=ing t-e #i)e mutants and determining t-e e.ui)a#ent ones
!-e entire poo# is an ade.uate test set 1?r?t? MA or IM
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
14/33
Mutation 2000 Symposium
San Jose CA October, 2000 6
Experiment escription
!est Set 9eneration
%rom eac- poo#, e#e)en ade.uate test set 1as
generated $or eac- program and eac- criteria
@o minimi=ation tec-ni.ue -as been app#ied
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
15/33
Mutation 2000 Symposium
San Jose CA October, 2000 6
Experiment escription
Strategies
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
16/33
Mutation 2000 Symposium
San Jose CA October, 2000 64
Experiment escription
Intra"p-ases !esting Strategies
Score"Cost !esting Strategy
App#ied $or eac- criterion indi)idua##y
integration #e)e#
unit #e)e#
!1o .uestions
How to select a subset of operators keeping the mutation score
as high as possible without reducing the overall effectiveness?How to achieve a mutation score of 1., with a minimal cost?
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
17/33
Mutation 2000 Symposium
San Jose CA October, 2000 65
Experiment escription
Su$$icient !esting Strategy
App#ied $or eac- criterion indi)idua##y
unit #e)e#
integration #e)e#
Inter"p-ase !esting Strategies
Combine bot- criteria, co)ering unit and
integration testing p-ases
3nit"Integration !esting Strategy
Su$$icient 3nit"Integration !esting Strategy
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
18/33
Mutation 2000 Symposium
San Jose CA October, 2000 68
Experiment escription
Score"Cost !esting Strategy
Estab#is- an incrementa# strategy o$ app#ication
E)a#uate t-e e$$ecti)eness o$ eac- operator 1?r?t? t-e $u##
set o$ operators
Ca#cu#ate t-e ratio bet1een t-e mutation score and t-e
cost o$ eac- operator
ata co##ected $or one program IM
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
19/33
Mutation 2000 Symposium
San Jose CA October, 2000 6;
Experiment escription
Score"Cost !esting Strategy
A)erage data $or t-e programs IM
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
20/33
Mutation 2000 Symposium
San Jose CA October, 2000 20
Experiment escription
Integration !esting
Score"Cost Inter$ace !esting Strategy /!"#!!"#!Strategy %irst Interaction
(( operators IM"ade.uate test set
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
21/33
Mutation 2000 Symposium
San Jose CA October, 2000 26
Experiment escription
!"#!!esting Strategy %ina# Step
2 operators
IM"ade.uate test set/54B o$ t-e e.ui)a#ent mutants
60 operatorsmutation score 0?;;
/6(B o$ t-e e.ui)a#ent mutants
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
22/33
Mutation 2000 Symposium
San Jose CA October, 2000 22
Experiment escription
3nit !estingScore"Cost 3nit !esting Strategy /!"$!
(6 operators MA"ade.uate test set
/85B o$ t-e e.ui)a#ent mutants
65 operatorsmutation score 0?;;/2?2B o$ t-e e.ui)a#ent mutants
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
23/33
Mutation 2000 Symposium
San Jose CA October, 2000 2(
Experiment escription
Su$$icient !esting Strategy
MA"Su$$icient Operator Set
SM!C, SS, OEDA, O++@, 7!& and 7!+
MA"Su$$icient MA
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
24/33
Mutation 2000 Symposium
San Jose CA October, 2000 2
Experiment escription
3nit !estingSu$$icient 3nit !esting Strategy /!$!
20 operatorsMA"ade.uate test set
/58B o$ t-e e.ui)a#ent mutants
operatorsmutation score o$ 0?;;
/65B o$ t-e e.ui)a#ent mutants
Comparing wit SCUSStrateg!
MA"ade.uate test C+ (2B and E. 85B Mutation Score o$ 0?;; C+ ;(B and E. 2?2B
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
25/33
Mutation 2000 Symposium
San Jose CA October, 2000 2
Experiment escription
Su$$icient !esting Strategy
IM"Su$$icient Operator Set
ArgAri@eg, Co)A##@od, ir7arDit@eg, Ind7arDit@eg,
Ind7ar+ep9#o, Ind7ar+epExt and Ind7ar+ep+e.
IM"Su$$icient IM
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
26/33
Mutation 2000 Symposium
San Jose CA October, 2000 24
Experiment escription
Integration !estingSu$$icient Inter$ace !esting Strategy /!#!
20 operatorsIM"ade.uate test set/5B o$ t-e e.ui)a#ent mutants
4 operators mutation score o$ 0?;;
/62B o$ t-e e.ui)a#ent mutants
Comparing wit SCISStrateg!
IM"ade.uate test C+ 2B and
E. 54B Mutation Score o$ 0?;; C+ 85B
and E. 6(B
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
27/33
Mutation 2000 Symposium
San Jose CA October, 2000 25
Experiment escription
3nit"Inter$ace !esting Strategy /$%#!
MA and IM are incomparab#e 1?r?t? subsume
re#ation
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
28/33
Mutation 2000 Symposium
San Jose CA October, 2000 28
Experiment escription
3nit"Inter$ace !esting Strategy /$%#!
65 operatorsIM"ade.uate test set/5B o$ t-e e.ui)a#ent mutants
operators mutation score o$ 0?;;
/?B o$ t-e e.ui)a#ent mutants
"#erall results
$%#!generates 65,552 /C+ (B 5,42 unit #e)e# 60,6(0 interation #e)e#
$%#!produces MA and IM"ade.uate
test sets
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
29/33
Mutation 2000 Symposium
San Jose CA October, 2000 2;
Experiment escription
Su$$icient 3nit"Inter$ace !esting Strategy /!$%#!
"#erall results
!$%#!generates 4,040
/C+ 55B 2,(0 unit #e)e# (,54 interation #e)e#
$%#!produces test cases sets
t-at detemines mutation scoreo)er 0?;; $or bot- MA and IM
criteria
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
30/33
Mutation 2000 Symposium
San Jose CA October, 2000 (0
Experiment escription
!esting Strategies
Summary o$ t-e +esu#ts
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
31/33
Mutation 2000 Symposium
San Jose CA October, 2000 (6
Conc#usion
MA and IM are comp#ementary criteria
!"#!, !#!, !"$!and !$!a##o1 t-e tester using t-e
mutant operators according 1it- t-e budget
restrictions and 1it- t-e mutation score t-at isexpected to be obtained
$%#!co)ers t-e unit and integration testing p-ases
and produces AM and IM"ade.uate test sets!$%#!co)ers a#so t-e unit and integration testing
p-ases and a mutation score o)er 0?;; is obtained
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
32/33
Mutation 2000 Symposium
San Jose CA October, 2000 (2
%uture &or'
%urt-er studies to in)estigate t-e sca#abi#ity o$ t-ese
resu#ts to #arge programs and ot-er app#ication
domains
In)estigate t-e e$$ecti)eness o$ t-ese strategies Impro)e t-e 1ay to se#ect t-e operators considering
t-e cost 1?r?t? t-e number o$ e.ui)a#ent mutants and
t-e e$$ecti)eness o$ eac- operator
Estab#is- -euristics to dea# 1it- e.ui)a#ent mutants
-
8/14/2019 10-stramaim-Estratgias de Teste Baseadas em Mutao
33/33
Mutation 2000 Symposium
San Jose CA October 2000 ((
oubts, FuestionsGGG