Searching for Liveness Property Violations in Concurrent Systems with ACO
-
Upload
jfrchicanog -
Category
Technology
-
view
196 -
download
2
description
Transcript of Searching for Liveness Property Violations in Concurrent Systems with ACO
1 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Searching for Liveness
Property Violations in Concurrent Systems with ACO
Francisco Chicano
and Enrique Alba
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
2 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
• Concurrent software is difficult to test ...
• ... and it is in the heart of a lot of
critical systems
• Techniques for proving the correctness
of concurrent software are required•Model checking
→ fully automatic
• In the past the work using metaheuristics
focused on safety properties• In this work we focus on liveness
properties
MotivationMotivation
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
3 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Intersection Büchi
automaton
• Objective: Prove that model
M satisfies the property :
• HSF-SPIN: the property f is an LTL formula
s0
s4s7
s6
s2
s1
s8s9
s5
s3
s0
s1s2
s3
s4
s5
Model M
s1
s0 s2
qp∧!q
!p∨q
LTL formula ¬
f(never claim)
∩ =
Explicit State Model CheckingExplicit State MC State Explosion Heuristic MC
Safety & Liveness
SCCs
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
4 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Intersection Büchi
automaton
• Objective: Prove that model
M satisfies the property :
• HSF-SPIN: the property f is an LTL formula
s0
s4s7
s6
s2
s1
s8s9
s5
s3
s0
s1s2
s3
s4
s5
Model M
s1
s0 s2
qp∧!q
!p∨q
LTL formula ¬
f(never claim)
∩ =
Explicit State Model CheckingExplicit State MC State Explosion Heuristic MC
Safety & Liveness
SCCs
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
5 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Intersection Büchi
automaton
• Objective: Prove that model
M satisfies the property :
• HSF-SPIN: the property f is an LTL formula
s0
s4s7
s6
s2
s1
s8s9
s5
s3
s0
s1s2
s3
s4
s5
Model M
s1
s0 s2
qp∧!q
!p∨q
LTL formula ¬
f(never claim)
∩
Using Nested-DFS
=
Explicit State Model CheckingExplicit State MC State Explosion Heuristic MC
Safety & Liveness
SCCs
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
6 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
s0
s4s7
s6
s2
s1
s8s9
s5
s3
• Number of states very large
even for small models
• Example: Dining philosophers with n
philosophers
→ 3n
states20 philosophers → 1039 GB
for storing the states
•
Solutions: collapse compression, minimized automaton representation, bitstate hashing, partial order reduction, symmetry reduction
• Large models cannot be verified but errors can be found
Memory
State Explosion Problem
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Explicit State MC
State Explosion
Heuristic MC
Safety & Liveness
SCCs
7 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
• The search for errors can be directed by using heuristic information
• Different kinds of heuristic functions have been proposed in the past:
• Formula-based
heuristics
• Structural
heuristics
s0
s4
s7
s6
s2
s1
s8s9
s5
s32
0
3
5
1
2
40
76
Heuristic value
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Heuristic Model Checking
• Deadlock-detection
heuristics
• State-dependent
heuristics
Explicit State MC
State Explosion Heuristic MC
Safety & Liveness
SCCs
8 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Safety property
• Counterexample ≡
path to accepting state
• Graph exploration algorithms can be used: DFS
and BFS
Liveness
property
• Counterexample
≡
path to accepting cycle
• It is not possible to apply DFS or BFS
s0
s4s7
s6
s2
s1
s8s9
s5
s3
Safety and Liveness
Properties
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
s0
s4s7
s6
s2
s1
s8s9
s5
s3
Explicit State MC
State Explosion Heuristic MC
Safety & Liveness
SCCs
9 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Strongly Connected Components
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
a b
ce
df g
Explicit State MC
State Explosion Heuristic MC
Safety & Liveness
SCCs
10 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Strongly Connected Components
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Explicit State MC
State Explosion Heuristic MC
Safety & Liveness
SCCs
11 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Strongly Connected Components
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Explicit State MC
State Explosion Heuristic MC
Safety & Liveness
SCCs
12 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Strongly Connected Components
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Explicit State MC
State Explosion Heuristic MC
Safety & Liveness
SCCs
a b
ce
df gF-SCC
N-SCC
P-SCC
13 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
OPTIMIZATION/SEARCH TECHNIQUESOPTIMIZATION/SEARCH TECHNIQUES
EXACTEXACT APPROXIMATEDAPPROXIMATED
Ad Hoc
HeuristicsAd Hoc
Heuristics METAHEURISTICSMETAHEURISTICS
Optimization/Search
Techniques
• Newton• Gradient
Based on CalculusBased on Calculus
• Depth
First
Search• Branch
and
Bound
EnumerativesEnumeratives
• SA• VNS• TS
Trayectory-basedTrayectory-based
• EA• ACO• PSO
Population-basedPopulation-based
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Metaheuristics
ACO
ACOhg
ACOhg-live
14 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
•
Ant Colony Optimization (ACO) metaheuristic
is inspired by the foraging behaviour of real ants
• ACO Pseudo-code
ACO: Introduction
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Metaheuristics
ACO ACOhg
ACOhg-live
15 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
• The ant selects its next node stochastically
• The probability of selecting one node depends on the pheromone trail and theheuristic value (optional) of the edge/node
• The ant stops when a complete solution is built
i
j
l
m
k
Ni
τij
ηij
kTrail
Heuristic
ACO: Construction Phase
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Metaheuristics
ACO ACOhg
ACOhg-live
16 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
• Pheromone update
During the construction phase
After the construction phase
• Trail limits (particular of MMAS)
Pheromones are kept in the interval [τmin, τmax]
ACO: Pheromone Update
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
with
Metaheuristics
ACO ACOhg
ACOhg-live
with
17 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
The length of the ant paths is limited by λant λant Objective node
What if…?
Starting nodes for path construction change
After σs
steps
Second stage Third stage
Initial node
ACOhg: Huge Graphs Exploration
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Metaheuristics
ACO
ACOhg
ACOhg-live
18 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
• The search is an alternation of two phases
First phase: search for accepting states
Second phase: search for cycles from the accepting states
ACOhg-live
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Metaheuristics
ACO
ACOhg
ACOhg-live
ACOhg-live
Pseudocode
First
phase
19 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
• The search is an alternation of two phases
First phase: search for accepting states
Second phase: search for cycles from the accepting states
ACOhg-live
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Metaheuristics
ACO
ACOhg
ACOhg-live
ACOhg-live
Pseudocode
Second
phase
20 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
• The search is an alternation of two phases
First phase: search for accepting states
Second phase: search for cycles from the accepting states
ACOhg-live
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Metaheuristics
ACO
ACOhg
ACOhg-live
ACOhg-live
Pseudocode
Tabu
list
21 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
• The search is an alternation of two phases
First phase: search for accepting states
Second phase: search for cycles from the accepting states
ACOhg-live
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Metaheuristics
ACO
ACOhg
ACOhg-live
ACOhg-live
Pseudocode • Improvement using SCCs
First phase: accepting states in N-SCC are ignored
Both phases: cycle detected in F-SCC is a violation
Tabu
list
22 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
•We used 11 Promela
models
for the experiments
• Parameters for ACOhg-live
• Formula-based
and finite state machine
heuristics• ACOhg-live implemented in HSF-SPIN• 100
independent executions and statistical validation
Model LoC Scalable Processes LTL formula (liveness)alter 64 no 2 □(p → ◊q) ^ □(r → ◊s) giopj 740 yes j+6 □(p → ◊q)phij 57 yes j+1 □(p → ◊q)elevj 191 yes j+3 □(p → ◊q)sgc 1001 no 20 ◊p
Promela
Models
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Models & parameters Results Discussion
Parameter msteps colsize λant σs ξ a ρ α β
1st phase100
10 204
0.75 0.2 1.0 2.0
2nd phase 20 4 0.5
23 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
•We used 11 Promela
models
for the experiments
• Parameters for ACOhg-live
• Formula-based
and finite state machine
heuristics• ACOhg-live implemented in HSF-SPIN• 100
independent executions and statistical validation
Model LoC Scalable Processes LTL formula (liveness)alter 64 no 2 □(p → ◊q) ^ □(r → ◊s) giopj 740 yes j+6 □(p → ◊q)phij 57 yes j+1 □(p → ◊q)elevj 191 yes j+3 □(p → ◊q)sgc 1001 no 20 ◊p
Promela
Models
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Models & parameters Results Discussion
j=10,15,20
j=20,30,40
Parameter msteps colsize λant σs ξ a ρ α β
1st phase100
10 204
0.75 0.2 1.0 2.0
2nd phase 20 4 0.5
j=10,15,20
24 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Results I: Influence of the SCC Improvement
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Models & parameters
Results
Discussion
SDSD
25 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Results I: Influence of the SCC Improvement
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Models & parameters
Results
Discussion
SDSD
SDSD
SD SD SDSD
26 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Results I: Influence of the SCC Improvement
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Models & parameters
Results
Discussion
SDSD
SDSD
SD SD SDSD
SDSD
SD
27 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Results I: Influence of the SCC Improvement
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Models & parameters
Results
Discussion
SDSD
SDSD
SD SD SDSD
SDSD
SD
SD
SD
575192
SD
SD SD
SD
SD
SD
28 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Results II: ACOhg-live+
vs. NDFS and INDFS
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Models & parameters
Results
Discussion
Models ACOhg-live+ NDFS INDFSaltergiop10giop15giop20phi20phi30phi40elev10elev15elev20sgc
29 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Results II: ACOhg-live+
vs. NDFS and INDFS
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Models & parameters
Results
Discussion
Models ACOhg-live+ NDFS INDFSaltergiop10giop15giop20phi20phi30phi40elev10elev15elev20sgc
10001
10001
1069
1059
9999
SD
30 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Results II: ACOhg-live+
vs. NDFS and INDFS
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Models & parameters
Results
Discussion
Models ACOhg-live+ NDFS INDFSaltergiop10giop15giop20phi20phi30phi40elev10elev15elev20sgc
10001
10001
1069
1059
9999
SD
392192 388096 15360
SD
31 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Results II: ACOhg-live+
vs. NDFS and INDFS
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Models & parameters
Results
Discussion
Models ACOhg-live+ NDFS INDFSaltergiop10giop15giop20phi20phi30phi40elev10elev15elev20sgc
10001
10001
1069
1059
9999
SD
392192 388096 15360
SD
1567015320
SD
32 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
How to use ACOhg-live
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Models & parameters
Results Discussion
Model
ACOhg-live
(I)NDFS
Large
model
ora short couterexample
is
needed
Small
model
andany
counterexampleis
needed
fast
•
ACOhg-live should be used in the first/middle stages
of the software development, when software errors are expected
•
ACOhg-live can also be used in other phases of the software development for testing concurrent software
33 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
•
ACOhg-live is the first algorithm
based on metaheuristics
(to the best of our knowledge) applied to the search for liveness
errors in concurrent models
•
The improvement based on the SCCs
of the never claim outperforms
the efficacy of ACOhg-live
•
ACOhg-live is able to outperform (Improved) Nested-DFS
in efficacy and efficiency in the search for liveness
errors
Conclusions
Future Work• Analysis of parameterization for reducing the parameters
• Include ACOhg-live into JavaPathFinder
for finding liveness
errors in Java programs
•
Combine ACOhg-live with techniques for reducing the memory
required for the search such as partial order reduction
(work in progress)
Conclusions & Future Work
Introduction Background Algorithmic Proposal Experiments
Conclusions & Future Work
Conclusions & Future Work
34 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008
Thanks for your attention !!!
Searching for Liveness
Property Violations in Concurrent Systems with ACO