Icse2010 malik
Transcript of Icse2010 malik
Haroon Malik
A METHODOLOGY TO SUPPORT LOAD TEST ANALYSIS
Software Analysis & Intelligence
Lab
Load Testing
Load Generator-1
Load Generator-2
Monitoring Tool
Large Scale System
Performance counter Log
Performance Repository
Research QuestionsCan we automatically compare tests?
Can we save unnecessary load test completion?Redundant Counter Traffic Time Constraints
Interfering WorkloadsUnder Provisioning
PC-1
PC-2
PC-3
PC-4
Lots of Data Methodology Signatures
PCA Mapping Importance Ranking
Preliminary Results
5 load tests- 50 hours. 7000 performance counters in total. 3 tests are marked as similar by performance analysts. 163 Important performance counters- 88% counter reduction
1 9 17 25 33 41 49 57 65 73 81 89 97 1051131211291371451531610.4
0.5
0.6
0.7
0.8
0.9
1.0 Test- A (Base-line) Test-B Test-C Test-D Test-E
Performance Counters
Coun
ter I
mpo
rtan
ce
Future Work and ContributionsFuture Work
Pinpointing the performance deviated subsystems. Comparison with other approaches.Implementation and testing in an industrial setup.
Contributions Reduced the dimensionality of performance counter data
set.Automated the ranking of performance counters.Crafted the performance signatures in a nontrivial way. Automated the process of load test comparison.