Haroon Malik

A METHODOLOGY TO SUPPORT LOAD TEST ANALYSIS

Software Analysis & Intelligence

Lab

Load Testing

Load Generator-1

Load Generator-2

Monitoring Tool

Large Scale System

Performance counter Log

Performance Repository

Research QuestionsCan we automatically compare tests?

Can we save unnecessary load test completion?Redundant Counter Traffic Time Constraints

Interfering WorkloadsUnder Provisioning

PC-1

PC-2

PC-3

PC-4

Lots of Data Methodology Signatures

PCA Mapping Importance Ranking

Preliminary Results

5 load tests- 50 hours. 7000 performance counters in total. 3 tests are marked as similar by performance analysts. 163 Important performance counters- 88% counter reduction

1 9 17 25 33 41 49 57 65 73 81 89 97 1051131211291371451531610.4

0.5

0.6

0.7

0.8

0.9

1.0 Test- A (Base-line) Test-B Test-C Test-D Test-E

Performance Counters

Coun

ter I

mpo

rtan

ce

Future Work and ContributionsFuture Work

Pinpointing the performance deviated subsystems. Comparison with other approaches.Implementation and testing in an industrial setup.

Contributions Reduced the dimensionality of performance counter data

set.Automated the ranking of performance counters.Crafted the performance signatures in a nontrivial way. Automated the process of load test comparison.