OpenStack CI by the Numbers
One proposed change generates:
I 4-25 Devstacks VMs
I ˜1.5k tempest tests per devstack
I ˜130 second level guests started per tempest run
Looking at December:
I ˜12K gating tempest jobs
I ˜15M tempest tests run in the gate
Looking at Kilo:
I ˜70K gating tempest jobs
I ˜>80M tempest tests run in the gate
What was missing?
I An aggregate view of individual tests
I Success and failure rates for tests over time
I Run times for each test over time
I All of this in a programatically queriable location
What is subunit2sql?
I A python library and utilities to aggregate and store subunit streams in a SQL DBI Currently 3 utilities:
I subunit2sql - Take in a subunit stream and store it in a subunit2sql DBI sql2subunit - Output a subunit stream from a subunit2sql DBI subunit2sql-db-manage - Setup and manage the DB schemaI subunit2sql-graph - Tool to generate graphs for analysis with data from a
subunit2sql DB
I Provides a python DB api for interacting with the DB which provides basic iooperations and some analysis tooling
Tracking an individual test’s failure rateYou can use subunit2sql-graph to figure out failure rates for tests:
which also displays percentages:
I Fail Percentage: 0.2045%
I Success Percentage: 99.7955%
I Skip Percentage: 0.0000%
Where to get more information
I subunit2sql Repository
I subunit2sql Documentation
I Infra Documentation for subunit2sql
I IRC #openstack-qa or #openstack-infra channel on Freenode
Top Related