Professional Portfolio
Harnessing Data For Performance Improvement
The following slides depict statistical analyses I conducted on patient level data and performance dashboards I developed that revealed performance improvement opportunities and catalyzed performance improvement projects.
Robert Sutter, RN MBA MHA
Quality Performance DashboardDeveloped this quality performance dashboard for a health system to assess and monitor the quality of care provided, as well as guide annual quality improvement planning.
The dashboard has several unique features:
Each category is comprised of sub-categories and associated metrics.
Category and sub-category performance is summarized by robust composite indicators.
Every metric is compared to an external benchmark.
The dashboard provides relevant information to all levels of the organization from the Board of Directors to middle managers and medical staff.
Dissemination of this information initiated the development of annual quality improvement planning and project reviews throughout the health system and stimulated the incorporation of quality improvement into the strategic planning process.
3 Robert Sutter, RN MBA MHA
Quality Performance DashboardThis figure depicts the additional information within each sub-category of the Quality Performance Dashboard.
On the prior slide, hospital H had a one star – less than the benchmark –performance in Core Measures.
Additional information available reveals that Pneumonia has a less than the benchmark performance and the following metrics are less than the benchmark:
Pneumococcal screening
Smoking cessation advice
Antibiotic selection
Antibiotic within 6 hours
Influenza vaccination
Subsequently hospital H launched performance improvement projects to close the performance gap.
4 Robert Sutter, RN MBA MHA
Cardiothoracic Performance Dashboard
Harnessing the data collected for the Society of Thoracic Surgeons Adult Cardiac Database, this dashboard is updated monthly in order to provide feedback to the hospitals more frequently than the quarterly report from STS.
The comparative nature of the dashboard catalyzed benchmarking and initiated performance improvement projects throughout the health system. The data was also used in several data analysis projects to answer questions posed by the cardiothoracic surgeons (see slides 9-12).
5 Robert Sutter, RN MBA MHA
Physician Performance MeasurementA physician performance measurement system was developed to answer three questions:
What proportion of variability is attributable to physicians?
Is there a statistically significant difference in physician performance?
Is there a distribution in outcome categories among physicians?
The answers to these questions provide the necessary information to develop an effective physician performance improvement strategy.
This analysis has notably enhanced physician engagement.
6 Robert Sutter, RN MBA MHA
3.9
37.2
99.2
x
DIABETES
HERNIORRHAPHY
CHEST PAIN
Physician Variability Percent
2.09
1.26
-0.07
-0.13
-0.34
-0.38
-0.43
-0.53
-0.54
-0.58
Median
10
1
6
3
8
5
2
4
9
7
Risk-Adjusted LOS Excess
P<0.05
Attending Physician
Chest Pain
43
12
7
40
56
59
46
11
2
19
Att
end
ing
Phys
icia
n
-2 -1 0 1 2 3 4 5 6
Risk-Adjusted Median Excess LOS Confidence Interval
Better Than Expected
As Expected
Worse Than Expected
Length of Stay Outcome Categories
Confidence Level = 0.95
Attending Physician
Chest Pain
SCIP Core Measures Data Analysis & ImprovementA multilevel logistic regression analysis of the SCIP core measures patient level data, comprising all hospitals, revealed the following factors significantly associated with administering an antibiotic within one hour prior to incision:
Surgical Procedure
Surgical Day of Week
Shift
This analysis catalyzed a system-wide performance improvement project that resulted in significant improvement.
.94 .93 .9 .91 .86.95 .91
0.2
.4.6
.81
Porp
ort
ion
CABG
Oth
er C
ardiac H
ip
Kne
e
Colon
Hys
tere
ctom
y
Vas
cula
r
P=0.0445
Surgical Procedure
Antibiotic Within 1 Hr Prior to Incision
.83.93 .93 .94 .94 .92 .96
0.2
.4.6
.81
Porp
ort
ion
Sun Mon Tue Wed Thu Fri Sat
P=0.0222
Surgery Day of Week
Antibiotic Within 1 Hr Prior to Incision
.9 .94
0.2
.4.6
.81
Porp
ort
ion
Evening Day
P=0.0186
Shift
Antibiotic Within 1 Hr Prior to Incision
.93 .96
0.2
.4.6
.81
Pro
port
ion
Baseline Improvement
P<0.000
System Performance
Antibiotic Within 1 Hour Prior to Incision
7 Robert Sutter, RN MBA MHA
SCIP Core Measures Data Analysis & ImprovementA multilevel logistic regression analysis of the SCIP core measures patient level data revealed that timely antibiotic discontinuation is significantly associated with patient’s acquiring an infection.
Further analysis revealed the following factors significantly associated with timely discontinuation of antibiotics post-operatively:
Hospitals
Surgical Procedure
These analyses catalyzed a system-wide performance improvement project that resulted in statistically significant improvement.
.013
.002
0
.005
.01
.015
Infe
ction R
ate
No Yes
P=0.037
Infection
Timely Antibiotic Discontinuation
.86 .85 .9.99
.911
.93
.79
1
0.2
.4.6
.81
Pro
port
ion
1 2 3 4 5 6 7 8 9P<0.000
Hospital Comparison
Timely Antibiotic Discontinuation
.95.88 .87 .91
.67
.96
.78
0.2
.4.6
.81
Pro
port
ion
CABG
Oth
er C
ardiac H
ip
Kne
e
Colon
Hys
tere
ctom
y
Vas
cula
r
P<0.0000
Surgical Procedure
Timely Antibiotic Discontinuation
.91 .93
0.2
.4.6
.81
Pro
port
ion
Baseline ImprovementP=0.003
System Performance
Timely Antibiotic Discontinuation
8 Robert Sutter, RN MBA MHA
SCIP Core Measures Data Analysis & ImprovementUsing the SCIP Core Measures patient level data, statistically significant differences in the proportion of cardiac surgery patients with appropriate post-operative glucose control among hospitals was revealed.
This resulted in launching a system-wide performance improvement project that yielded a significant system-wide improvement.
9
.48
.87
.95
.8
.94 .94.91
.73
.81
0.2
.4.6
.81
Pro
port
ion
1 2 3 4 5 6 7 8 9P<0.000
Hospital Comparison
Cardiac Surgery Glucose Control
.82
.94
0.2
.4.6
.81
Pro
port
ion
Baseline ImprovementP<0.000
System Performance
Cardiac Surgery Glucose Control
Robert Sutter, RN MBA MHA
Society of Thoracic Surgeons Data Analysis The following analyses of the STS patient level data catalyzed numerous performance improvement projects throughout the hospitals that are currently underway.
In addition, a monthly STS report was developed and disseminated via SharePoint to provide hospitals with more frequent and timely information to assist in their improvement projects.
A propensity score analysis revealed that pre-operative beta-blocker use in isolated CABG patients was significantly associated with a lower mortality rate.
Further analysis exposed significant differences among hospitals in pre-operative beta-blocker use as well as composite medication performance in isolated CABG patients.
10
.029
.013
0
.01
.02
.03
Mort
alit
y R
ate
No YesOdds Ratio 0.360: P<0.000
Pre-Operative Beta Blocker
Isolated CABG
.59.66
.78.7 .72
.57
0.2
.4.6
.8
Pro
port
ion
1 2 3 4 5 6P<0.000
Hospital Comparison
Isolated CABG Pre-OP Beta-Blocker
.39
.62.68
.55.49
.71
.45
0.2
.4.6
.8
Pro
port
ion
1 2 3 4 5 6 7P<0.000
Hospital Comparison
Isolated CABG Composite Medication
Robert Sutter, RN MBA MHA
Society of Thoracic Surgeons Data Analysis A multilevel logistic regression analysis uncovered highly significant relationships between the occurrence of isolated CABG post-operative complications and mortality.
Numerous performance improvement projects were launched to reduce the incidence of post-operative complications.
11
.053
.18
0
.05
.1.1
5.2
Mort
alit
y R
ate
No YesOdds Ratio 3.0: P=0.010
Post-Operative Stroke
Isolated CABG
.029
.27
0.1
.2.3
Mort
alit
y R
ate
No YesOdds Ratio 12.4: P<0.000
Renal Failure
Isolated CABG
.026
.23
0
.05
.1.1
5.2
.25
Mort
alit
y R
ate
No YesOdds Ratio 12.2: P<0.000
Prolonged Ventilation
Isolated CABG
.043
.2
0
.05
.1.1
5.2
Mort
alit
y R
ate
No YesOdds Ratio 5.4: P<0.000
Reoperation
Isolated CABG
.016
.17
0
.05
.1.1
5.2
Mort
alit
y R
ate
No YesOdds Ratio 13.0: P<0.000
Prolonged Post-Operative LOS
Isolated CABG
Robert Sutter, RN MBA MHA
Society of Thoracic Surgeons Data Analysis A multilevel logistic regression analysis uncovered highly significant relationships between the occurrence of isolated CABG post-operative complications and prolonged post-operative length of stay.
Numerous performance improvement projects were launched to reduce the incidence of post-operative complications.
12
.064
.25
0
.05
.1.1
5.2
.25
Pro
lon
ge
d P
ost-
OP
Lo
s R
ate
No YesOdds Ratio 4.9: P=0.001
Post-Operative Stroke
Isolated CABG
.05
.23
0
.05
.1.1
5.2
.25
Pro
lon
ge
d P
ost-
OP
Lo
s R
ate
No YesOdds Ratio 5.7: P<0.000
Renal Failure
Isolated CABG
.032
.28
0.1
.2.3
Pro
lon
ge
d P
ost-
OP
Lo
s R
ate
No YesOdds Ratio 13.7: P<0.000
Prolonged Ventilation
Isolated CABG
.062
.16
0
.05
.1.1
5.2
Pro
lon
ge
d P
ost-
OP
Lo
s R
ate
No YesOdds Ratio 3.0: P<0.005
Reoperation
Isolated CABG
Robert Sutter, RN MBA MHA
Society of Thoracic Surgeons Data Analysis Surgeon specific risk-adjusted mortality and reoperation performance was derived for hospitals to facilitate focusing improvement efforts.
13
3.5
2.8
0
5.5
1.7
9.8
0
02
46
810
Obse
rved
/Expe
cte
d M
ort
alil
ty R
atio
1 2 4 6 7 8 9
Surgeon
Isolated CABG Observed/Expected Mortality
.9
1.1
0
2.1
1
1.9
0
0.5
11.5
2
Obse
rved
/Expe
cte
d R
eo
pe
ration
Ratio
1 2 4 6 7 8 9
Surgeon
Isolated CABG Observed/Expected Reoperation
Robert Sutter, RN MBA MHA
American College of Cardiology Data Analysis The American College of Cardiology patient level data was analyzed to determine if there were significant differences in hospital utilization of contraindicated antithrombotics in dialysis patients undergoing PCI.
The results revealed highly significant differences in hospital utilization of contraindicated antithrombotics.
This information was presented to the medical staff at each hospital and subsequent changes in practice patterns were initiated.
14
.25
.29
.06
.43
.29
.087
0.1
.2.3
.4
Pro
port
ion
1 2 3 4 5 6P<0.000
Hospital Comparison
PCI Dialysis Contraindicated Antithrombotics
.88
.38
.25
.85
1
.33
.67
.42
.57
.43
0
1
0.2
.4.6
.81
Pro
port
ion
1 2 3 4 5 6
Hospital Comparison
PCI Dialysis Contraindicated Antithrombotics
mean of enoxaparin
mean of eptifibatide
Robert Sutter, RN MBA MHA
American College of Cardiology Data Analysis The American College of Cardiology patient level data was analyzed to determine if there were significant differences in the incidence of vascular complications among hospitals.
The results revealed highly significant differences.
This stimulated benchmarking and process improvement at various hospitals.
15
.0041
.012 .014
.041
.011
.02
0
.01
.02
.03
.04
Pro
port
ion
1 2 3 4 5 6P<0.000
Hospital Comparison
Cardiac Catheterization Vascular Complications
.0084
.037
.015
0
.022
.048
0
.01
.02
.03
.04
.05
Pro
port
ion
1 2 3 4 5 6P=0.014
Hospital Comparison
Percutaneous Coronary Intervention Vascular Complications
.0012 0
.012
.053
.0078.0039
0
.01
.02
.03
.04
.05
Pro
port
ion
1 2 3 4 5 6P<0.000
Hospital Comparison
Diagnostic Catheterization Vascular Complications
Robert Sutter, RN MBA MHA
American College of Cardiology Data Analysis Based on the previous analysis one of the hospitals wanted to answer the following questions regarding diagnostic catheterization:
Is there a significant difference among physicians?
Are certain patient characteristics associated with vascular complications?
The results revealed highly significant differences among physicians.
Multilevel logistic regression analysis indicated that patient characteristics are not significantly associated with vascular complications.
This information stimulated evaluating physician practice patterns.
16
.8
0 0 0.034
.067
0
0.2
.4.6
.8
Pro
po
rtio
n
1 2 3 4 5 6 9P<0.000
Physician Comparison
Diagnostic Catheterization Vascular Complications
Variable P Value
Gender 0.265
Hypertension 0.508
Prior MI 0.273
Prior Heart Failure 0.494
Diabetes 0.867
Dyslipidemia 0.636
Peripheral Arterial Disease 0.337
Prior PCI 0.372
Age_spline1 0.444
Age_spline2 0.673
Robert Sutter, RN MBA MHA
Top Related