Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI...

59
Slide 1 of 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept 2011 Hugo McClean On behalf of the BASHH National Audit Group

Transcript of Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI...

Page 1: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 1 of 59

2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS)

Updated presentation 20 Sept 2011

Hugo McCleanOn behalf of the BASHH National

Audit Group

Page 2: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 2 of 59

2011 STIMS Audit- presentation scheme

• STIMS Key Performance Indicators (KPIs)• Methods• Results• Key messages• Areas for improvement• Use of STIMS Audit findings

Page 3: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 3 of 59

2011 Audit against the BASHH MedFASH STI Management Standards (STIMS)

• Standards:• Standards for the management

of sexually transmitted infections

• Posted on BASHH and MedFASH websites Jan 2010

• Aims: “to support the commissioning and provision of high quality care for STIs across all settings”

Page 4: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 4 of 59

STIMS KPIs

• Practice in 9 patient management and policy areas• Many required further definition to allow auditing• (Continuing role of BASHH Clinical Standards Unit,

Dr Immy Ahmed)

Page 5: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 5 of 59

STIMS KPIsPatient management: data from case notes

KPI Summary Further definition needed

1.4.1 48 hour access (patient registers) No3.4.1 Sexual history Yes3.4.1 STI/HIV risk assessment Yes3.4.2 HIV testing & uptake No4.4.1 Test results within 7 working days Yes5.4.1 Partner notification Yes

Page 6: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 6 of 59

STIMS KPIsClinic policies

KPI Summary Further definition needed

2.4.1 Staff competency/updating Yes6.4.1 Information governance Yes7.4.1 Care pathways to Level 3 services Yes8.4.1 Audit participation and planning No9.4.1 Patient and Public Engagement plan Yes

Page 7: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 7 of 59

Methods- eligible services

• Eligible services:– All UK Nations

– Genitourinary medicine clinics

– Services providing STI management at either level 2 or 3 defined in the BASHH MedFASH STIMS Project definitions in STIMS Appendix B

• Faculty of Sexual & Reproductive Healthcare consultant-led services

• Primary Care practitioner-led services at level 2

• Commissioned by PCTs to be provided by the independent or third sectors

• Not included:– Pharmacy-based services– National Chlamydial Screening Programme services– Non PCT-commissioned independent or third sector services

Page 8: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 8 of 59

Methods- principles

• Clinics seeing smaller numbers of cases• Senior clinical staff asked to assist with data

collection• Asked to seek help from managers for

information governance questions• Informed of planned re-audit in 2014• Case note data returned at clinic level

Page 9: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 9 of 59

Methods: data collection- 1

• Audit interval:– Clinic policies: as of 31 December 2010– Cases seen 1 October to 31 December 2010

• Data collection– Launched 6 Jan 2011– Closed 30 April 2011– Complete data set presented

• Participation:– Level 2 services: PCT Sexual Health Leads via Andrea Duncan (DH

Sexual Health & HIV Programme Manager)– Level 3 services: BASHH NAG Regional Chairs network– BASHH website

• Comments in free text boxes for each question: BASHH website

Page 10: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 10 of 59

Data collection- 2

• Data collected by clinics in an Excel workbook• Data submitted using an online form

Page 11: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 11 of 59

Data collection- 3

• Clinic policy data• Case note data:

– Up to consecutive 40 cases*– Clinic level performance

computed: no individual patient data

• Free text comments

*RCP Local Clinical Audit: handbook for physicians: http://old.rcplondon.ac.uk/clinical-standards/ceeu/Documents/Local-clinical-audit-handbook-for-physicians-August-2010.pdf

Page 12: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 12 of 59

Data collection- 4

• Data transcribed and submitted using an online form:

Page 13: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 13 of 59

Results- clinic level data

• Clinic level data• Percentage of clinics in each BASHH Region

meeting KPI performance target• Overall national performance

Page 14: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 14 of 59

Number (%) of clinics participating, by RegionBASHH Audit Group Region Level 3 (%) Level 2 (%) Total (%)

North Thames 27 (18%) 4 (11%) 31 (17%)South Thames 24 (16%) 10 (27%) 34 (18%)Trent 15 (10%) 3 (8%) 18 (10%)South West 12 (8%) 3 (8%) 15 (8%)Yorkshire 12 (8%) 6 (16%) 18 (10%)Anglia 10 (7%) 6 (16%) 16 (9%)Wales 10 (7%) 3 (8%) 13 (7%)Northern 8 (5%) None 8 (4%)West Midlands 8 (5%) None 8 (4%)Oxford 6 (4%) 1 (3%) 7 (4%)Northern Ireland 5 (3%) None 5 (3%)Wessex 5 (3%) None 5 (3%)Cheshire and Merseyside 4 (3%) None 4 (2%)North West 3 (2%) 1 (3%) 4 (2%)National (%) 149 (100%) 37 (100%) 186 (100%)

Page 15: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 15 of 59

KPI 1. 48 hour access

• Percentage of people offered an appointment, or seen by a healthcare worker on walking-in, within 48 hours of contacting an STI provider

• Standard 98%

Page 16: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 16 of 59

KPI 1. Percentage of clinics in each Region with ≥98% 48 hour access

N. Ireland L3=5, L2=0

Oxford L3=6, L2=1

Yorkshire L3=12, L2=6

Wales L3=10, L2=3

South West

L3=12, L2=3

Trent L3=15, L2=3

South Thames L3=24, L2=10

North Thames L3=27, L2=4

Anglia L3=10, L2=6

Cheshire & M'side

L3=4, L2=0

North West L3=3, L2=1

Northern L3=8, L2=0

Wessex L3=5, L2=0

West Midlands

L3=8, L2=0

National L3=149, L2=37

0 0.2 0.4 0.6 0.8 1

20%

67%

67%

70%

83%

87%

88%

96%

100%

100%

100%

100%

100%

100%

86%

100%

17%

100%

33%

50%

100%

50%

100%

51%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 17: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 17 of 59

KPI 2. Staff competency

• Percentage of staff who have completed competency-based training

• No nationally agreed standards for competency for some clinical staff groups

• Percentage of staff with documentation of competency

• Question: no justification of how competency was achieved, only whether it was documented

• Standard 100%

Page 18: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 18 of 59

KPI 2. Percentage of clinics in each Region with 100% staff with competency documented

N. Ireland L3=5, L2=0

Trent L3=15, L2=3

Wales L3=10, L2=3

South West

L3=12, L2=3

North West L3=3, L2=1

Oxford L3=6, L2=1

Northern L3=8, L2=0

West Midlands

L3=8, L2=0

Anglia L3=10, L2=6

Wessex L3=5, L2=0

North Thames L3=27, L2=4

Cheshire & M'side

L3=4, L2=0

Yorkshire L3=12, L2=6

South Thames L3=24, L2=10

National L3=149, L2=37

0 0.2 0.4 0.6 0.8 1

20%

20%

20%

25%

33%

33%

38%

38%

40%

40%

48%

50%

50%

58%

40%

67%

0%

33%

0%

100%

50%

50%

67%

100%

62%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 19: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 19 of 59

KPI 2. Staff updating

• Percentage of staff who have fulfilled update requirements

• No nationally agreed standards for update requirements

• Percentage of staff with documentation of updating• Question: no justification of how updating was

achieved, only whether it was documented• Standard 100%

Page 20: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 20 of 59

KPI 2. Percentage of clinics in each Region with 100% staff with updating documented

N. Ireland L3=5, L2=0

Cheshire & M'side

L3=4, L2=0

Wales L3=10, L2=3

North West L3=3, L2=1

South West

L3=12, L2=3

Trent L3=15, L2=3

Anglia L3=10, L2=6

Northern L3=8, L2=0

Oxford L3=6, L2=1

Wessex L3=5, L2=0

West Midlands

L3=8, L2=0

North Thames L3=27, L2=4

South Thames L3=24, L2=10

Yorkshire L3=12, L2=6

National L3=149, L2=37

0 0.2 0.4 0.6 0.8 1

20%

25%

30%

33%

42%

47%

50%

50%

50%

60%

63%

67%

67%

67%

54%

0

0

1

0.333333333333333

0.5

1

1

0.9

0.5

0.648648648648649

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 21: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 21 of 59

KPI 3. Sexual history

• KPI: “Percentage of individuals with STI concerns who had a sexual history taken. Standard 100%”• Case definition: eligible for STI screening because of concern about STIs spontaneously

expressed, or elicited during the visit to a service (either verbally or indicated on a triage or similar form) • Scoring system based on data elements from:– BASHH 2006 National guidelines on undertaking consultations requiring sexual history taking*

• Competency (if age <16 years)• Symptoms• Reason for attendance• Sexual contact details• Previous STIs• First day of last menses, or documentation about vaginal bleeding (women)• Contraception (women)• Cervical cytology (women age >=25 years)

– Score weighting based on gender and age• Standard: 75% documentation based on questions

Page 22: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 22 of 59

KPI 3. Percentage of clinics in each Region with ≥75% sexual history documentation

Oxford L3=6, L2=1

N. Ireland L3=5, L2=0

Wessex L3=5, L2=0

Trent L3=15, L2=3

Yorkshire L3=12, L2=6

Anglia L3=10, L2=6

Wales L3=10, L2=3

Northern L3=8, L2=0

West Midlands

L3=8, L2=0

South Thames L3=24, L2=10

South West

L3=12, L2=3

North Thames L3=27, L2=4

Cheshire & M'side

L3=4, L2=0

North West L3=3, L2=1

National L3=149, L2=37

0% 20% 40% 60% 80% 100%1

0.333333333333333

0.666666666666667

0.333333333333333

1

0.7

0.333333333333333

0.75

0.594594594594595

50%

60%

60%

67%

67%

70%

70%

75%

75%

79%

83%

93%

100%

100%

77%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 23: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 23 of 59

KPI 3. STI/HIV risk assessment

• KPI: Percentage of individuals with STI concerns who had a STI/HIV risk assessment made. Standard 100%• Case definition: eligible for STI screening because of concern about STIs

spontaneously expressed, or elicited during the visit to a service • Scoring system based on data elements from:– BASHH 2006 National guidelines on undertaking consultations requiring sexual

history taking*• Lifetime injecting drug use• Sex abroad• Risk factors for hepatitis B • Medical treatment abroad • HIV testing history • Lifetime sexual contact with another man (men)

– Score weighting based on gender• Standard: 75% documentation based on questions

Page 24: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 24 of 59

KPI 3. Percentage of clinics in each Region with ≥75% STI/HIV risk assessment documentation

Trent L3=15, L2=3

Cheshire & M'side

L3=4, L2=0

Anglia L3=10, L2=6

North West L3=3, L2=1

Oxford L3=6, L2=1

Yorkshire L3=12, L2=6

North Thames L3=27, L2=4

Wales L3=10, L2=3

N. Ireland L3=5, L2=0

South Thames L3=24, L2=10

West Midlands

L3=8, L2=0

Wessex L3=5, L2=0

South West

L3=12, L2=3

Northern L3=8, L2=0

National L3=149, L2=37

0% 20% 40% 60% 80% 100%0.333333333333333

0.5

0

1

0.333333333333333

0

1

0.5

0.333333333333333

0.432432432432432

13%

25%

30%

33%

50%

58%

59%

60%

60%

63%

75%

80%

83%

88%

56%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 25: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 25 of 59

KPI 3. HIV testing & uptake

•Case definition: eligible for STI screening because of concern about STIs spontaneously expressed, or elicited during the visit to a service • Standards:–Offer 100%–Uptake, by those offered, 60%

Page 26: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 26 of 59

KPI 3. Percentage of clinics in each Region with 100% HIV test offer

N. Ireland L3=5, L2=0

Cheshire & M'side

L3=4, L2=0

North West L3=3, L2=1

Oxford L3=6, L2=1

Yorkshire L3=12, L2=6

South Thames L3=24, L2=10

South West

L3=12, L2=3

Wales L3=10, L2=3

Trent L3=15, L2=3

North Thames L3=27, L2=4

Northern L3=8, L2=0

West Midlands

L3=8, L2=0

Anglia L3=10, L2=6

Wessex L3=5, L2=0

National L3=149, L2=37

0 0.2 0.4 0.6 0.8 1

20%

25%

33%

33%

33%

46%

50%

50%

53%

56%

63%

63%

70%

80%

50%

1

0

0.333333333333333

0.3

0.333333333333333

0.666666666666667

0.666666666666667

0.25

0.666666666666667

0.432432432432432

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 27: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 27 of 59

KPI 3. Percentage of clinics in each Region with ≥60% HIV test uptake (of those offered)

Cheshire & M'side

L3=4, L2=0

Wales L3=10, L2=3

Trent L3=15, L2=3

Northern L3=8, L2=0

South West

L3=12, L2=3

Yorkshire L3=12, L2=6

South Thames L3=24, L2=10

Anglia L3=10, L2=6

North Thames L3=27, L2=4

North West L3=3, L2=1

N. Ireland L3=5, L2=0

Oxford L3=6, L2=1

Wessex L3=5, L2=0

West Midlands

L3=8, L2=0

National L3=149, L2=37

0 0.2 0.4 0.6 0.8 1

75%

80%

87%

88%

92%

92%

96%

100%

100%

100%

100%

100%

100%

100%

94%

100%

67%

67%

100%

50%

67%

50%

100%

100%

70%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 28: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 28 of 59

KPI 4. Test results within 7 days

• Standard– “Percentage of reports (or preliminary reports) that

are received by clinicians within 7 working days of a specimen being taken”

• Further definition:– Chlamydial test results chosen– 'Received' = date report accessible to a relevant

clinician, either paper report, or electronically. – Paper reports- date stamped on the report – Electronic reports- date report electronically posted

by laboratory

Page 29: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 29 of 59

KPI 4. Percentage of clinics in each Region with 100% positive chlamydia test results within 7 days

Yorkshire L3=12, L2=6

N. Ireland L3=5, L2=0

Wales L3=10, L2=3

Cheshire & M'side

L3=4, L2=0

Trent L3=15, L2=3

North West L3=3, L2=1

Oxford L3=6, L2=1

North Thames L3=27, L2=4

Anglia L3=10, L2=6

South West

L3=12, L2=3

South Thames L3=24, L2=10

Northern L3=8, L2=0

West Midlands

L3=8, L2=0

Wessex L3=5, L2=0

National L3=149, L2=37

0% 20% 40% 60% 80% 100%0.666666666666667

1

0

0

1

0.75

0.333333333333333

0.333333333333333

0.7

0.567567567567567

17%

20%

20%

25%

27%

33%

33%

37%

40%

42%

46%

63%

75%

80%

39%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 30: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 30 of 59

KPI 5. Partner notification

• Standard– “Rate of partner notification for chlamydia and gonorrhoea for each STI provider”

– Standard:

– At least 0.4 contacts per index cases in large conurbations, 0.6 elsewhere

– Within four weeks

• Measured for chlamydial infection• Further definition:

– Contact event = seen for management

– Resolution: both verified by a healthcare worker AND reported by an index case

– Verified = contacting another agency if necessary

– Four weeks start = from date of first PN interview

• See slide 52 for London and outside-London PN performance

Page 31: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 31 of 59

KPI 5. Percentage of clinics in each Region with ≥0.4 chlamydial contacts verified by a healthcare worker

South West

L3=12, L2=3

North Thames L3=27, L2=4

South Thames L3=24, L2=10

Anglia L3=10, L2=6

N. Ireland L3=5, L2=0

West Midlands

L3=8, L2=0

Oxford L3=6, L2=1

Yorkshire L3=12, L2=6

Cheshire & M'side

L3=4, L2=0

Northern L3=8, L2=0

Trent L3=15, L2=3

Wales L3=10, L2=3

North West L3=3, L2=1

Wessex L3=5, L2=0

National L3=149, L2=37

0% 20% 40% 60% 80% 100%0%

75%

30%

50%

0%

50%

0%

67%

100%

41%

42%

44%

50%

60%

60%

63%

67%

67%

75%

75%

87%

90%

100%

100%

63%

Level 3, n=149Level 2, n=37

Percentage of clinics meeting Standard

Null returns: Level 2, 49%; Level 3, 7%

Page 32: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 32 of 59

KPI 5. Percentage of clinics in each Region with ≥0.6 chlamydial contacts verified by healthcare workers

South West

L3=12, L2=3

North Thames L3=27, L2=4

Cheshire & M'side

L3=4, L2=0

South Thames L3=24, L2=10

West Midlands

L3=8, L2=0

Oxford L3=6, L2=1

N. Ireland L3=5, L2=0

Yorkshire L3=12, L2=6

Anglia L3=10, L2=6

Northern L3=8, L2=0

Wales L3=10, L2=3

Wessex L3=5, L2=0

North West L3=3, L2=1

Trent L3=15, L2=3

National L3=149, L2=37

0% 20% 40% 60% 80% 100%0%

75%

20%

0%

50%

50%

33%

100%

0%

35%

17%

22%

25%

25%

25%

33%

40%

42%

50%

50%

50%

60%

67%

73%

38%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Null returns: Level 2, 49%; Level 3, 7%

Page 33: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 33 of 59

KPI 5. Percentage of clinics in each Region with ≥0.4 chlamydial contacts reported by index cases

Cheshire & M'side

L3=4, L2=0

Northern L3=8, L2=0

South West

L3=12, L2=3

Trent L3=15, L2=3

Wales L3=10, L2=3

N. Ireland L3=5, L2=0

West Midlands

L3=8, L2=0

North West L3=3, L2=1

Anglia L3=10, L2=6

Yorkshire L3=12, L2=6

Wessex L3=5, L2=0

North Thames L3=27, L2=4

Oxford L3=6, L2=1

South Thames L3=24, L2=10

National L3=149, L2=37

0 0.2 0.4 0.6 0.8 1

25%

50%

50%

60%

60%

60%

63%

67%

70%

75%

80%

81%

83%

88%

70%

0.333333333333333

0.333333333333333

0.666666666666667

0

0.5

0.333333333333333

0.5

1

0.4

0.432432432432432

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Null returns: Level 2, 41%; Level 3, 5%

Page 34: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 34 of 59

KPI 5. Percentage of clinics in each Region with ≥0.6 chlamydial contacts reported by index cases

Cheshire & M'side

L3=4, L2=0

Northern L3=8, L2=0

North West L3=3, L2=1

South West

L3=12, L2=3

Oxford L3=6, L2=1

West Midlands

L3=8, L2=0

N. Ireland L3=5, L2=0

Yorkshire L3=12, L2=6

Trent L3=15, L2=3

Anglia L3=10, L2=6

Wales L3=10, L2=3

North Thames L3=27, L2=4

South Thames L3=24, L2=10

Wessex L3=5, L2=0

National L3=149, L2=37

0% 20% 40% 60% 80% 100%0

0

0.333333333333333

1

0.333333333333333

0.333333333333333

0.333333333333333

0.666666666666667

0.5

0.4

0

0.405405405405406

0%

25%

33%

33%

33%

38%

40%

42%

47%

60%

60%

63%

75%

80%

52%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Null returns: Level 2, 41%; Level 3, 5%

Page 35: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 35 of 59

National PN performance: percentage of clinics with ≥0.4 & ≥0.6 performance levels

Service level

Verified/reportedPerformance level: contacts

seen/index cases≥0.4 ≥0.6

% Clinics with this level of performance:

Level 3, n=164

Verified 63% 38%Reported 70% 52%

Level 2, n=37

Verified 41% 35%Reported 43% 41%

Page 36: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 36 of 59

Level 3 PN performance: 2011 vs 2007

Performance level: contacts/index cases≥0.4 ≥0.6

Audit Verified Reported Verified Reported% Clinics with this level of performance:

2011 Standards Audit(contacts seen)

63% 70% 38% 52%

2007 BASHH Chlamydia Audit (contacts screened)

55% 52% 31% 25%

Page 37: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 37 of 59

KPI 6. Information governance

• BASHH Standards: “Provision of data by all providers of services managing STIs complies with national and local reporting requirements”

• Please check with your service manager and/or senior clinical staff to help answer these questions!

• Scoring system based on 22 information governance components

• Standard 100%

Page 38: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 38 of 59

KPI 6. Information governance- questions

Page 39: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 39 of 59

KPI 6. Percentage of clinics in each Region with 100% information governance score (22/22)

Trent L3=15, L2=3

West Midlands

L3=8, L2=0

North West L3=3, L2=1

Oxford L3=6, L2=1

Northern L3=8, L2=0

N. Ireland L3=5, L2=0

Wales L3=10, L2=3

Yorkshire L3=12, L2=6

Cheshire & M'side

L3=4, L2=0

South West

L3=12, L2=3

South Thames L3=24, L2=10

North Thames L3=27, L2=4

Anglia L3=10, L2=6

Wessex L3=5, L2=0

National L3=149, L2=37

0% 20% 40% 60% 80% 100%0.333333333333333

0

0

1

0.166666666666667

0.333333333333333

0.7

0

0.166666666666667

0.378378378378379

20%

25%

33%

33%

38%

40%

40%

42%

50%

50%

63%

63%

80%

80%

50%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 40: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 40 of 59

KPI 7. Care pathways to Level 3 services

• 2 elements, documented evidence of explicit:1. Agreed care pathways linking all providers of

services managing STIs in your area with Level 3 services

2. Level 3 leadership role for your area

Page 41: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 41 of 59

KPI 7. Percentage of clinics in each Region with care pathways to Level 3 services documented

N. Ireland L3=5, L2=0

South West

L3=12, L2=3

West Midlands

L3=8, L2=0

Yorkshire L3=12, L2=6

North West L3=3, L2=1

Northern L3=8, L2=0

Anglia L3=10, L2=6

North Thames L3=27, L2=4

Cheshire & M'side

L3=4, L2=0

Trent L3=15, L2=3

Wales L3=10, L2=3

Wessex L3=5, L2=0

South Thames L3=24, L2=10

Oxford L3=6, L2=1

National L3=149, L2=37

0 0.2 0.4 0.6 0.8 1

0%

25%

25%

25%

33%

38%

40%

41%

50%

53%

60%

80%

92%

100%

50%

100%

100%

100%

83%

100%

100%

0%

100%

100%

89%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 42: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 42 of 59

KPI 7. Percentage of clinics in each Region with Level 3 leadership documented

South West

L3=12, L2=3

North Thames L3=27, L2=4

Trent L3=15, L2=3

Yorkshire L3=12, L2=6

Anglia L3=10, L2=6

West Midlands

L3=8, L2=0

Wales L3=10, L2=3

Cheshire & M'side

L3=4, L2=0

Northern L3=8, L2=0

N. Ireland L3=5, L2=0

Oxford L3=6, L2=1

South Thames L3=24, L2=10

North West L3=3, L2=1

Wessex L3=5, L2=0

National L3=149, L2=37

0% 20% 40% 60% 80% 100%100%

50%

67%

83%

67%

100%

100%

100%

0%

81%

33%

41%

47%

58%

60%

63%

70%

75%

75%

80%

83%

83%

100%

100%

62%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 43: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 43 of 59

KPI 8. Audit: annual participation & plan

• 2 elements, annual:– Participation in a regional or national audit– Completion of an audit plan

Page 44: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 44 of 59

KPI 8. Percentage of clinics in each Region with evidence of annual participation in audit

Cheshire & M'side

L3=4, L2=0

N. Ireland L3=5, L2=0

South Thames L3=24, L2=10

Trent L3=15, L2=3

Northern L3=8, L2=0

Yorkshire L3=12, L2=6

Anglia L3=10, L2=6

North Thames L3=27, L2=4

North West L3=3, L2=1

Oxford L3=6, L2=1

South West

L3=12, L2=3

Wales L3=10, L2=3

Wessex L3=5, L2=0

West Midlands

L3=8, L2=0

National L3=149, L2=37

0 0.2 0.4 0.6 0.8 1

75%

80%

83%

87%

88%

92%

100%

100%

100%

100%

100%

100%

100%

100%

93%

0.7

0

0.333333333333333

0.166666666666667

0.5

0

1

0.666666666666667

1

0.486486486486487

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 45: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 45 of 59

KPI 8. Percentage of clinics in each Region with evidence of completion of annual audit plans

North West L3=3, L2=1

N. Ireland L3=5, L2=0

Cheshire & M'side

L3=4, L2=0

Yorkshire L3=12, L2=6

South West

L3=12, L2=3

Northern L3=8, L2=0

Oxford L3=6, L2=1

Trent L3=15, L2=3

South Thames L3=24, L2=10

Wales L3=10, L2=3

North Thames L3=27, L2=4

Anglia L3=10, L2=6

Wessex L3=5, L2=0

West Midlands

L3=8, L2=0

National L3=149, L2=37

0% 20% 40% 60% 80% 100%1

0.166666666666667

0

0

0.333333333333333

0.7

1

0.75

0

0

0.432432432432432

33%

40%

50%

50%

58%

63%

67%

67%

83%

90%

93%

100%

100%

100%

77%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 46: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 46 of 59

KPI 9. Patient and Public Engagement (PPE) plan

• 4 components:1. Documented PPE plan for 2010• Engagement with service users about services used, and

services they wished to attend• Engagement with the public, including non-users of STI

services, when– Any redesign or major service development is planned– Finding out why some groups don’t use services

2. Implementation, any part of PPE plan3. Service user feedback arising from implementation4. Response to service user feedback

Page 47: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 47 of 59

KPI 9: Percentage of clinics in each Region with a documented plan for PPE for 2010

N. Ireland L3=5, L2=0

North West L3=3, L2=1

Trent L3=15, L2=3

Cheshire & M'side

L3=4, L2=0

South West

L3=12, L2=3

West Midlands

L3=8, L2=0

Anglia L3=10, L2=6

Wales L3=10, L2=3

North Thames L3=27, L2=4

South Thames L3=24, L2=10

Yorkshire L3=12, L2=6

Oxford L3=6, L2=1

Northern L3=8, L2=0

Wessex L3=5, L2=0

National L3=149, L2=37

0 0.2 0.4 0.6 0.8 1

20%

33%

40%

50%

50%

50%

60%

60%

63%

71%

75%

83%

88%

100%

62%

100%

100%

0%

17%

0%

100%

80%

33%

100%

54%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 48: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 48 of 59

KPI 9. Percentage of clinics in each Region with implementation of PPE plans

N. Ireland L3=5, L2=0

North West L3=3, L2=1

Trent L3=15, L2=3

Anglia L3=10, L2=6

Cheshire & M'side

L3=4, L2=0

Wales L3=10, L2=3

South West

L3=12, L2=3

Yorkshire L3=12, L2=6

West Midlands

L3=8, L2=0

South Thames L3=24, L2=10

North Thames L3=27, L2=4

Oxford L3=6, L2=1

Northern L3=8, L2=0

Wessex L3=5, L2=0

National L3=149, L2=37

0 0.2 0.4 0.6 0.8 1

20%

33%

47%

50%

50%

50%

58%

58%

63%

71%

74%

83%

88%

100%

63%

100%

100%

17%

0%

0%

33%

80%

100%

100%

54%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 49: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 49 of 59

KPI 9. Percentage of clinics in each Region with evidence of service user feedback arising from implementation

North West L3=3, L2=1

Anglia L3=10, L2=6

N. Ireland L3=5, L2=0

Cheshire & M'side

L3=4, L2=0

South West

L3=12, L2=3

Trent L3=15, L2=3

Wales L3=10, L2=3

West Midlands

L3=8, L2=0

South Thames L3=24, L2=10

North Thames L3=27, L2=4

Yorkshire L3=12, L2=6

Northern L3=8, L2=0

Oxford L3=6, L2=1

Wessex L3=5, L2=0

National L3=149, L2=37

0% 20% 40% 60% 80% 100%100%

17%

0%

67%

0%

80%

100%

33%

100%

51%

33%

40%

40%

50%

50%

53%

60%

63%

71%

74%

75%

88%

100%

100%

66%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 50: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 50 of 59

KPI 9. Percentage of clinics in each Region with evidence of response to service user feedback

Anglia L3=10, L2=6

Wales L3=10, L2=3

North West L3=3, L2=1

N. Ireland L3=5, L2=0

Cheshire & M'side

L3=4, L2=0

South West

L3=12, L2=3

West Midlands

L3=8, L2=0

Trent L3=15, L2=3

Yorkshire L3=12, L2=6

South Thames L3=24, L2=10

North Thames L3=27, L2=4

Northern L3=8, L2=0

Oxford L3=6, L2=1

Wessex L3=5, L2=0

National L3=149, L2=37

0% 20% 40% 60% 80% 100%17%

0%

100%

0%

67%

33%

80%

100%

100%

51%

30%

30%

33%

40%

50%

50%

50%

53%

58%

63%

67%

75%

100%

100%

58%

Level 3, n=149Level 2, n=37 Percentage of clinics meeting Standard

Page 51: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 51 of 59

Weaknesses

• Definitions• Validation of questions used for data collection• Under-representation of Level 2 services• Reporting bias• Representation of smaller services

Page 52: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 52 of 59

Summary National Performance, ranked by Level 3 KPI performance

Results received by clin-cians

within 7 days

Staff com-pe-

tency docu-

menta-tion

PN index-reported 0.6 (out-side Lon-

don)

Informa-tion

gover-nance

HIV test offer

Care pathways

Staff updat-

ing docu-

menta-tion

STI/HIV risk as-

sessment

PPE re-sponse to

patient feedback

PPE plan

Level 3 lead-

ership

PPE im-ple-

menta-tion

PPE pa-tient

feedback

Sexual history

Audit plan

PN index-reported 0.4 (Lon-

don)

48 hour access

Audit participa-

tion

HIV test uptake

0% 20% 40% 60% 80% 100%57%

62%39%

38%43%

89%65%

43%51%

54%81%

54%51%

59%43%43%

51%49%

70%

39%40%

43%50%50%50%

54%56%

58%62%62%63%

66%77%77%

85%86%

93%94%

Level 3 Level 2 Percentage of clinics meeting KPI standard

KPIs

Page 53: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 53 of 59

Key messages

• BASHH KPIs required further definition to allow auditing→BASHH Clinical Standards Unit

• KPIs are achievable by both Level 2 & Level 3 services

• Variable performance against all KPIs across regions

• Areas for improvement

Page 54: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 54 of 59

Areas for improvement

• Information governance– Training– Policies

• Time to test results– Laboratories/clinic systems

• Documentation of staff competency– Further national work in progress to describe

competencies• PN at 0.6 level (verification)– “Dedicated ” PN time– ePN (Ann Sullivan)

Page 55: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 55 of 59

Making the best use of audit results- 1

• Presenting/discussing in clinical meetings– Interventions needed to improve practice– Implementation

• Relating change to individual and team practice – Recording staff competency and updating– Structures and processes to follow up and document partner

notification outcomes– Re-design of case note or EPR recording– Improve uptake of HIV testing– Improve access to test results– Care pathways to connect all STI management providers to Level

3 services– Planning ahead for audit work

Page 56: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 56 of 59

Making the best use of audit results- 2

• Planning change management – Managers – Commissioners – Meetings: clinical, operational, clinical governance, staff

meetings, meetings

• Patients involvement in change management• Using the audit exercise for appraisal, revalidation

and performance review

Page 57: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 57 of 59

Change Champions

• Collect examples from clinics where the STIMS Audit data was used– To improve practice– How this was achieved– Share learning with other clinics, whose patients

might also benefit

Page 58: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 58 of 59

Acknowledgements

• Funding: Sexual Health and HIV, Department of Health

• Planning: BASHH National Audit Group• Participation: – Level 2 services: Andrea Duncan Sexual Health and

HIV Programme Manager, Department of Health– Level 3 services: BASHH Regional Audit Chairs– All clinics– Service managers

• Data collection and aggregation: Hilary Curtis

Page 59: Slide 1 of 59 2011 Audit against the Key Performance Indicators (KPIs) in the BASHH MedFASH STI Management Standards (STIMS) Updated presentation 20 Sept.

Slide 59 of 59

Acknowledgements: BASHH Regional Audit Chairs & Members

Chair Hugo McCleanVice Chair Chris CarneHon Sec Ann SullivanDirector of Development Anatole Menon-JohanssonBCCG Representative Phil KellBHIVA Representative Alison Rodger, Ed WilkinsScotland Daniel ClutterbuckWales Helen Baley, Sarah McAndrew, Carys KnapperNorthern Ireland Say QuahAnglia Raouf MoussaCheshire & Mersey Ravindra Gokhale

Essex Gail CroweNorth Thames Ann Sullivan, Alan SmithNorthern Sarup TayalNorth-West Ashish SukthankarOxford Gill WildmanSouth East Thames Cindy SethiSouth-West Zoe WarwickSouth-West Thames Steven EstreichTrent Jyoti DharWessex Neelam Radja, Leela SanmaniWest Midlands Sashi AcharyaYorkshire Amy Tobin-MammenCo-opted Members David Daniels, Nicola Low, Lindsay Emmett