Critical appraisal100-2-25 EBM Asking answerable question Finding the best evidence Appraising the evidence Making a decision Evaluating your performance45

6ABCD 7 (Systematic review) meta-analysis () 8 (Ramdomized Control Studies)

9 10Advantages: Disadvantages: 11

(Crossover Design)

12 13Advantages: Disadvantages: washout period 14 (Cohort Study)

15 16Advantages: timing and directionality of eventsRCT Disadvantages: 17 (Case-Control Studies)

18 19Advantages: .Disadvantages: 20 (Cross-Sectional Survey)

21 22Advantages: Disadvantages: bias (for therapy)

: Validity (Reliability) / Can we believe it ?: Importance (Impact) We believe it ! But does it matter?: Practice (Applicability) If we believe it does it apply to our patients?

Different study design has different level of evidence, the best quality study design for each question: Frequency Prevalence (case control), Incidence (cohort) Etiology / Harm cohort Diagnosis case control Prognosis cohort Treatment / Intervention RCTQuestion type()Study design ()Diagnostic test Prospective, blinded cross-sectional study comparing with gold standardPrognosisCohort study > Case control study > Case series study > > EtiologyCohort study > Case control study > Case series study > > TherapyRandomised control trial (RCT)PreventionRandomised control trial (RCT)Cost effectivenessEconomic analysis

H0H0H0H0H0H0H0

: H0 H0 = 0.05P H0 H1 P H0 P

H0 H1H0 H0 I ( ) (1- )

H0 (1-) II ()

H0 H1

I II Power of a test: 1-

(point estimation): : 20117120123.62 cm, s= 4.75 cm (1-) (1- ) (confidence interval, CI)= 0.05, 100955

Diagnostic test disease non-disease positive A B negative C DSensitivity: A/A+CSpecificity: D/B+DPositive predictive value(PPV): A/A+BNegative predictive value(NPV): D/C+DIn therapy study Relative risk (RR) : Odd ratio Relative risk reduction (RRR) Absolute risk reduction (ARR) Number needed to treat (NNT): 1/ARR Relative risk()/ Risk ratio () A 100 , Relative risk: 1/100= 0.01B 50 , Relative risk: 1/50= 0.02BA risk ratio= 0.02/0.01= 2

Odds () / Odds ratio ()Odds (): A, 100 , odds ()= 1:99=1/99= 0.0101odds ratio( ): Odds ratio: : A 0.5 B 0.2; , A 1(=0.5/0.5) B1/4 (=0.2/0.8)AB(odds ratio), : 1 41

Effect of Therapy (300700800200 (EER) = 300 / (300+700) = 30%(CER) = 800 / (800+200) = 80%

(Risk Ratio, RR) = EER / CER = 0.3 / 0.8 = 0.375

(ARR) = CER EER = 80% - 30% = 50%

(RRR) = CER EER / CER = (80% - 30%) / 80% = 62.5%

EER: Experimental event rateCER: Control event rateARR: Absolute risk reductionRRR: Relative risk reductionNumber Needed to Treat (NNT)NNT = 1 / ARR or 100 / ARR (%) VSHow large was the treatment effect Relative Risk ()

RR = 0.10 / 0.15 = 0.67RR=1RR1RR 80%)3. Were all patients analyzed in the groups to which they were randomized? (ITT, Intension To Treat analysis)4. Were patients and clinicians kept blind to treatment?5. Were groups treated equally, apart from the experimental therapy?6. Were the groups similar at the start of the trial?Validity: selection bias, information bias, confoundingReliability of measurement: intraobserver, interobserver, internal consistencySD, variance, 95% CI (confidence interval), p valueImpact (size of the effect):NNT (number needed to treat) = 1/ARR (absolute risk reduction)NNH (number needed to harm) = 1/ARI (absolute risk increase)Applicability (usefulness in our clinical practice)Integrating the evidence with our clinical expertise and patients values and preferences.48PECOTPECOT RecruitmentAllocationMaintenance

Measurement of outcomes blind or objectiveThe RAMMbo* acronym: assessing study bias

49Cover RAMMbo when I introduce critical appraisal

ParticipantsIntervention Group (IG) & Comparison Group (CG)Outcome

IGCG+-+-DCBARepresentative?Allocation?Selection?Maintenance of allocation?VALIDITY

Maintenance? treated equally? compliant?

Measurements blind subjective? OR objective?

QUESTION:Allocation? Randomised? comparable groups?Measurement of outcomes?DESIGN:5050

Critical Appraisal Skills Programme (CASP) 10 questions to help you make sense of randomised controlled trials How to use this appraisal tool Three broad issues need to be considered when appraising the report of a randomised controlled trial: Is the trial valid? What are the results? Will the results help locally?

1. Did the study ask a clearly-focused question? -> Yes, Cant tell, No Consider if the question is focused in terms of: the population studied the intervention given the outcomes considered 2. Was this a randomised controlled trial (RCT) and was it appropriately so? -> Yes, Cant tell, No Consider: why this study was carried out as an RCT if this was the right research approach for the question being asked 3. Were participants appropriately allocated to intervention and control groups? 4. Were participants, staff and study personnel blind to participants study group? 5. Were all of the participants who entered the trial accounted for at its conclusion? 6. Were the participants in all groups followed up and data collected in the same way? 7. Did the study have enough participants to minimise the play of chance? 8. How are the results presented and what is the main result? 9. How precise are these results? 10. Were all important outcomes considered so the results can be applied? Will the Results Help Me in Caring for My Patients ? Are the people in the study like my patient ?AgeGeneral state of healthType and severity of disease processTime in the course of the diseaseDid the study cover all aspects of problem ?eg treatment effect on symptom relief, quality of life, mortality etcIs the treatment feasible in my setting ?Will the potential benefits of treatment outweigh the potential harms of treatment for my patients ?Does it suggest a clear and useful plan of action ?Help to clarify a patients prognosisSuggest a useful plan to improve patients state of healthMeta-analysis ()If the results of the individual studies are similarGives weighted values to each of the individual studies according to their sizeThe individual results of the studies expressed in a standard way, eg relative risk, odds ratio or mean difference between groupsResults are displayed in a figure called forest plot Exploring heterogeneity ()Cochran chi-square ( Cochran Q test)Definite heterogeneity ()Cochran Q ( P < 0.1 )Possible heterogeneity ()Cochran Q is not statistically significantCochran Q / degrees of freedom (Q/df) > 1Heterogeneity unlikely ()Cochran Q is not statistically significantQ/df < 1Systematic reviewsMeta-analysisForest PlotComparison: Treatment VS PlaceboOutcome: Effect of treatment on mortality Treatment Control OR Weight ORStudy n/N n/N (95% CI) % (95% CI)

Brown 1998 24/472 35/499 9.6 0.71(0.42, 1.21)

Geoffrey 1997 120/2850 182/2838 51.8 0.64(0.51, 0.81)

Mason 1996 56/2051 84/2030 24.4 0.65(0.46, 0.92)

Peters 2000 5/81 4/78 1.1 1.22(0.31, 4.71)

Scott 1998 31/788 46/792 13.1 0.66(0.42, 1.06)

Total (95% CI) 236/6242 351/6237 100.0 0.66(0.56, 0.78).1.21510Favors treatmentFavors controlTest for heterogeneity chi-square=0.92 df=4 p=0.92Test for overall effect z=-4.82 p