實証醫學 Critical Appraisal In Evidence-Based Medicine
description
Transcript of 實証醫學 Critical Appraisal In Evidence-Based Medicine
實証醫學Critical Appraisal In
Evidence-Based Medicine
柯德鑫 , MD, PhD.奇美醫學中心 神經內科
Evidence-Based Medicine (EBM)
The consistent use of current best evidence derived from published clinical and epidemiologic research in management of patients, with attention to the balance of risks and benefits of diagnostic tests and alternative treatment regimens, taking account of each patient’s unique circumstances, including baseline risk, comorbid conditions and personal preferences.
Evidence-Based Medicine (EBM)
Meta-analysis is an overview in which uses quantitative methods to summarise the results from different clinical trials.
Odds Ratio describes the odds of an experimental patient suffering an adverse event relative to a control patient.
What evidence-based medicine is:
The practice of EBM is the integration of individual clinical expertise
with the best available external clinical evidence
from systematic research.and
patient’s values and expectations
What is Evidence-Based Medicine?
See a patient Ask a question Seek the best evidence for that
question Appraise that evidence Apply the evidence Monitor the change
健康狀態
病人
感覺不適
形成最可能的假設診斷
證實假設診斷
病史詢問理學檢查臨時診斷思考
診斷條件:特異性高的發現統計上相關性相近的病態生理 / 病理機制
相關的檢驗與檢查
病態生理變化
疾病
醫師
Evidence-Based Medicine
Ask a question Acquire some articles Appraise the evidence Apply the findings Assess your performance
證據等級之定義
第一級:最高級證據 (Level Ⅰ) 來源 (1) 隨機雙盲試驗,並有適當病人數所原發性終點分析; (2) 適當執行的優質隨機分組試驗之巨集分析第二級:中級證據 (Level Ⅱ)來源 (1) 隨機非雙盲試驗 (2) 小規模隨機試驗 (3) 大規模隨機試驗並有事先界定的次發性終點分析第三級:次級證據 (Level Ⅲ)來源 (1) 前瞻性病例系列,並有同時或前後對照 (2) 隨機試驗之事後分析第四級:證據未明 (Level Ⅳ)來源 (1) 小規模病例系例無對照或病例報告 (2) 雖然沒有對照試驗之科學證據,但專家普遍同意
Critical appraisal is the process of systematically examining research evidence to assess its validity, results and relevance before using it to inform a decision.
What is critical appraisal?
Critical Appraisal
The process of deciding whether a piece of research can help you in answering your clinical question. Three questions you need to ask about any kind of research:
1. Is it valid? 2. Is it important? 3. Is it applicable to the patient?
Critical Appraisal Skills
VIP Valid: 研究方法的探討 Important: 結論的分析 Practice: 如何運用來照顧病人 RCT/Cohort study/Case-control
study
· By Expertise -Secondary journal -CAT(critical appraisal topics) -Cochrane database -JAMA & BMJ Guide -Basic Clinical Statistics·Have an appraisal course·EBM style journal meeting
How to Appraise Evidence?
Evaluation of A Diagnostic Test
Sensitivity is the proportion of people with the disease who have a positive test.
Specificity is the proportion of people free of the disease who have a negative test.
Positive Predictive Value (+PV) is the proportion of people with a positive test who truly have the disease.
Negative Predictive Value (-PV) is the proportion of people with a negative test who are free of the disease.
Evaluation of A Diagnostic Test
SnNout when a sign/test has a high sensitivity, a negative result rules out the diagnosis; e.g. the sensitivity of a history of ankle swelling for diagnosing ascites is 92%, therefore if a person does not have a history of ankle swelling, it is highly unlikely that the person has ascites.
SpPin when a sign/test has a high specificity, a Positive result rules in the diagnosis; e.g. the specificity of fluid wave for diagnosing ascites is 92%. Therefore, if a person has a fluid wave, it is highly likely that the person has ascites.
Critical Appraisal for Diagnosis
Are the results of this diagnostic study valid?
Are the valid results of this diagnostic study important?
Can you apply this valid, important evidence about a diagnostic test in caring for your patient?
Are the results of this diagnostic study valid?
Was there an independent, blind comparison with a reference ("gold") standard of diagnosis?
Was the diagnostic test evaluated in an appropriate spectrum of patients (like those in whom we would use it in practice)?
Was the reference standard applied regardless of the test result?
Was the test (or cluster of tests) validated in a second, independent group of patients?
Are the valid results of this diagnostic study important?
Are the valid results of this diagnostic study important?
Sensitivity = a / (a+c) = 371/809 = 90 %
Specificity = d / (b+d) = 1500/1770 = 85 %LR+ = sens / (1-spec) = 90/15 = 6LR- = (1-sens) / (spec) = 10/85 = 0.12Positive Predictive Value = a / (a+b) = 731/1001 = 73 % Negative Predictive value = d / (c+d) = 1500/1578 = 95 % Prevalence = (a+c) / (a+b+c+d) = 809/2579 = 32 %Pre-test odds = prevalence / (1-prevalence) = 31/69 = 0.45Post-test odds = pre-test odds * LRPost-test Probability = post-test odds / (post-test odds + 1)
Can you apply this valid, important evidence about a diagnostic test in caring
for your patient?
Is the diagnostic test available, affordable, accurate and precise in your setting?
Can you generate a clinically sensible
estimate of your patient's pre-test probability (from practical data, from personal experience, from the report itself, or from clinical speculation)?
Can you apply this valid, important evidence about a diagnostic test in caring
for your patient?
Will the resulting post-test probabilities affect your management and help your patient? (Could it move you across a test-treatment threshold? Would your patient be a willing partner in carrying it out?)
Would the consequences of the test help your patient?
Evidence-Based Medicine (EBM) in Clinical Trial
Number Needed to Treat (NNT) is the number of patients who need to be treated to prevent one bad outcome.
Number Needed to Harm (NNH) is the number of patients under treatment that is need to develop one bad outcome or adverse effect.
Evidence-Based Medicine (EBM) in Clinical Trial
Intention-to-treat analysis: including all patients in the analysis who are randomized to treatment, regardless of what occurs during the clinical trial.
Adherence-to-protocal (On treatment) analysis: analyzing patients based on the treatment they received finally.
Therapies
To treat or not to treat? Validity Importance Applicability
Therapies
Validity• Was it randomised?• Was the allocation concealed?• Were the all the subjects analysed
correctly?• Was it blinded?• Were the groups similar?
Therapies
Importance• What were the results?• Over what time period?• With what precision?
Therapies
Number needed to treat Relative risk reduction Absolute risk reduction Event rates
Therapies
Event rates• n with event / total
Control event rate (CER) Experimental event rate (EER)
Therapies
Absolute risk reduction• difference in two event rates• CER - EER = ARR
Relative risk reduction• proportion of control rate• CER-EER / CER = RRR
Therapies
Number needed to treat• number of extra patients you need to
treat to prevent one bad outcome• 1 / ARR = NNT
Therapies
95% confidence interval• range within which the true value falls
with 95% confidence
• use computer (e.g. CATMaker)
Occurrence of death, stroke, or other major complications
Adverse eventsNumber
needed to treat NNT
Patient status at entry
Placebo P
Active A
RRR ARR 1/ARR=NNT
Prior target organ damage
.22 .08 64% .14 1/.14=7
No prior organ damage
.10 .04 60% .06 1/.06=17
THERAPY WORKSHEETCitation:
Are the results of this single preventive or therapeutic trial valid?
Was the assignment of patients to treatments randomized?-and was the randomization list concealed?
Were all patients who entered the trial accounted for at its conclusion? -and were they analyzed in the groups to which they were randomized?
Were patients and clinicians kept “blind” to which treat-ment was being received?
Aside from the experimental treatment, were the groups treated equally?
Were the groups similar at the start of the trial?
THERAPY WORKSHEET Are the valid results of this randomized trial
important?
Occurrence of diabeticneuropathy
Relative RiskReduction
RRR
Absolute RiskReduction
ARR
Number Neededto Treat
NNT
Usual InsulinControl Event
RateCER
IntensiveInsulin
ExperimentalEvent Rate
EER
CER - EERCER
CER - EER 1/ARR
9.6% 2.8% 9.6% - 2.8% = 71%
9.6% 9.6% - 2.8% = 6.8%
(4.3% to 9.3%) 1/6.8% = 15 pts,
(11 to 23)
SAMPLE CALCULATIONS:
THERAPY WORKSHEET SAMPLE CALCULATIONS:
YOUR CALCULATIONS:
Relative RiskReduction
RRR
Absolute RiskReduction
ARR
Number Neededto Treat
NNT
CER EER CER - EER
CER CER - EER 1/ARR
95% Confidence Interval (CI) on an NNT = 1 / (limits on the CI of its ARR) =
%4.2/711
972.0028.0
730
904.0096.096.1/
.exp#
)1(
.#
)1(96.1/
ptserof
EEREER
ptscontrolof
CERCER
Therapies
Application• Can it be applied to my patient?• Can it be done here?• How do patient values affect the
decision?
Therapies
Is it valid? Is it important?
• NNT for whatover how longwith what precision
Does it apply?
Six guides to distinguish useful from useless or even harmful therapy
1. Was the assignment of patients to treatments really randomized ?
2. Was all clinically relevant outcomes reported ?3. Were the study patients recognizably similar to your own ?4. Were both clinical and statistical significance considered ?5. Is the therapeutic maneuver feasible in your practice ?6. Were all patients who entered the study accounted for its
conclusion ?
The likelihood of help vs. harm (LHH)
In applying a SR or RCT to an individual patient, we need to consider:
•our patient's risk, relative to patients in the trial, of the event we hope to prevent with the treatment: ft
•our patient's risk, relative to patients in the trial, of the side-effect we might cause from the treatment: fh
•our patient's perception of the severity of the event we're trying to prevent relative to the side-effect we might cause: s
The likelihood of help vs. harm is(1 /NNT) x ft x s vs. (1/NNH)x fh
For example, suppose we're applying a trial with an NNT of 9 and an NNH of 12 and we think our patient is at just half the risk of the event but at twice the risk of the side-effect, then the "raw" LHH before we adjust it for our patient's perception of relative severity is 1/9 x 0.5 vs. 1/12 x 2 = 1/18 vs. 1/6, or three times as likely to harm vs. help the patient. However, if our patient regards the severity of the event that the treatment might prevent to be six times worse than the side-effect it might cause, then the final LHH = 1/18 x 6 vs. 1/6, or two times as likely to help vs. harm
Should these valid, potentially important results change the treatment of our patient?
1. Is our patient so different from those included in the study that its results don’t apply?
2. What is our patient’s risks of the adverse event? What is our patient’s potential benefit from the therapy?
3. What are our patient’s preferences, concerns and expectations from this treatment?
4. What alternative treatments are available?
Are the results of this systematic review of therapy valid?
1. Is this a systematic review of randomized trials?
2. Does it include a methods section that describes:
(a) finding and including all the relevant trials?
(b) assessing their individual validity?
3. Were the results consistent from study to study?
(4. Were individual patient data used in the analysis (or aggregate data)? )
Is the systematic review important?
Are the valid results of this systematic review important?
Translating odds ratios to NNTs
Are the valid, important results of this systematic review applicable to our
patient?
1.Is our patient so different from those in the study that its results cannot apply?
2.Is the treatment feasible in our setting?
3.What are our patient's potential benefits and harms from the therapy?
4.What are our patient's values and preferences for both the outcome we are trying to prevent and the side-effects we may cause?
Are the recommendations in this guideline valid?
1. Did its developers carry out a comprehensive, reproducible literature review within the past 12 months?
2. Is each of its recommendations both tagged by the level of evidence upon which it is based and linked to a specific citation?
Are the results of this prognosis study valid?
1. Was a defined, representative sample of patients assembled at a common (usually early) point in the course of their disease?
2. Was patient follow-up sufficiently long and complete?
3. Were objective outcome criteria applied in a “blind” fashion?
4. If subgroups with different prognoses are identified, was there adjustment for important prognostic factors?
5. Was there validation in an independent group (“test set”) of patients?
Are the valid results of this prognosis study important?
1. How likely are the outcomes over time?
2. How precise are the prognostic estimates?
Can we apply this valid, important evidence about prognosis in caring
for our patient?
1. Were the study patients similar to our own?
2. Will this evidence make a clinically important impact on our conclusions about what to offer or tell our patient?
Are the results of this harm/etiology study valid?
1. Were there clearly defined groups of patients, similar in all important ways other than exposure to the treatment or other cause?
2. Were treatments/exposures and clinical outcomes measured in the same ways in both groups (was the assessment of outcomes either objective or blinded to exposure)?
3. Was the follow-up of study patients complete and long enough?
Are the results of this harm/etiology study valid?
4. Do the results satisfy some “diagnostic tests for causation”?
• Is it clear that the exposure preceded the onset of the outcome?
• Is there a dose-response gradient?• Is there positive evidence from a “dechallenge-
rechallenge” study?• Is the association consistent from study to
study?• Does the association make biological sense?
Are the valid results from this harm/etiology study important?
Adverse outcome Totals
Present (case)
Absent (control)
Exposed to the
treatment
Yes (cohort) a b a+bNo (cohort) c d c+d
Totals a+c b+d a+b+c+d
In a randomized trial or cohort study:Relative risk (RR) = [a/(a+b)]/[c/(c+d)]In a case-control study: relative odds (RO) = ad/bcTo calculate the NNH for any OR and PEER:
Are the results of this clinical decision analysis valid?
1. Were all the important therapeutic alternatives (including no treatment) and outcomes included?
2. Are the probabilities of the outcomes valid and credible?
3. Are the utilities of the outcomes valid and credible?
4. Was the robustness of the conclusion tested?
Are the valid results from this decision analysis important?
1. Did one course of action lead to clinically important gains?
2. Was the same course of action preferred despite clinically sensible changes in probabilities and utilities?
Are the valid, important results of this decision analysis applicable to
our patient?
1. Do the probabilities apply to our patient?
2. Can our patient state his/her utilities in a stable, usable form?