C4 pechlivanoglou

“New methods in generating evidence for everyone: Can we improve evidence synthesis

approaches?” Network Meta-Analyses and Economic Evaluations

Petros Pechlivanoglou PhD

Child Health Evaluative SciencesThe Hospital For Sick Children, Toronto

Institute for Health Policy Management and EvaluationUniversity of Toronto, Toronto CADTH 2016

Disclosure

• No funding received for this work• Talk relies in part on the work of Cooper et al (2015)

and Dias et al (2013).

AcknowledgementsMSc students: Troy Francis Yasmin Saeed

Dias,S, et al. "Evidence synthesis for decision making 3 heterogeneity—subgroups, meta-regression, bias, and biasadjustment." Medical Decision Making 33.5 (2013): 618-640.

Cooper et al RFP Topic: Use of Network Meta – analysis to Inform Clinical Parameters in Economic Evaluations; white paper, CADTH (2015)

Aims

• Usefulness of relying NMA in EE

• Main concepts in embedding NMAs in EE

• Preliminary results of literature review

• Some recommendations on using NMA evidence in EE

Introducing NMA in EE

• Evidence-based decision making: evaluation of both cost and health impact of all relevant treatments.

• Ideally, optimal decisions are based on multi-arms RCTs, including all relevant treatments.

BUT• High costs and regulatory issues preclude such RCTs.

• New health technologies compared to standard care or placebo.

NMAs allow the economic evaluation of all treatment options using comparative effectiveness estimates

NMA development pushed by EE !!

Uncertainty, NMA and EENMAs help us to more properly characterize uncertainty in treatment

comparisons

The importance of accurate uncertainty estimate:– Non-linearity of decision models implies inaccurate uncertainty estimates

result to bias• E(g(x)) ≠ g(E(x))

– Accurate characterization of uncertainty is important for decision making

– Value of Information (VOI) increasingly used as a method to guide investments and research – VOI relies on accurate estimates of uncertainty.

HTA Process• Clinical Evidence

– Identify relevant evidence (Systematic review)– Synthesize Evidence (Meta – Analysis / NMA / narrative)– Interpret / Appraise

• Economic Evidence– Identify relevant EE evidence– Conceptualize model– Identify input – Execute model– Interpretation / Appraise

• ELSI

Conceptualizing an NMA-EE model

• Comparators

• Study Design

• Outcomes

• Model Structure


Comparators– Wider sets than those clinically meaningful to take

advantage of indirect comparisons– Lumping of Comparators

• Differences in dose/ treatment durations / administration

Study DesignRCTs for treatment effect BUTOpen-label extensions for long-term follow up (!)RCT/ surveys on HRQoL (!)

(i) : new methods in development


Outcomes– Discrepancy between what is clinically and what

economically relevant in evidence synthesis

– Sparsity / large presence of zeros

– Surrogate endpoints

– Correlated outcomes


Model structure

BAYESIAN• One step NMA – EE (NMA and EE estimated together)• Two Step NMA – EE (NMA posterior embedded in EE)

FREQUENTIST• Two Step NMA – EE (assuming Multivariate distribution)• Two Step NMA – EE (Ignoring Correlations)• Separate MAs – EE

Conceptualizing an NMA-EE model The Baseline treatment

NMAs: comparing treatments relative effectiveness.EE: estimates of absolute risk ( the transition probabilities)

Estimate of baseline risk required: • Standard of care• Alternatively Placebo/No treatment (not always

appropriate)• The most tested treatment in the network


The Baseline treatment

NMAs focus on comparing treatments with respect to their relative effectiveness.

Decision models require estimates of absolute risk ( the transition probabilities)

To make use of NMA results in EE an estimate of baseline risk is required.

Lit. review of NMA in EE

What is “current practice” in NMA – EE integration?

• Literature review for applications of NMA-informed EE• Searched through MEDLINE, EMBASE, Pubmed, NHS

EED• Non systematic component: Google scholar, pearl

growing search.

• Outcomes / Design / Comparators / Model structure

Preliminary results:Studies Identified 107 (39 analyzed so far)

NMA EE

No. of treatments 6.4 4.7

No. of outcomes in 3.3 3.3

No. of studies in NMA 26.6

Inconsistency check 21.6%

Software 50% WinBUGS10% OpenBUGS3% EXCEL

8 EXCEL2 TreeAge2 Str8

Bayesian framework 70%

Baseline 40% from NMA

Control over model 67%

Type of model 60% MTC only 18% ITC/MTC22% TC

51%State transition17% decision tree1 study AUC, DES

Country 64% UK

Evidence synthesis in QoL 13%

Incorporating uncertainty around NMA parameters

“SE to match 95% confidence intervals provided by ITC” “the 95% confidence limits from the MTC were used to

estimate the variability”

“ PSA values sampled from the WinBUGS CODA output“

“…were defined directly from values recorded in the 10,000 iterations of the NMA”

Conclusion• NMAs can provide information that is compatible with the

needs of EEs

• Care needs to be taken on choosing the appropriate NMA method as potential for large variations exists

• Preliminary results of review show mixed quality on methods used to incorporate NMAs in EE

• Standardizing practice (guidelines update /software development) and developing know-how needed

C4 pechlivanoglou

Healthcare

Transcript of C4 pechlivanoglou