Adaptive Trials and the importance of firewalls …….can ... · – Sponsor audits and UAT Case...
Transcript of Adaptive Trials and the importance of firewalls …….can ... · – Sponsor audits and UAT Case...
www.cytel.com
Adaptive Trials and the importance of firewalls…….can technology play a role?
Judith Quinlan VP; Adaptive Clinical [email protected]
• More complex logistics and the importance of firewalls• Case study
– Background– Challenges
• Design and logistics• Managing Firewalls
– Combining technology and SOPs– Why we have chosen this approach
• Summary
Today
ADWG KOL series ; October 20102
Adaptive “by Design”
Strategic Goal supported by
clearly defined: Objectives &
decision rules
Possible design options selected
Including traditional approach
Simulation to compare design Performance
(within context of CDP)
Simulation adding value by helping to quantifydecision process to
select the design best able to deliver study
objectives
Ensuring appropriateFirewalls in place, more complex logistics
3
Ensuring appropriateFirewalls in place, more complex logistics
n
3ADWG KOL series ; October 2010
• General feeling that guidance presented no surprises
• Positive acceptance coupled with understandable cautions
• In particular concerns relating to documentation regarding “who saw what and when”
• FDA noted the potential to “Develop software for audit trails to document design modifications”
• Also discussion comments made by Bob O’Neil that the FDA wants documentation of who knew what when during the trial conduct prior to unblinding
• John Jenkins’s closing remarks also spoke to the need to build trust and verify
March meeting on:FDA Adaptive Design Guidance
ADWG KOL series ; October 20104
Case Study
ADWG KOL series ; October 20105
• Compound– Safety issues considered unlikely
• Compound already approved in another indication– Now being studied in area of “high unmet medical need”
• Development strategy– Originally phase II followed by two phase III trials
• Traditional phase II originally planned– Already a large trial
• Later redesigned as adaptive trial – But still considered phase II trial
• Agency feedback proposed the possibility, considering this was already a large trial, of converting it to a confirmatory trial “provided”– Able to demonstrate trial is an AWC trial– Demonstrate adequate firewalls were in place
Case Study : Background
ADWG KOL series ; October 20106
• Required further increase in “potential” size of the trial• However financial risk considered worth taking• “Extensive” discussions and simulation work completed in respect
to decision rules for dose selection (s)• DESIGN Challenges :
– Selecting not just the “best” dose, but ensuring we have the minimum effective dose (MED) question covered
• Logistical challenges– Time!!
• Simulations (including custom software & validation)• Documentation!
– Protocol, DMC charter, Simulation report• Establishing firewalls
Case Study : Trial Redesigned again
ADWG KOL series ; October 20107
• Two stage adaptive study design• Part A
– Placebo & 3 active doses• Interim Analysis
– Select either one or two doses (plus placebo) to carry forward into Part B
– Possible sample size increase dependent on results (max cap extra 300 subjects)
• based on Promising Zone concept (ref : April KOL presentation by Cyrus Mehta)
– Guidelines for dose selection predefined.• IDMC expected to combine recommendation with clinical
judgment in making final recommendation for Part B• Part B
– End of study analysis combining data from Part A & B
Case Study : Final Study Design
ADWG KOL series ; October 20108
• Protecting the blind & trial integrity…what were we going to do about firewalls??
– Ensuring restricted access to information during the interim analysis process
• Decision• Do we use our standard approach for managing the interim workflow and
notification process OR• Use newly developed technology for managing the interim workflow
process
• Challenges for software development team : Time!– Focus development on features necessary for this interim – IT Set up requirements for hosted solution– System Validation (including CRF Part 11 compliance)– User Testing– Documentation (user guide)– Sponsor audits and UAT
Case Study : Firewalls
ADWG KOL series ; October 20109
Banks have a responsibility to meet the expectations of their customers.
To put systems &processes in place to provide
Security
Protection
Safe guards
10ADWG KOL series ; October 2010
•Restricted access to the bank vault
• Access overseen & administered by limited bank personnel
Further security protection through restricted access for individuals
Implementing Security & Protection
11ADWG KOL series ; October 2010
• Secure logins• Dashboard of personal accounts
– Actions:• Status review• Fund transfers
– Audit tracking• All transactions time and date stamped• Know when we accessed our accounts & what changes we
made• Account Transparency
Beyond the vault : web based banking
ADWG KOL series ; October 201012
ADWG KOL series ; October 201013
Borrowing from the banking industry to create Firewalls for
Adaptive Designs
DMCDrug Supply
Randomization coordinator
Independent Statistician
Independent PK
Project team
Sponsor Company
ACES
Blinded summaries for project team
Adaptive Design Implementer (ADI)
14ADWG KOL series ; October 2010
ACES
CRO/data Management
1: test secure
Data Transfer
2: Produces validatedSAS program that will be used to generate
response file for primary endpoint as
defined in interim SAP
IDMC secretary & programmer
IDMC Independent Statistician
ACES Design Implementer
(ADI)
3: uploads into system initial
randomization list, design engine &
program that will produce response file. Sets security access &
build study
2: IDMC secretary uploads study documentsIncluding protocol, charter
IDMC
6: test notifications sent to DMC members. DMC members test login to ACES
4: Programs to produce semi-blind safety & secondary analysis reportscompleted and QC’d by Independent statistician
Sponsor contact(blinded
summaries?)
IVRS(new rand list or treat prob)
DrugSupply
NotifiedOf
change
6: test runs completed including notifications
that reports are ready for collection
5: Automated event driven data extraction tested
15Interim Workflow tested pre-interim
Case Study
ADWG KOL series ; October 2010
ACES
CRO/data Management
1: secure Data
TransferIDMC secretary & programmer
IDMC Independent Statistician
ACES Design Implementer
(ADI)
3 : ACES is set to run to produce primary
report automatically
5: IDMC secretary uploads semi –blind interim safety & secondary reports into ACES
IDMC
6: IDMC collects reports from ACES’s DMC portal after ACES automatically notifies DMC members that
Reports are ready for collection
4: Produces semi-blind safety & secondary analysis reportsas defined in interim SAP & stored in secure area at ISC. Theses are then
reviewed by IDMC Independent statistician or designee
7: at request of IDMC Chair, independent statistician
verballyunblinds IDMC
8: DMC decision Is converted into
reports for communication to
Sponsor, IVRS & drug supply. Reports are
uploaded into ACES by IDMC
chair or secretary Sponsor contact
(blinded summaries)
IVRSDrugSupply
9: Once loaded & approved by DI, ACES automatically notifies IVRS, drug supply &
Sponsor their reports are ready for collection
8: “if” IDMC recommendation is to stop the trial for serious safety
IDMC chair contacts Sponsor
3: Automated data extraction
16Interim Workflow
ACES Design Implementer
(ADI)
3a ; 5a ; 8a DI approves reports prior to distribution & notification
Case Study
ADWG KOL series ; October 2010
ADWG KOL series ; October 201017
ADWG KOL series ; October 201018
ADWG KOL series ; October 201019
This is a view of the execution summary Showing the results of two test runs and one active run
Screen view for ADI
ADWG KOL series ; October 201021
Once ADI accepts & delegates a report a notification is sent to the relevant user that a report is in the system for them to collect
Here a DMC has logged on to see the “system generated “ report is available for review & download
DMC member logged on
ADWG KOL series ; October 201022
The ADI selects and reviews the report .The ADI must accept the report before the system sends out notifications
ADI can allow reports to be visible to others but by default only ADI & DMCHave full access to documents
ADWG KOL series ; October 201023
Two other reports have been uploaded into the system by the ISC & accepted by the ADI. These reports are now visible to the DMC
So here a DMC member has logged on to see these two reports are available for review & downloading by clicking on the link
DMC member logged on
ADWG KOL series ; October 201024
However here the Sponsor has logged on & because no documents have been assigned to the Sponsor, no documents are available
ADWG KOL series ; October 201025
In this snapshot of the audit trail we can see who has been assigned access to the system
Here we can see that a ISC report & DMC min have been uploaded by the ISC and then viewed by a DMC member
• Also a need for SOPs as part of the process• Building on what already exists
– Secure FTP transfer for data and initial randomization list– Internal SOPs governing
• separation of blinded and unblinded staff involved on project– file storage
• Usual confidentiality agreements for DMC members• Audit tracking for access to clinical data base
– Preventing back-door access – However, such audit tracking already exists already for most
companies
Combining technology with SOPs
ADWG KOL series ; October 201026
• Many approaches for managing firewalls exist today
• However for us at least, we believe the combination of technology and SOPs provides us with the secure environment we need for managing the interim analysis workflow process
– A system with audit tracking capabilities & report summary capabilities to address the “who saw what and when” question
• Benefits also for managing the increasing number of adaptive trials we see
– “standardization” for how we present audit trail documentation to agencies• Hopefully to help make agency review easier
– Scalability through workflow automation• Overcoming the high manual workload involved in the interim analysis
process• Allowing a experienced statistician to manage several studies
– Particularly when frequent interims are required
Why we have chosen this approach
ADWG KOL series ; October 201027
• The release of the draft FDA guidance on Adaptive Trials has been a significant milestone– Particularly for those who have worked hard over the past
decade to promote the responsible use of adaptive trials• However the guidance document alone will not guarantee the future
success of adaptive trials• Industry has to continue to build “trust” that we can not only design
adaptive trials that satisfy acceptable statistical criteria, but we can also effectively manage the execution of these trials, without compromising the integrity of the trial
• Acceptable Firewalls are critically important in achieving this goal• Many approaches exist today for managing Firewalls, & I have
shown you just one approach
Summary
ADWG KOL series ; October 201028