It is striking how collaborative the real-world evidence (RWE) ecosystem is: a distributed community with multiple nodes of authority working toward a coherent, transparent approach to real-world data and evidence generation that can enhance a global industry.
“Big data has changed everything,” US FDA Office of Medical Policy associate director for RWE analytics John Concato observed at a recent Drug Information Association meeting on RWE. The resulting conversations have coalesced around the identification of “common challenges,” key among them “attention to the fundamental principles regarding data, study design and analysis.”
Making progress against the challenges will require “communication and collaboration with regulatory agencies and other stakeholders,” Concato emphasized.
Across the globe, regulatory authorities are diligently erecting RWE frameworks in their remits. The International Coalition of Medicines Regulatory Authorities (ICMRA) is preparing to develop common guidance. (Also see “Global Regulators Join Hands To Boost RWE Use In Decision-Making” – Pink Sheet, 26 Jul, 2022.)
The European Union is setting up the European Health Data Space (EHDS). The US FDA issued a draft framework for its RWE program in December 2018 and followed up with energetic production of guidance.
Most recently, the FDA launched the Advancing RWE Program on 19 October 2022 to enable agency advice before protocol development and study initiation. (Also see “RWE Program Offers Sponsors Chance At Novel Uses, But In Exchange For Disclosing Proposals” – Pink Sheet, 19 Oct, 2022.)
But if government bodies are supplying the bricks for a new regulatory edifice, the full spectrum of the drug development world is involved in providing the mortar: common terminology, standards and practices for the identification, collection and interpretation of data from real-world sources like electronic health records (EHRs).
The DIA meeting on RWE in San Diego, Calif. on 10-11 November 2022 highlighted many of the collaborative efforts underway, from high-level policy plans to efforts to standardize data reporting and establish a sound academic and theoretical basis for the RWD/RWE field.
EMA vs. FDA Staffing Focus
“I’m very impressed with the concordance at this meeting,” Concato commented as the conference drew to a close.
“Over time the whole RWE ecosystem has upped its game,” Concato’s predecessor David Martin agreed. Martin is now VP clinical safety and risk management at Moderna, Inc..
While the ecosystem is global, distinct regional differences remain, like the strict privacy laws in Europe and Asia that are driving interest in synthetic data approaches to RWE.
The EMA, which must coordinate with national regulatory authorities in EU member states, has put more effort into governance than the FDA, Martin said. EMA is “great at public communication of policies, but moves more slowly than FDA,” he observed. “FDA is more opaque; it doesn’t have to worry about internal governance.” While the EMA has more administrative staff, the FDA is “more densely populated with science people.”
Nonetheless, “we share very common issues and goals,” EMA Data Analytics and Methods Task Force pharmacoepidemiology senior specialist Catherine Cohet stated. “We share the same problems,” she continued, citing RWD sources, heterogeneity, and suitability comparisons as universally challenging issues.
“Now is the time to see where we align,” Cohet declared.
DARWIN Steers EMA RWE Voyage
The European Union chose healthcare for its first common data space in a specific area, launching the European Health Data Space in May 2022 as a “health-specific ecosystem comprised of rules, common standards and practices, infrastructures and a governance framework.” (Also see “RWE Studies To Hit 100+ Per Year In Europe By 2025” – Pink Sheet, 1 Aug, 2022.)
EMA Launches First RWE Studies Under Big Data Project
By Vibha Sharma25 Nov 2022
The European Medicines Agency has partnered with eight institutions having access to real-world healthcare data to enable its DARWIN EU project to deliver real-world evidence to the EU medicines regulatory network when needed.Read the full article here
To coordinate the emerging approaches to health data across member state health systems, the EMA and Heads of Medicines Agencies (HMA) has established a Big Data Steering Group and adopted a Big Data Workplan in July covering 2022 to 2025.
One of the first priority deliverables of the Big Data workplan kicked off in early 2022 when the EMA selected Erasmus University Medical Center Rotterdam to establish the Coordination Centre of the Data Analysis and Real-World Interrogation Network (DARWIN EU).
The network of European Union real-world healthcare data sources will connect to the EHDS and is intended to promote “access to valid and trustworthy real-world evidence.” (Also see “EU DARWIN Project To Catalog Real-World Data Sources” – Pink Sheet, 12 Jul, 2022.)
DARWIN EU is targeted to be fully operational by 2024, Cohet told the DIA meeting. The federated network is “taking off nicely,” she commented, but the project is still building data sources and starting pilot studies. EU data sources are being uploaded at a rate of 10 a year, and disease registries will be included in the next two years.
The first set of studies using the network were announced last week. (See sidebar.)
DARWIN EU will also include non-EU data sources to cover products like the COVID-19 vaccines that were licensed in Europe but saw little use, she said. DARWIN EU serves as secondary data collection, Cohet noted, stressing the importance of primary data collection.
A good practice guide for the use of real-world metadata is slated to become available in the first quarter 2023, she said. The guide will build on two drafts released this autumn: a draft good practice guide to the DARWIN EU metadata catalog of RWD sources was issued in September, followed by a draft data quality framework in October (Also see “EU Drafts Recommendations On Identifying Suitable RWD Sources For Studies” – Pink Sheet, 28 Sep, 2022.) and (Also see “EU Consults On Data Quality Framework For Medicines Regulation” – Pink Sheet, 12 Oct, 2022.).
A catalog of RWD sources will ultimately replace the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP) resources catalog, Cohet reported. A catalog of RWD studies will enhance and replace the EU Electronic Register of Post-Authorisation Studies (EU PAS Register).
HARPER Bring Structure To Non-Interventional Protocols
DARWIN EU is just one of the international bodies that aims to work with the HARmonized Protocol Template to Enhance Reproducibility (HARPER), an initiative of the International Society for Pharmacoepidemiology (ISPE) and International Society of Pharmacoeconomics and Outcomes Research (ISPOR) to “create a harmonized protocol template for RWE studies that evaluate a treatment effect and are intended to inform decision-making.”
The HARPER template will provide a structure for evaluating suitability of RWD sources for a given research question based on the forthcoming EMA metadata catalogue for data sources, Cohet said. HARPER also provides a “template for the development of generic protocols to be used in DARWIN EU studies”
FDA Office of Surveillance and Epidemiology Division of Epidemiology I acting deputy director Wei Hua also pointed to HARPER’s promise for RWE. “Structured reporting improves transparency and facilitates quality assessment,” she said. “Generating quality evidence from secondary data that are not collected for research purposes can involve numerous complex design and analytic decisions.”
“Transparent reporting on key design elements helps reviewers understand the scientific decisions underpinning the reported findings and facilitate evaluation of study validity,” Hua stated. She emphasized the importance of pre-specifying both the study protocol (to prevent data dredging and address potential confounding and bias) and analyses (to assess the impact of potential confounding and bias on the sufficiency of the protocol and the adequacy of evidence).
FDA Demonstration Projects: OneSource Reduces Data Re-Entry…
Sharing data is a “not a tech problem, it’s a culture problem,” University of California – San Francisco professor Laura Esserman declared during the DIA meeting. In the fractured US healthcare environment, if UCSF couldn’t partner with the FDA to develop its electronic data capture (EDC) intiative, “we’d have no chance,” she said. Collaborating with the FDA “gets you in the door.”
Esserman is a leader on OneSource, a framework to “automate the flow of structured [electronic health record] data into external systems and thereby reduce operating costs, save time, and improve data quality for clinical trials,” according to the FDA.
USCF and its Silicon Valley collaboration Quantum Leap Healthcare are working with FDA on OneSource as one of the agency’s many RWE demonstration projects. (See sidebar for list of FDA pilots.)
Piloting Real-World Evidence: US FDA RWE Demonstration Projects
By Bridget Silverman28 Nov 2022
Agency offers three informal ‘buckets’ organized around data, study design and tools.Read the full article here
Electronic health records (EHR) are fundamental to many efforts to use real-world data, but use of the RWD for clinical trial analyses still requires manual re-entry of data between the EHR and the clinical trial EDC system, a burdensome process that accounts for 30-40% of clinical trial costs, Esserman said.
Furthermore, the contents of case report forms and patients chart often don’t match, she observed, while doctors are already burning out in the face of rising documentation requirements.
The first phase of OneSource focused on populating electronic case report forms for the Quantum Leap-Phase II I-SPY 2 platform trial in breast cancer with structured data from the UCSF EHR system, Esserman noted.
When the COVID-19 pandemic hit, Quantum Leap Healthcare Collaborative essentially ported the I-SPY 2 trial and its OneSource network to the COVID-19 setting, she recalled. The I-SPY COVID-19 platform trial enrolled 3,000 critically ill COVID patients over one and a half years, and assessed 11 agents.
OneSource was integrated with electronic health records at 17 of 30 I-SPY COVID sites, she said, adding that the hard part was “getting site governance to agree.” Researchers compared data entry time before and after implementing the OneSource v2 system, finding a 60% reduction in time to complete the electronic case report form daily data entry, Esserman reported.
OneSource’s goal for data is “enter once, use many times,” she said.
… DETECTe Addresses Confounded Treatment Effects
The risk of confounding in RWE studies, from selection bias to unbalanced risk factors, remains a significant hurdle for the use of RWE to support regulatory decision-making. The FDA turned to the University of North Carolina – Chapel Hill for a demonstration project that will inform future guidance development, known as Detailing and Evaluating Tools to Expose Confounded Treatment Effects (DETECTe).
“A modest amount of data to shed light on the actual distribution of an otherwise unmeasured confounder can enable us to quantify the impact in a specific study,” DETECTe principal investigator and UNC Center for Pharmacoepidemiology director Michele Jonsson Funk told the DIA meeting.
DETECTe was designed to “evaluate the performance of methods used to evaluate potential bias due to uncontrolled confounding in non-interventional RWE studies” and “enhance the capacity of FDA to critically evaluate these methods and integrate findings into assessments of non-interventional studies,” she said.
DETECTe uses simulation in a “very complex” trial design to expose confounded treatment effects, she explained. The researchers start with data from a randomized controlled trial that can be trusted to have no confounding bias and then introduce confounding and misclassification. The data is then analyzed using usual methods for non-interventional studies, including different sensitivity analyses.
The project looked at five trials encompassing HIV/AIDS, metastatic colorectal cancer and type 2 diabetes. DETECTe used the data to simulate 106 unique scenarios, Jonsson Funk said.
“Methods that used information about the confounder from a validation sample performed better than restriction,” she reported. “Quantitative bias analysis performed well overall,” while “multiple imputation produced slightly greater reduction in bias in some settings, but at the expense of increased variance.”
Industry Joins Forces: TransCelerate In Biopharma…
It is “really critical” that “we all use the same data sources,” Amgen, Inc. VP R&D Data Strategy Cathy Critchlow, declared. There will be a period where we will need intense cooperation between pharma and data providers to make that happen, she continued.
Critchlow is the executive sponsor of the RWD initiative at TransCelerate, a non-profit collaboration of 20 biopharma companies dedicated to addressing persistent issues in clinical trials, aims to work with global regulatory agencies to develop a “framework for RWD that defines tools, methodologies, use cases, and appropriate levels of evidence to support regulatory decision-making.”
TransCelerate’s foundational RWD initiatives focus on health authority engagement and audit readiness, she said, with the goals of building trust and reducing barriers to use of RWD and demonstrating RWD that is fit-for-purpose. Data relevance, accrual, provenance, completeness and accuracy were identified as key concepts.
A draft of the RWD Audit Readiness Considerations tool is expected later this year. The tool aims to “operationalize” work done by the Duke-Margolis Center for Health Policy on data quality management and assurance, she said. (Also see “Real-World Data: Precertification Could Aid Use For Regulatory Decisions” – Pink Sheet, 17 Oct, 2018.)
… RWE Alliance In Data And Analytics …
Five companies in the data and analytics industry, including Aetion, IQVIA Holdings Inc. and Flatiron Health Inc., formed their own coalition in 2021 to advance RWE, known as the RWE Alliance. (Also see “Data Firms Form Real-World Evidence Alliance, Giving Them A Voice With US FDA, Congress” – Pink Sheet, 19 May, 2021.)
Five more companies joined the RWE Alliance in 2022, IQVIA VP Clinical Evidence and Epidemiology Jennifer Christian told DIA.
Christian named three top-line policy priorities of the RWE Alliance:
- Advancing the FDA’s RWE Framework,
- Increasing the use of RWD to understand treatment effects in patients who are under-represented in clinical trials, and
- Enhancing consulting opportunities with the FDA, especially on methods and application.
In 2022, the group filed public comments on four FDA guidances, held eight meetings with Capitol Hill staff, and participated in 19 policy group meetings and eight external stakeholder engagement meetings, she reported.
The RWE Alliance’s biggest working group is focused on data quality and access, Christian said. The work group aims to clarify data collection and analysis expectations and foster access to patient-level data. The group also supports conversion to a supported common standard and data quality assessment.
… DiMe For Digital Health Tools
The Digital Medicine Society (DiMe) was launched in 2019 and has grown rapidly as the digital health tool (DHT) space has expanded. Program lead Sarah Sheehan told the DIA meeting that 101 sponsors have now collected data for more than 325 unique endpoints in DiMe’s library of digital endpoints.
Most digital data is never analyzed, Sheehan observed, even though there is broad acceptance of the concept of digitally-enabled real-world evidence generation to address evidence gaps and promote precision medicine.
“Standardized best practices” and “robust methodologies” are needed to unlock DHT’s potential for RWE, she said. A “unified, digitally-enabled evidence generation strategy” is needed.
The FDA has been working with the DHT industry on one of its most prominent early RWE demonstration projects, the myStudies app. (Also see “Real-World Evidence Demo Projects From US FDA To Include New Mobile App” – Pink Sheet, 6 Nov, 2018.)
DHT can help to “decentralize the data collection footprint,” Sheehan said. The technologies, which can be implemented inside or outside of the clinic setting, offer the prospect of the ability to measure events over a longer time period and to broaden patient participation.
But while DHT can be a part of equitable design, “you can’t digitize out of health inequities,” she advised.
Aetion senior VP-RWD and delivery operations Wendy Turenne noted an “explosion of data providers,” but advised that collaborators assure access to patient-level data and source documentation.
Solutions to many challenging RWE issues are available today but are not exploited, she said, pointing to tokenization and linkage as solutions for issues of relevancy and privacy-preserving technologies like synthetic data.
Regulatory Agnosticism
The tools, digital and otherwise, and regulatory frameworks for analysis of RWD can’t translate the data into evidence. The FDA is “agnostic” about the range of approaches for RWE, Concato noted. The agency’s role is “trying to help the field advance in general,” but based on methodology not popular appeal.
Participants at the DIA meeting repeatedly emphasized any RWE analysis needs data that is fit for purpose. “All data has its place,” Flatiron regulatory strategy director Jillian Rockland stated. Researchers need to articulate to authorities why their programs are “fit for purpose for your use case, not in general.”
You need the right tool, and the right patients, and the right disease, AstraZeneca PLC executive regulatory science director Charles Lee emphasized. That makes it hard to generalize.
“The future isn’t going to come quickly,” Turenne advised. “It’s a lot of small steps.”
Source: https://pink.pharmaintelligence.informa.com/PS147376/Collaboration-Makes-The-Real-World-Go-Round-Global-Efforts-Strive-To-Anchor-RWE-With-RealWorld-Data-Standards-And-Practices