Reacting to new regulatory draft guidance on observational studies, stakeholders asked the US Food and Drug Administration (FDA) to flesh out its advice with examples and to ensure consistency with other real-world evidence (RWE) reports.
FDA outlined its thoughts on protocol design and approaches as “considerations” for non-interventional studies of drugs and biological products in draft guidance in March. (RELATED: FDA offers guidance on observational studies as RWE, Regulatory Focus 21 March 2024). Sponsors were guided to explain, among other things, the rationale for using a noninterventional study, ways to prevent bias and approaches for addressing ethical concerns.
The guidance complements FDA’s RWE framework and several other guidance documents designed for use in supporting regulatory decisions with RWE. (RELATED: FDA finalizes guidance on real-world evidence in drug approvals, Regulatory Focus 31 August 2023). In its March 2024 report, the agency noted that the topics in the guidance “should be considered in conjunction with all of these other guidances.”
Several stakeholders commenting on the latest guidance by FDA’s June 21 deadline feel that greater clarity and consistency is needed across the RWE documents.
“The totality of the now available guidance documents provides valuable support for the research challenges that currently limit our ability to bring new treatment options to patients, in particular in situations where a randomized controlled clinical trial is not feasible, e.g., when very few patients are available,” wrote EMD Serono.
But EMD Serono went on to ask for a table that lists all FDA documents related to real-world data (RWD) and evidence as a navigation aid.
“The FDA’s guidances on RWD/RWE for broad topics such as the use of RWE for demonstration of safety and the use of external control arms for single-arm trials appear fragmented across several guidance documents, which make it challenging for stakeholders to capture all relevant elements for a given use case,” EMD Serono wrote.
In the same vein, the Biotechnology Innovation Organization (BIO) advised FDA to “review and update relevant existing RWE guidances to ensure that they reference this new draft guidance and incorporate recommendations, as appropriate, to ensure consistency.”
Terminology draws scrutiny
Flatiron Health pointed out that the latest document uses the term “fitness for use” in references to reliability and relevance assessments, while “fit-for-purpose assessment” was used in other RWE documents.
“We recommend that FDA clarify whether these terms are meant to convey the same concept and, where possible, that FDA use a single defined term across all Agency RWE guidance to minimize confusion and promote correct and consistent interpretation of FDA’s RWE guidance broadly,” Flatiron wrote.
The RWE Alliance, a coalition of real-world data and analytics organizations, also flagged fitness for use vs. fit-for-purpose terminology and asked for use of a single term to ensure all stakeholders interpret the guidance correctly and consistently.
A different tune from EMA
Regeneron Pharmaceuticals queried FDA’s definition of non-interventional studies:
“For the purposes of this guidance, a non-interventional study (also referred to as an observational study) is a type of study in which patients received the marketed drug of interest during routine medical practice and are not assigned to an intervention according to a protocol. Examples of non-interventional study designs include, but are not limited to, (1) observational cohort studies, in which patients are identified as belonging to a study group according to the drug or drugs received or not received during routine medical practice, and subsequent biomedical or health outcomes are identified and (2) case-control studies, in which patients are identified as belonging to a study group based on having or not having a health-related biomedical or behavioral outcome, and antecedent treatments received are identified.”
This definition differs from the EMA and a harmonized approach would be preferable, Regeneron suggested.
“EMA’s definition specifies that there are no diagnostic or monitoring procedures in addition to normal clinical practice as part of the non-interventional study, whereas this criterion is not included with FDA’s definition,” Regeneron wrote. “As a result, a given study might be considered non-interventional in the US but not non-interventional in Europe.”
Guidance too broad?
Several stakeholders made suggestions for ways FDA could build on its guidance. BIO wrote that the guidance is “notably more high-level and lacking in detail” compared to the agency’s other recent RWE drafts.
“The lack of detail limits the utility of the draft guidance for guiding drug sponsors conducting non-interventional clinical trials,” BIO wrote.
“It would be helpful for the FDA to provide suggestions regarding recommended best practices for non-interventional studies,” BIO added. “We request the FDA clarify how to define the population, set the target population for some specific groups, and augment the population based on RWD.”
Among other specific requests for revisions that would clarify the agency’s thinking, BIO asked for examples of types of studies that might constitute “contributions of substantial evidence” cited in the guidance.
BIO also asked for more explanation of what exactly is expected for validation efforts.
In its comments, the American Society of Hematology said it appreciated the overall guidance, including specific considerations around study design and analysis, but asked FDA to strengthen its guidance with numerous additions including:
- Provide additional specificity regarding how the agency determines whether a data source is fit for purpose to support RWD research.
- Include guidance on the development of data sources that can be considered reliable for observational analyses and provide insight on how these sources will be evaluated (i.e., should data sources be fully reevaluated on a case-by-case basis for each new project under consideration?).
- Include a flowchart or checklist summarizing essential attributes of well-designed non-interventional studies to help provide clarification for those who use this guidance.
- Consider including examples of data validation techniques and best practices to ensure acceptable data quality from RWD sources.
- Encourage sponsors to include input from patients and their caregivers and other stakeholders during the design, implementation, and dissemination phases of non-interventional studies.
Given the challenges of conducting traditional randomized, double-blinded placebo-controlled trials in small patient populations, observational studies are especially important for research in rare diseases, and “substantially more specific” guidance is needed, commented the National Organization for Rare Disorders (NORD).
“N-of-1 studies, as well as other studies with small patient populations, heterogenous disease manifestations, a large number of potential confounders, or particularly long periods of follow-up are all areas that could benefit from additional actionable guidance,” NORD wrote. “The absence of such specific and targeted information arguably limits the utility and applicability of the draft guidance in its current form to the rare disease field.”
Meanwhile, the science of conducting non-interventional studies to support drug approvals is evolving, Regeneron pointed out.
“As such, this guidance document should make clear that this is a dynamic topic and highlight the Agency’s flexibility and receptivity in engaging with Sponsors to advise on and explore approaches on the design and analysis of non-interventional studies, concerns and considerations specific to individual drug development programs,” Regeneron wrote.