Managing uncertainty in medicines approval: empirical analyses of non-randomised evidence and regulatory practice

Abstract

The introduction of new medicines to the market requires robust evidence about efficacy and safety, traditionally obtained from randomised controlled trials (RCTs). Aiming to accelerate market access, regulatory bodies in Europe (EMA) and the United States (FDA) increasingly accept uncertainty by granting approval based on non-randomised studies and surrogate endpoints. This thesis applies documentary analysis and meta-epidemiological methods to critically examine the methodological assumptions underpinning regulatory acceptance of uncertainty and the regulatory instruments used to manage it in three empirical studies. First, a meta-epidemiological study of 346 meta-analyses (2,746 studies) found no strong evidence of systematic over- or underestimation of treatment effects in non-randomised studies compared to RCTs overall. However, for a substantial proportion of clinical questions, the two study types led to different conclusions about existence or magnitude of effect, highlighting uncertainties about benefits and risks when substituting RCTs with non-randomised studies. Second, a comparative analysis between EMA and FDA of 21 cancer medicines approvals (2009-2013) with uncertainties in the pivotal trial evidence showed frequent divergence in regulatory outcomes, with one regulator granting full and the other conditional approval requiring confirmatory post-marketing studies or denying approval. When confirmatory studies were imposed, they were frequently delayed and continued to use non-randomised designs and surrogate endpoints. Third, a study of 55 EMA-approved cancer medicine indications (2014–2023) found that single-arm trials increasingly serve as pivotal evidence, yet justifications for accepting methodologically limited evidence were often unclear. For most indications, RCTs appeared feasible, but post-approval RCT evidence only demonstrated clinical benefit on patient-relevant outcomes for a minority of indications. The findings indicate that reliance on non-randomised studies for regulatory approval introduces uncertainty that may not be resolved post-approval. Regulators should leverage their position as gatekeepers to incentivise robust evidence generation as part of research and development programmes.