External control arms: a cutting edge methodology to de-risk and accelerate clinical trials

What are external control arms (ECAs)?

An external control arm is a method used to use external data to create a comparator arm that mimics the characteristics of a RCT control arm. They use data collected from outside a clinical trial – either from other trials or real-world data – to provide a comparator group for efficacy analysis. These control patients are selected based on their clinical profiles matching the profiles of patients enrolled in the treatment arm.

‍

When are they used today?

Cancer clinical trials often have low success rates, with more than 80% of therapies failing to meet their primary endpoint in phase 3 trials. Major reasons for this include the lack of knowledge on the drug‘s effectiveness before phase 3 trials and recruitment difficulties, especially for rare diseases. To address these issues, there is growing interest in using external control arms to better assess the efficacy of treatments being tested in single-arm trials.

ECAs are already being used in regulatory applications. However, no marketing applications in oncology have used an external control arm as part of the primary efficacy analysis. Instead, ECA data has been used to establish the natural history of the disease, to isolate the treatment effect, or for comparative efficacy analysis.

‍

How do they work?

To build an external control arm today, researchers must match the clinical profiles (age, sex, lifestyle factors, e.t.c.) to patient data outside the trial. This is typically done using a statistical matching technique called propensity score matching (PSM). However, there are alternative approaches to infer efficacy based on machine learning (ML) predictions of control patient outcomes, such as G-computation and Doubly Debiased Machine Learning (DDML).

Numerical simulations suggest that G-computation reduces bias and variance of causal inference estimates compared to propensity-score approaches and that DDML was among the top-performing methods to estimate the average treatment effect. However, comparisons based on actual trial data are insufficient. We set out to investigate how much outcome-based modeling could improve statistical analysis.

‍

What did we do?

We used data from five type 2 diabetes randomized clinical trials provided by the Yale University Open Data Access project to compare the effect of the different approaches.

‍

What did we show?

Our results showed that methods based on outcome prediction models are more powerful than propensity score approaches in external control arm analyses. We also showed that there are marked differences between the results obtained with G-computation and DDML but that the choice of technique can be guided by sample size. The sample sizes for external control arms can range from dozens to thousands of patients. In oncology, after the application of inclusion and exclusion criteria, the sample size can be smaller than 𝑛=100. In this case G-computation should be preferred. In other cases, sample sizes can exceed 𝑛=500, in which case DDML should be used. By using these computational prediction methods, we can get more accurate estimates of the treatment efficacy in external control arms.

To see the full results, read the paper here.

Nicolas Loiseau, first author of this research paper said:

The use of prognostic models to assess the benefits of a new treatment shows promising results compared to the widely used methods relying on the propensity score. As knowledge grows, I hope we will witness an increasing interest in these new methods from clinical trial sponsors and regulators.

‍

Why does this research matter?

This groundbreaking research not only moves us closer to optimizing clinical trials with cutting-edge machine-learning techniques but also presents strong evidence for regulators to support these innovative approaches as they begin to consider more regulatory applications that incorporate external control arms. By de-risking and accelerating clinical trials, we hope to get better and safer treatments to patients sooner.

‍