- Covariate adjustment is a method to reduce sample size or increase statistical power in clinical trials;
- It leverages meaningful clinical patient characteristics, including risk scores;
- Machine learning (‘ML’) can improve the predictive accuracy of these risk scores; and
- ML therefore allows for more efficient clinical trials.
The achievement of sufficient statistical power to prove a therapeutic effect requires the recruitment of large numbers of patients — a fact reflected in the great cost and long duration historically associated with clinical trial programs.
However, there is another, more effective, way to achieve sufficient power beyond increasing the sample size: by adjusting the efficacy analysis on covariates associated with the endpoint of a clinical trial. Through this approach, much of the variance that would otherwise obscure the treatment effect can be mitigated, reducing the noise so the signal clearly stands out.
For example, age is a well-known predictor of Covid-19 severity, and taking this factor into account in ongoing clinical trials would allow for better estimates of treatment effect.
In this article we will explain the concept of covariate adjustment, and its potential impact on clinical development.
What is Covariate Adjustment?
Covariate adjustment can be best understood through a metaphor regarding signal and noise.
Imagine you are on the subway, and trying to listen to music like in the cover image above! The noise of the subway drowns out the music coming from your headphones. You could increase the volume, and risk long-term damage to your hearing. But a better alternative would be to use a noise-canceling headset that compensates for the noise of the subway, allowing you to listen to music at a safe volume.
In a clinical trial, the signal (ie, the music) we want to listen to is the treatment effect. The noise of the subway is the variability in treatment outcomes that can be explained by covariate factors, such as medical history. To better “hear” the signal, we must either increase the volume (ie, the sample size), or cancel out the noise (adjust for covariates that are predictive of the endpoint, thereby reducing the heterogeneity in treatment outcomes not explained by treatment effect).
The effectiveness of covariate adjustment is well documented in clinical trial methodology literature1, and accepted by regulators. In particular, the European Medicines Agency (‘EMA’) Guideline on adjustment for baseline covariates in clinical trials states:
The main reason to include a covariate in the analysis of a trial is evidence of a strong or moderate association between the covariate and the primary outcome measure. Adjustment for such covariates generally improves the efficiency of the analysis and hence produces stronger and more precise evidence (smaller p-values and narrower confidence intervals) of an effect.2
As covariate adjustment allows researchers to capture the same information with a smaller sample size, it presents an opportunity to run more efficient trials aligned with regulatory standards.
How to choose the right covariates
An important practical consideration is the choice of covariates for adjustment. Fortunately, many diseases have been thoroughly researched, and the associated literature, prognostic models and/or risk scores can inform the choice of relevant factors to include.
In fact, adjustments on an available prognostic model or risk scores actually pull information from all of the variables contained within that model or score, maximizing the gain in precision while adjusting on the basis of one covariate.
To ensure generalizability, it is recommended that prognostic models/risk scores are evaluated on external datasets that have not been used to build the model/score. As the predictive capacity of the model/score increases, so does the potential for a more efficient clinical trial (ie, sufficient power with smaller patient cohorts).
How can Covariate Adjustment impact clinical trails?
New technologies, such as advances in genomics and artificial intelligence, may have a rapid and positive impact on clinical development by supplying new predictive covariates. For example, the establishment of polygenic risk scores (risk scores based on your genetics), while still ongoing, may eventually be of use in trials for the prevention of cardiovascular disease3. Similarly, deep convolutional neural networks have already revolutionized computer vision, and are being applied to imaging data across medicine, most notably in radiology, ophthalmology, dermatology, and histopathology4. Using these techniques enables researchers to capture previously unexplained heterogeneity in prognosis. In fact, the application of our tailored deep learning algorithm has already been shown to improve the prediction of overall survival vs. the current pathologist-led actual predictions in mesothelioma5 (a lung cancer predominantly caused by asbestos).
Mesothelioma Use Case
At Owkin, we developed MesoNet, a machine learning model that predicts the overall survival of malignant mesothelioma patients using whole slide (histology) images of tumor tissue. These new survival predictions are better than the existing subtype classification utilized by pathologists — meaning MesoNet can be leveraged to reduce the sample size of mesothelioma trials, lowering costs and reducing trial lengths. In fact, we estimate that MesoNet’s deep learning approach compared to standard subtyping would allow researchers to reduce the sample size requirements of three, large phase 3 clinical trials in mesothelioma by 6-13%, resulting in millions of dollars of savings. Furthermore, assuming that patient enrollment is conducted on a rolling basis, the decrease in sample size obtained using MesoNet would allow for a 2-8 month reduction in trial duration vs current practice.
Figure 1: MesoNet Simulations for Three Large Clinical Trials
Figure 1: All information pertaining to the clinical trials was obtained through clinicaltrials.gov. Simulation results illustrate the use of MesoNet risk scores as a covariate adjustment in clinical trial primary analysis in addition to histological subtype. Current clinical trials are assumed to stratify patients and adjust the analysis on histological subtype (epithelioid vs. sarcomatoid vs. mixed), and to have a sample size corresponding to 80% statistical power. Clinical trial costs per patient have been sourced from Evaluate Pharma to estimate cost savings. These simulations were run using the MesoNet training dataset from the Mesobank.
Covariate adjustment powered by machine learning stands to fundamentally change the way clinical trials are conducted. By systematically adjusting predictions for well-established covariates, we can accelerate clinical development programs and reduce costs — and do so at a scale that will bring real progress.
At Owkin, we are currently collaborating with regulators to set the foundations of machine learning-powered clinical trials. But optimizing trial protocols is just the beginning — machine learning is poised to make a lasting impact on other steps of the clinical development pipeline as well. After early-stage clinical trials, a pharmaceutical company must decide, based on very limited data, whether the drug efficacy is promising enough to start a long and expensive phase 3 trial — and this will be the topic of another post.