A new way to do controlled experiments in medicine: simulate the control

A randomized controlled trial (RCT), the gold standard for testing new medicines and treatments, is where ethics and patient safety collide with statistics. To be able to say with any statistical certainty that a treatment, like a new drug, had an effect, large numbers of patients in an RCT are typically divided into two groups that are otherwise similar, except one randomly gets the drug or treatment, and the other doesn’t. Having two ‘arms’ in a study requires lots of people, can be costly and, in the case of expensive drugs or rare diseases, can appear ethically problematic.

The possibility of running an RCT with only a single arm is therefore enticing, and an active area of debate and research, says Manuel Haussmann, a postdoctoral researcher at the University of Southern Denmark. “The idea of extending RCTs with data from electronic health records or synthetic data is gaining steam, and different ways to do this using machine learning have been proposed.” Haussmann, who developed this work as a postdoc at the Finnish Center for Artificial Intelligence FCAI, and colleagues have recently published a way to identify patients from real-world data to mimic the control arm of an RCT, essentially creating a simulated control group to match a real treatment group. Their work is being presented at the AISTATS conference in early May 2024.

The research draws on the extensive electronic health records that are available for secure use in Finland. The high standard of these health databases plus data protection laws are globally rather unique, says Haussmann, and enabled the research collaboration between FCAI, Helsinki University Hospital and Bayer.

The researchers had to match each treated patient with a simulated one to successfully create a synthetic control group. This is where the deep learning model comes in. “There’s no way to easily match the two groups. We built a model that creates a high-level summary across all the possible variables, like age and gender, in a huge medical and diagnostic history, and then matches patients on the features that are relevant to predict the outcome of the RCT,” explains Haussmann.

Because even this extensive history can be incomplete, the model includes a generative element that can fill in the blanks. “The generative part is important for handling missing data and noisy observations, like variability in blood tests for example,” says Haussmann. Generative in this case is another word for ‘probabilistic’, or how accurately the model has learned, compressed and summarized what is really important about patients for predicting the outcome of a clinical trial.

There are vast amounts of health data out there, and this is one big step in the direction of making use of that data, says Haussmann. “We can’t beat RCTs for quality, but we can learn how to optimally match patients using machine learning, which shows that single-arm studies are viable.”

Reference:

Manuel Haussmann, Tran Minh Son Le, Viivi Halla-aho, Samu Kurki, Jussi Leinonen, Miika Koskinen, Samuel Kaski, and Harri Lähdesmäki (2023). "Estimating treatment effects from single-arm trials via latent-variable modeling." doi: 10.48550/arXiv.2311.03002