We need to industrialize virtual synthetic repurposing trials

A call for a Focused Research Organization

We need to industrialize virtual synthetic repurposing trials (or, as they’re more commonly known, emulated trials)¹.

By virtual synthetic repurposing trials, I mean using healthcare data, like the healthcare records of national medical services, to examine how different drugs, which coincidentally are given with certain diseases, can slow or stop those diseases. The details on how this is done is given in the name.

1. Virtual, because they are based on data that already exists in the cloud of health datasets

2. Synthetic, because they require combining together a bunch of matched cases, including controls, rather than actually running a trial where everyone gets the same treatment

3. Repurposing, because these are drugs that already exist and already have coincidentally been given to the patients in question

4. Trials, because we’re going to try to do everything else the same, including randomization and controls. This will be tricky, given the synthetic part, but we can approximate.

These trials already exist, and they’ve had some successes. There have been virtual synthetic repurposing trials that have found:

1. A 51% reduction in esophageal cancer among Taiwanese patients given metformin

2. A 21% reduction in all-cause dementia and 20% reduction in Parkinson’s among Korean patients given SGLT2 inhibitors

Both of these have sparked additional analyses, follow-on trials, and, for at least the SGLT2 inhibitor trial, changes in clinical practice.

God bless AI.

The issue with these trials, though, is that they’ve all been bespoke. They’ve all required 10+ academics to get the regulatory clearance, clean the data, then comb through the massive dataset and slowly test their one or two hypotheses. Then, once the data is released, it’s released as a pdf and there’s literally nobody who can or will replicate their work or even see the provenance of it.

It doesn’t need to be like this. The tech industry has gotten really good at analyzing massive datasets. If we set up a non-profit, like an FRO, that took this seriously as a tech challenge, we could be releasing a new one of these studies every week, along with “reverse repurposing” trials that see which diseases are causally connected with which other diseases, like the awesome EBV causes MS study I’ve discussed before. These could all be released as interactive dashboards with technical info about each virtual trial and what assumptions were made.

There have already been some efforts in this direction, like OHDSI and EHDEN, but these are volunteer-based efforts that focus predominantly on software that researchers can use and data science tools. This is important, but there’s only so much they can do as volunteer organizations to enforce quality and cadence, and there are no efforts towards making this info more accessible than the standard paywalled PDFs. And, besides, the biggest problem isn’t on the software side: it’s on the regulatory.

The best longitudinal healthcare datasets are locked behind regulatory bottlenecks. Not only are there privacy and data protection requirements to access the datasets, but they tend to only be accessible by a select group of researchers. The Taiwanese national health dataset can only be accessed by a National University of Taiwan researcher or their affiliate. Likewise, the US Department of Defense military health dataset, which includes the frozen serum samples that were so crucial to the EBV/MS study, can only be accessed by DoD researchers and their affiliates.

So, every single study ends up being bottlenecked not only by the resources of the scientists involved, and not only by the regulators, but also by the time of the few key people who are allowed to actually run the studies. There’s a better alternative: industrialize it.

By that, I mean have full-time teams for the software development, the data cleaning, and the regulatory. The software development team should just work on developing a regulatory sufficient pipeline that auto-runs on the datasets, taking in suggestions for specific things to check from outside scientists. The data cleaning team should solely work on adding new health records and making sure they are up to snuff.

And last, the regulatory team needs to have pre-existing relationships with every single agency that controls access to health records. They need to communicate with the software team, making sure their work is following the appropriate regulations (e.g. GDPR). They also need to communicate with the data cleaning team, making sure the data that’s exposed to the public abides by privacy requirements. But, they also need to be proactive, trying to push agencies to allow more access and more thorough access. This should be a combination of lawyers, lobbyists, and scientific liaisons.

The output of all of this should be an interactive dashboard with APIs. Ideally, anyone will be able to see all of the correlations and trials that were run, how they were run, and perform their own meta-analyses. The metric of success will be engagement with the dashboard and APIs, including publications, changes in clinical practice, and follow-on trials.

Will there be questions about validity? Of course. These sorts of correlational trials are never perfect. Any time you’re blindly running a bunch of statistics on a huge dataset you will get spurious correlations, as well as issues with selection biases, measurement biases, and everything else that comes with relying on data input by harried primary care doctors who never knew their work was going to be used like this.

But the potential payoffs are still immense. Any strong signals, especially if they can be mechanistically backed², backed by Mendelian randomization, or, ideally, both, will be an immediate cause for confirmatory trials. It could also change how we prioritize biomedical development, like how the EBV/MS study made an EBV vaccine much more important.

I can write a full budget out for this, including some chances for revenue (e.g. priority analysis slots and API keys), but I’ll stop here. Hopefully, someone can take this ball and run with it.

1. Note: click the link for a good discussion on the technicalities of emulated trials and how best to run them.

2. Like by the EvE FRO.