Validating AI for Clinical Decision-Making in Oncology

Drs. Matthew Kurian and Arturo Loaiza-Bonilla discuss the challenges of validating AI tools for clinical decision-making in oncology. They explore retrospective versus prospective evidence, using Artera AI as an example of how emerging technologies may move toward broader clinical adoption.

Transcript:

Dr. Kurian:  We see these models being presented at major meetings all the time, saying, “Okay, we’ve created this model. Here are the clinical variables. We’re using a single H&E slide to be able to tell these different variables.” But then you don’t really see them being adopted clinically, and I want to pick your brain regarding how you view exactly what we’re doing in terms of clinical decision-making tools.

So Artera AI on the prostate cancer side is a great example. They went back and retrospectively validated all their studies on cooperative group data, right? On old data, and then were able to gain an NCCN designation just using old data and not using any prospective data at all. So do you think that’s the model that many other companies are thinking of, or do you think we need to require folks to create prospective trials that obviously are going to be more costly in the long run and are going to take a long time to read out too?

Dr. Loaiza-Bonilla: Yeah, well, knowing that typically folks will follow a path that led to success, I would not be surprised if many folks follow the same path because it was paved by the team at Artera and beyond. I think the challenge is going to be, once you have two or three of the same models by different folks, how do you choose which one? They’re all based on the same data, so it’s becoming like me-too kind of slop over time.

So I think that the differentiator would be for those aggressive, innovative folks who are willing to take the chance to say, “Okay, our technology is really good. It’s better than what we’ve seen in the space, and now we’re going to run it on prospective data.”

I would not be surprised if we start using these internal control arms or some form of alternative models using AI as well to validate certain cohorts. So we know that real-time validation can be better, and that may also speed things up because we don’t want to wait for 10 or 15 years to get the output that we’re looking for.

So I think we can run the two things at the same time. Maybe it’s like when the FDA gives an accelerated approval for a medication and says, “Okay, we see a signal here, and we’re going to give you approval for the time being, but in the meantime, you have to run prospective trials that show that it’s going to work for survival.”

So I think that kind of combined pathway makes sense. The pie is not getting bigger in terms of reimbursement, so we need to be smart. The one that differentiates itself better is the one who’s going to take the risk, and with risk comes upside.

Dr. Kurian: Yeah, absolutely. I totally agree with you.

I think there’s going to be a combination approach that will be required, and even Artera AI now is carrying out their own prospective study to prove exactly what you’re saying too. So I think regulators will want that moving forward too.