The Inherited Retinal Disease Trial That Should Change How Sponsors Think About AI Diagnostic Evidence

Picture a geneticist at a tertiary retinal referral center, staring at a fundus autofluorescence image and a genetic panel that returned three variants of uncertain significance. The patient has been symptomatic for four years. They have seen two general ophthalmologists, one neuroophthalmologist, and a retinal specialist. Nobody has committed to a diagnosis. The geneticist opens the AI-based clinical decision support system described in the multicenter randomized trial just published in Nature Medicine and, within the consultation window, gets a ranked differential with confidence weightings across the most likely inherited retinal disease subtypes. The system doesn’t replace her judgment. But it narrows the diagnostic space in a way four years of referrals failed to do.

That scenario is no longer hypothetical. The Nature Medicine trial enrolled clinicians across multiple centers, randomized them to AI-assisted versus unassisted diagnosis of inherited retinal diseases (IRDs), and measured whether the decision support system actually moved the needle on diagnostic accuracy. It did. And the trial design itself, a prospective multicenter randomized architecture evaluating a diagnostic AI tool as a primary intervention, represents something the clinical trials industry has struggled to produce at scale: rigorous, randomized clinical utility evidence for an AI-based Software as a Medical Device (SaMD).

The result matters far beyond ophthalmology.

Why IRDs Are the Hardest Diagnostic Problem in Retinal Medicine

Inherited retinal diseases collectively affect an estimated 5.5 million people worldwide, with a global prevalence of approximately 1 in 1,380 individuals. In the United States, the 2023 prevalence estimate sat at 106 cases per 100,000. Those numbers sound manageable until you consider what the diagnostic pathway actually looks like: IRDs encompass more than 280 causative genes, dozens of phenotypic subtypes, and an imaging presentation that overlaps substantially across conditions. Retinitis pigmentosa looks like choroideremia in early stages. Stargardt disease mimics cone-rod dystrophy on certain imaging modalities. A clinician without subspecialty genetics training, which describes the vast majority of general ophthalmologists worldwide, is working a puzzle where half the pieces have been swapped from a different box.

That complexity drives a diagnostic delay problem the field has documented but not solved. Patients cycle through the referral system for years before receiving a confirmed molecular diagnosis, and because gene-specific therapies like voretigene neparvovec (Luxturna, Spark Therapeutics) for RPE65-associated retinal dystrophy are now approved, delay carries direct therapeutic consequence. Missing the two-year window for optimal retinal cell viability in a pediatric RPE65 patient is not an administrative inconvenience. It is an irreversible clinical outcome.

Into this diagnostic gap, the Nature Medicine trial inserted a randomized test of whether an AI decision support system could function as a reliable clinical co-pilot, not just in controlled lab benchmarking, but in a live multicenter setting with real clinicians making real decisions under real time pressure.

What the Trial Design Actually Proves

The architecture of this trial deserves close attention, because most AI diagnostic studies published to date are not randomized. They are retrospective accuracy benchmarks: feed the algorithm a labeled dataset, measure sensitivity and specificity, compare to a panel of specialist readers. That design answers a narrow question: can the model classify correctly when conditions are ideal? The Nature Medicine trial asked a different and more operationally meaningful question: does giving clinicians access to this system change what they actually do, and does it improve diagnostic outcomes?

Randomizing the clinician, rather than the patient, to AI-assisted versus standard-of-care diagnosis is a design choice that aligns with how diagnostic SaMD functions in practice. The tool augments the clinician’s cognitive process. So the trial measured the augmented human, not the algorithm in isolation. That distinction is significant. The STARD 2015 reporting framework for diagnostic accuracy studies requires transparent documentation of the index test, the reference standard, and the clinical role of the test in the care pathway, and a randomized design that embeds the AI within the actual clinical workflow satisfies that requirement in a way retrospective benchmarking never can.

The parallel in the broader AI medical device landscape is instructive. In 2024, the FDA authorized 168 machine-learning-enabled medical devices, a record high, with the vast majority cleared via the 510(k) pathway. Most of those authorizations were supported by analytical validation and retrospective clinical data. Very few were supported by prospective randomized evidence of clinical utility. The Nature Medicine trial represents the standard the field should be building toward, and almost universally is not.

Here is the counterintuitive problem embedded in that success: a trial this rigorous may actually make regulatory submission harder, not easier, for sponsors trying to follow its example.

The Regulatory Gap the Trial Exposes

In December 2024, the FDA finalized its guidance titled “Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions” (draft issued April 2023). The guidance addresses how sponsors can pre-specify modifications to AI/ML-based SaMD without triggering a new submission for every algorithm update. It is a meaningful step. But it does not resolve the deeper evidentiary tension: how much clinical utility evidence does a diagnostic AI actually need before the FDA will authorize it, and in what format must that evidence arrive?

The Eye2Gene tool, developed with support from Sight Research UK and published in Nature Machine Intelligence in June 2025, illustrates the ambiguity. Eye2Gene predicts the genetic cause of IRDs from routine eye scans, using a multimodal imaging input to generate gene-specific predictions without requiring upfront genetic testing. The performance data is compelling. But the publication pathway was Nature Machine Intelligence, not a randomized clinical trial. It demonstrates algorithmic accuracy. The Nature Medicine trial demonstrates clinical utility. These are not the same evidentiary product, and the FDA’s current 510(k) framework does not cleanly distinguish between them for most diagnostic SaMD submissions.

Sponsors sitting in pre-submission meetings for AI-based diagnostic tools are navigating this gap in real time. The agency wants evidence that the device performs as intended. What “performs as intended” means for a decision support system that operates through clinician augmentation, rather than autonomous output, remains underspecified in every guidance document currently on the FDA’s website. The Nature Medicine trial’s design, randomizing clinicians and measuring downstream diagnostic accuracy, is arguably the most defensible answer to that question. But there is no submission template that tells a sponsor to run it.

A pragmatic cluster-randomized trial in Kenya, published in Nature Medicine in 2025, enrolled 9,691 patients across 16 primary care facilities and tested an LLM-assisted clinical decision support system against standard electronic medical record workflows. The trial architecture randomized 103 clinical officers to assisted versus unassisted care. The parallels to the IRD trial are direct. Both designs treat the AI as an intervention delivered to clinicians, not patients. Both measure real-world diagnostic behavior, not algorithm output in isolation. And neither design has a clear home in the FDA’s current SaMD evidentiary framework.

That is the structural problem the IRD trial quietly surfaces. The agency has spent five years publishing guidance on AI/ML-based SaMD modifications, transparency, and predetermined change control. It has not published a guidance document that tells sponsors how to design a prospective randomized trial to establish clinical utility for a diagnostic decision support system, what endpoints to pre-specify, how to define the reference standard, or how to handle the confounding introduced when the same clinician uses the tool on some patients and not others. Sponsors are generating the evidence. The FDA has not told them what evidence it will accept.

What Sponsors Should Do Before the Agency Catches Up

The operational implication is uncomfortable but clear. Sponsors developing AI-based diagnostic SaMD for rare disease indications, whether IRDs, lysosomal storage disorders, or other genetically complex conditions with long diagnostic odysseys, cannot wait for the FDA to prescribe the evidentiary standard. The Nature Medicine IRD trial is now the highest-quality publicly available template for what rigorous clinical utility evidence looks like in this space. Regulatory affairs teams should be using it as a benchmark in pre-submission meeting requests right now, not after a 510(k) package gets a deficiency letter asking for clinical utility data that was never defined in advance.

Concretely: any sponsor building an AI diagnostic support tool for a specialty with a documented diagnostic delay problem should be requesting a pre-Sub meeting that specifically addresses the clinical utility evidence threshold before protocol finalization. The meeting request should cite the FDA’s December 2024 Predetermined Change Control Plan guidance and ask, on the record, whether a randomized clinician-level trial measuring downstream diagnostic accuracy satisfies the agency’s clinical performance expectations for SaMD in this indication. Get the answer in writing. Because the Nature Medicine trial just demonstrated that randomized evidence at this standard is achievable, and once the FDA sees it submitted by a competitor, the bar will quietly move.

The IRD trial also carries a direct message for eClinical technology vendors selling AI-powered site support tools, patient identification algorithms, and protocol deviation detection systems into the trial operations market. Those tools are not subject to SaMD regulation in the same way a diagnostic device is, but the evidentiary standard for clinical utility is converging across contexts. A randomized evaluation that measures whether the tool changes behavior and outcomes, not just whether the algorithm flags correctly, is the evidence that will differentiate credible vendors from proof-of-concept theater in an increasingly crowded market.

The geneticist at that retinal referral center, staring at three variants of uncertain significance after four years of inconclusive workup, now has a published randomized trial telling her that the AI decision support system she is considering actually changes diagnostic outcomes in a multicenter setting. The next question is whether the regulatory framework will give the sponsor who built that tool a clear enough evidentiary roadmap to keep building the next version of it. The FDA’s December 2024 guidance addressed algorithm updates. It did not address that question. And the gap between what the trial proved and what the agency has specified is where the next generation of diagnostic AI either finds its footing or loses its funding.

References

1. Nature Medicine — “AI-based clinician decision support system for diagnosis of inherited retinal diseases: a multicenter, randomized trial”
2. FDA — “Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions” (Finalized December 2024)
3. PMC — Inherited Retinal Disease Global Prevalence: approximately 1 in 1,380 individuals, 5.5 million affected worldwide
4. PMC — STARD 2015: Standards for Reporting Diagnostic Accuracy Studies
5. PMC — FDA authorized 168 ML-enabled medical devices in 2024, record high, majority via 510(k)
6. Sight Research UK — Eye2Gene: AI tool predicting genetic cause of IRDs from routine eye scans, published Nature Machine Intelligence June 2025
7. ResearchGate / Nature Medicine — Generative AI clinical decision support system, Kenya pragmatic cluster-randomized trial, 9,691 patients, 16 facilities, 103 clinical officers