The one-sided 0.025 significance threshold is not a law of nature. It is a ratio, and somebody chose it.
Under a non-informative prior, holding that threshold is roughly equivalent to requiring a 97.5 percent posterior probability of benefit. Work backwards from there and you find an implicit statement about the relative cost of two mistakes: approving an ineffective therapy is treated as about 39 times worse than failing to approve an effective one.
A statistician on the second panel at the FDA and Duke-Margolis rare disease workshop on September 29 put it plainly. That ratio is defensible for a common condition where good alternatives already exist. In a severe rare disease where nothing is approved, and where a failed trial may mean decades before anyone attempts a second one, it is much harder to defend.
He has used a 90 percent threshold instead. That implies a 9 to 1 ratio rather than 39 to 1. It cuts required sample size by roughly 40 percent.
“Whatever threshold we use, you have to follow the consequences, study the error, write down the proper reasoning and share it with clinicians, regulators and the patient.”
Statistician, on relaxing the success threshold
The Argument Nobody Should Be Making
If the first session established that the control arm is harder to give up than the alpha level, the second explained why sponsors keep trying anyway, and an FDA reviewer named the move directly.
When a sponsor says it is incorporating unmet medical need into the trial design, he said, what it usually means is that it wants a single-arm trial because no effective therapy exists. Sometimes that is right. Single-arm designs work well where natural history is predictable with essentially invariable outcomes on an objective endpoint, and they work where a product has a near-miraculous effect. Outside those narrow cases, dropping the controlled comparison does patients a disservice.
His objection was not statistical purity. It was what happens afterwards.
“We are left at the end of the day with ambiguous evidence that makes our job harder. But that is not the important part. The important part is that it makes it hard for patients to make informed benefit-risk decisions after a product is approved.”
FDA reviewer, on single-arm designs justified by unmet need
His conclusion reframes where unmet need belongs in the process: on the decision back end, not the design front end. Severity of need should shape how a regulator weighs an answer. It should not be the reason the trial cannot produce one.
Borrowing Is Not Free
Rare disease programs often have information sitting outside the trial. Natural history studies. An adult trial that might inform a pediatric program. Bayesian methods offer a principled way to fold that into the analysis through an informative prior, and the panel spent real time on how to set one.
The cost was stated more cleanly than it usually is. Each patient’s worth of information you borrow is also that patient’s worth of risk, if the external data turn out not to match the trial population.
That framing makes the trade-off concrete rather than abstract. Borrowing thirty patients of prior information is not thirty free patients. It is thirty patients of exposure to the possibility that the historical cohort was different in a way nobody modeled.
The safeguards the panel converged on were procedural rather than mathematical: simulate extensively, study the operating characteristics under optimistic and pessimistic scenarios as well as through the null, pre-specify sensitivity analyzes as a stress test against credible alternative scenarios, and be transparent about prior selection and about any conflict between the prior and the data once it arrives.
A Different Question to Ask the Posterior
One advantage of the Bayesian framing came up repeatedly and is easy to miss, because it is about the question rather than the method.
A frequentist test answers whether the effect is distinguishable from zero. A posterior distribution can answer something more useful: how likely is it that the benefit exceeds the size that patients and clinicians consider meaningful. Patient preference data can set that line, including how much risk is acceptable for a given benefit.
Which changes what you can tell a family. As one panelist put it, a family facing a progressive disease is probably better served knowing there is a 75 percent chance of a meaningful benefit than a 99 percent chance of a small one.
Designs That Buy Efficiency Without Buying Bias
The panel ran through the options that improve efficiency without abandoning the comparison.
The prognostic score point deserves its own sentence, because it resolves a tension sponsors run into constantly. Covariate adjustment is the cheapest efficiency gain available and almost nobody uses enough of it, but in a 40-patient trial you cannot adjust for twelve covariates. Collapsing the prognostic information into one score, learned before the trial from external data, gives the adjustment without spending the degrees of freedom. Because it is fixed at the design stage, it does not introduce bias.
On Composite Endpoints
Composites get proposed in rare disease for an understandable reason: no single component will accrue enough events to power the trial. The FDA reviewer set two conditions. Every component should be individually meaningful, and the components should be of approximately equal importance.
Where that fails you get the paradox the condition exists to prevent, an apparent effect on the composite while patients are flat or worsening on the component that matters most.
His own preference went further. He would rather have very clear endpoints and locate the flexibility in how the decision weighs each outcome, than compress everything into one number before the trial starts. That gives a regulator a richer basis for judgment than smashing the components together at the front end.
What a Sponsor Can Use Tomorrow
Three things from this session translate directly into a pre-submission conversation.
Bring the ratio, not just the threshold. Asking for a 90 percent posterior probability rather than 97.5 is easier to defend when it is framed as a deliberate statement about the relative cost of two errors in a disease with nothing approved, supported by simulation showing what it does to false positive and false negative risk.
Do not use unmet need as the argument for dropping the control arm. The agency’s own reviewers were publicly dismantling that move. Use it where it belongs, in the benefit-risk judgment at the end.
Spend the design effort on covariate adjustment and allocation before spending it on borrowing. Prognostic scores, unequal randomization and within-patient designs buy efficiency without importing an assumption about an external population. Borrowing buys more, and costs more, and the panel’s own framing was that each borrowed patient carries a borrowed risk.
The workshop produced no recommendations and was explicitly not a federal advisory committee. What it offers sponsors is a reading of how the agency’s statistical reviewers are currently reasoning, several months ahead of anything that might become guidance.
References
- FDA, RISE Workshop 4: RISE to the Challenge: Statistical Considerations for Rare Disease Clinical Investigations, September 29, 2026, National Press Club, Washington, DC. Session two: tailoring clinical investigations and navigating trade-offs.
- RISE Workshop #4 pre-read (PDF), co-convened by Duke-Margolis and the FDA Rare Disease Innovation Hub.
- Federal Register, “Statistical Considerations for the Design of Rare Disease Clinical Investigations,” September 15, 2026. Docket FDA-2026-N-10165.
- Docket FDA-2026-N-10165 on Regulations.gov.
- Workshop agenda (PDF), Duke-Margolis Institute for Health Policy.
- Duke-Margolis, RISE Workshop Series.
- Full workshop recording (video). Quotations are taken from the recording and reported by role rather than by name.
- Part one of this series: what the agency said about randomization and control arms.
Moe Alsumidaie, MBA, MSF, is founder and Chief Editor of Vanguard Publications, which publishes Clinical Trial Vanguard, Pharma Vanguard and BullScope, and Head of Research at CliniBiz. He has two decades in clinical trial operations and data science, with earlier roles at Genentech, Abbott Vascular and Stanford University Medical Center, and is a guest lecturer in clinical trial sciences at Rutgers University.
