Picture a coordinator three weeks into a new oncology study, running the eligibility checklist against her first five screen candidates. Two fail on a biomarker criterion that the PI flagged during the SIV as potentially over-restrictive. She logs the screen failures, escalates to the CRA, and gets the answer sponsors always give when you catch this problem at the site level: “We’ll look at it during the next protocol review.” Four more screen failures later, the sponsor submits an amendment. The protocol amendment triggers a central IRB resubmission, an average of 21 days of lost enrollment time, and a budget renegotiation that the site never fully recovers from. The coordinator already knew the eligibility criterion was wrong. She knew it on day three.
That scenario — flawed assumptions embedded in protocol design, unchallenged until site execution exposes them — is exactly what Phesi’s new analysis is pointing at. The clinical trial data company, founded in 2007 by Dr. Gen Li, found that fewer than one in three trial protocols are connected to documented patient data and outcomes at the design stage. The industry response to this failure has been to layer AI on top of the existing protocol development process. Phesi’s conclusion is blunt: flaws are being scaled, not solved.
The Problem Lands at the Site, Not the Sponsor
Sponsors tend to frame protocol-data disconnects as an upstream design problem — something for medical affairs and biostatistics to sort out before the protocol gets to the site. That framing is operationally wrong. By the time a flawed protocol reaches a site, it has already been embedded in the EDC build, the CTMS enrollment projections, the central lab kit manifest, the IRT randomization logic, and the ICF language. Changing any one of those after site initiation is not a revision; it is a cascade.
The amendment data tells the story numerically. Research published in Clinical Trials found that the prevalence of protocols with at least one amendment increased from 57% in 2015 to 76%, with the mean number of amendments per protocol rising 60% to 3.3. Phase III protocols now average 3.5 substantial amendments each, and the total average duration to implement a substantial amendment has nearly tripled over the same period. Every one of those amendments hits a coordinator’s desk as an updated protocol version, a consent form re-execution workflow, and a set of open data queries that reference procedures the site may have already performed under the old version. The coordinator who caught the eligibility problem on day three is now re-consenting enrolled subjects, fielding a data clarification request from the CRA, and updating the site’s deviation log — all because the protocol was never stress-tested against real patient population data before it was finalized.
ICH E6(R2) Section 5.0 is unambiguous on where responsibility sits: the ultimate obligation for the quality and integrity of trial data always resides with the sponsor, regardless of what has been delegated to a CRO. The addendum is explicit that oversight of delegated functions is a non-negotiable sponsor duty. In practice, the gap between that regulatory expectation and actual protocol development workflows is where Phesi’s finding lives. If fewer than one in three protocols are grounded in documented patient data, then sponsors are delegating design decisions to assumptions — and then delegating the consequences of those assumptions to sites.
The monitoring economics make this worse, not better. TransCelerate’s risk-based monitoring position paper found that 100% SDV drives only 2.4% of queries on critical data, yet SDV still consumes over 50% of on-site monitoring budgets at many sites. That imbalance exists partly because sponsors default to intensive on-site monitoring as a backstop for data quality problems they should have prevented at the protocol design stage. When the protocol is poorly mapped to actual patient populations, EDC forms fill with anomalies that look like data entry errors but are actually structural — the visit window doesn’t reflect how the patient population actually presents, the dosing schedule assumes compliance rates the disease population cannot sustain, the endpoint definition doesn’t match what source documents at this site type will ever contain. SDV finds none of that. It just generates queries.
What AI Scales When the Foundation Is Wrong
The uncomfortable operational truth in Phesi’s analysis is that AI optimization tools in trial design are being applied to a flawed input. If the protocol was never connected to documented patient data outcomes in the first place, an AI tool trained to optimize that protocol is learning from a fiction. It will produce a faster, more internally consistent fiction — but the screen failure rate at your site will tell you what the algorithm did not know.
Sites I work with across our network have begun seeing this pattern in a specific way: a protocol arrives that has clearly been through AI-assisted feasibility and enrollment modeling, and the enrollment projections are aggressive but defensible on paper. Then the SIV happens, and the PI starts asking about the assumed screen-to-enroll ratio. The projection often assumes a 3:1 or 4:1 screen failure rate. In practice, for that therapeutic area, at that site type, with that patient population, 8:1 is closer to reality. The AI model didn’t know that because the protocol wasn’t built on outcomes data from comparable patient cohorts at comparable sites. It was built on the same structural assumptions that have produced the amendment rate surge documented in the literature — just produced faster.
The FDA’s enforcement history reinforces why this matters beyond budget overruns. The agency’s actions against Raptim Research, documented in an FDA untitled letter finding studies conducted by Raptim Research Pvt. Ltd. unacceptable, turned in part on data-protocol misalignment that was traceable through the trial record. Inspectors following GCP inspection standards look for exactly the kind of disconnect that Phesi’s analysis is quantifying: source data that cannot be reconciled to protocol-specified procedures, EDC entries that reflect what the protocol said should happen rather than what actually happened with a real patient. A 483 observation for inadequate source documentation is often a downstream symptom of a protocol that was never grounded in how that procedure would actually be executed in a real clinical environment.
What Changes Monday Morning
For sites, the immediate operational lever is the feasibility questionnaire — specifically, demanding that it be treated as a binding operational input rather than a pre-award formality. Sites across our network have started adding a pre-startup reconciliation step: before the site initiation visit, the site coordinator or director reconciles the protocol’s assumed screen failure rate, visit window tolerances, and endpoint source documentation requirements against the site’s actual historical data for that patient population. If the reconciliation surfaces a gap of more than 30% on any parameter, the site escalates formally before the SIV — not after the first monitoring visit. Sponsors may push back on the timing; sites that have done this consistently find that the pushback costs far less than the amendment cycle that follows when the gap surfaces in the enrollment data six months later.
For sponsors and CROs, Phesi’s finding points to a specific change in protocol development governance: documented patient data and outcomes need to be a required input artifact in the protocol finalization package, the same way statistical analysis plans and IB updates are. FDA’s draft guidance on protocol deviations creates accountability for what happens when protocols fail in execution — but the cleaner intervention is upstream. A protocol review committee that cannot produce a data reconciliation showing alignment between the eligibility criteria and documented outcomes from a comparable patient population is not ready to submit. AI tools can accelerate the review once that foundation exists. Applied before it exists, they accelerate the mistake.
The coordinator who flagged those screen failures on day three was the last human checkpoint before six months of amendment-driven pain. The industry keeps building faster tools to bypass that checkpoint. Until protocols are built on documented patient data — not assumed populations — sites will keep being the place where the fiction meets reality, query by query, deviation by deviation, amendment by amendment.
References
- FierceBiotech — “Clinical trial flaws ‘being scaled, not solved’ by AI: report”
- LeadIQ — Phesi company profile, founding year 2007, Dr. Gen Li
- PubMed — Protocol amendment prevalence and amendment rate data, Clinical Trials journal
- ICH — E6(R2) Addendum: Sponsor responsibility for CRO-delegated functions and data integrity
- Medidata — TransCelerate RBM position paper: 2.4% of critical data queries driven by SDV
- FDA — Untitled letter: Raptim Research Pvt. Ltd. studies deemed unacceptable
- FDA — Draft Guidance: Protocol Deviations in Clinical Investigations
