“Should we buy or build?” is usually the first question organizations ask about radiology AI. It is useful, but it is no longer the whole question.
Today, you can choose from cleared clinical algorithms, enterprise AI platforms, generative-AI tools, and internally developed workflows. The real challenge begins after a contract is signed or a prototype works. AI becomes another clinical system: one that can help, distract, confuse, or quietly change behavior.
Anyone who has lived through an enterprise IT rollout, PACS replacement, or hospital merger (let alone lead an aspect of it) knows the pattern. Installation is only one part of deployment. The hard work is making a tool fit the people, data, incentives, exceptions, and failure modes of the place where it will be used.
Start With the Problem
AI deployment still needs ordinary project discipline: a clear problem, an accountable owner, resources, risk management, and a definition of success. “Improve workflow” is not a metric. Neither is “use more AI.” A useful plan identifies the workflow that will change, who is affected, what tradeoff is acceptable, and how the organization will know whether patients or clinicians are actually better off. We wrote about it here.
That framing also helps separate a promising demonstration from a durable product.
Usability Is Safety
AI systems are often judged by their models, but clinicians experience them through interfaces. Where does the result appear? Is it understandable? Does it arrive when a radiologist can still act on it? Does it create an alert that is easy to ignore, or hard to dismiss when it should be?
A vendor has to build a usable product. The health system has to determine whether it is usable locally. A model can perform well in a study and still be poorly placed in the real workflow.
Validation Is Local and Ongoing
Regulatory clearance, security review, and vendor documentation matter.
They are starting points, not substitutes for local testing. For radiology AI, that means following the actual data path, worklist, report workflow, and the cases in which the tool is likely to fail.
Neither building nor buying exempts the due diligence for what happens after launch. Protocols change. Patient populations differ. Software versions and integrations evolve. Generative tools can produce plausible but incorrect text. Performance, safety, and adoption should be reviewed over time not assumed from a pre-launch checklist.
Don’t Forget Your Outcome Metrics
A tool that saves a few minutes in a controlled pilot may create more downstream work if it adds clicks, exceptions, or new review queues at scale. A tool that optimizes one team can add twice the equivalent work to another.
Just because the model works doesn’t guarantee that it helps solve the problem. An outcome metric is a measurement of whether the issue is overall better (or worse) and requires holistic evaluation of the whole workflow not just the AI.
If a model validates well when predicting 30-day readmission rate with local data, does it actually bend the curve when implemented? Has 30-day readmission rate changed six months after implementation, hopefully for the better?
Regardless of whether you are buying or building, forgetting the measure the impact of the technology means you are trading a definite cost with uncertain benefit.
Governance Is Part of the Product
The current questions go beyond accuracy. Who is responsible for monitoring the tool? What data leave the organization, and how may they be used? Can the system be audited? How are cybersecurity vulnerabilities, model updates, and unexpected behavior handled? Can the organization switch vendors or retrieve its own data if the partnership ends?
These are not procurement details to settle later. An AI governance committee/team’s job here is to make sure whether a useful pilot can become a safe, sustainable service.
The Work Continues
Most clinicians will not learn an AI tool by reading a manual. They will learn by using it, encountering its limits, and asking better questions after real cases. That makes training, support, feedback, and a clear way to pause or revise the workflow essential.
Buy versus build still matters. But the better question is whether the organization can deploy and govern the tool as a force for good: measured, monitored, corrected, and shaped around patient care rather than around the mere availability of an algorithm.