Category Archives: Data Science and Machine Learning

Learning AI From Radiology’s Messy Middle

I am frequently asked by high school and college students what AI in radiology means, and by residents how to enter the technical side of imaging AI models. More importantly, how to build one from the real life of radiology?

This year’s RSNA Knee Abnormality Detection Challenge is one answer.

It is not a pristine tutorial dataset. Competitors will work from carefully pooled knee MRI images and their reports in 10 languages, sourced from 19 sites across 5 continents: the hedges, imperfect descriptions, and occasional errors that make clinical data both difficult and familiar.

The task is also not accuracy at all costs. Models must be efficient, and they must address a representative set of 12 knee abnormalities, not one model for one diagnosis.

That tension is the point. Building useful AI means confronting what radiology actually produces, then deciding what is worth measuring and improving.

As of August 17, there have been 12,853 joined users, 1,890 competitors across 1,761 active teams, and 11,456 submissions; the public leaderboard’s top, median, and last-listed scores are 0.951, 0.8745, and 0.468, respectively.

I am grateful to the RSNA staff, the Challenge Task Force, and especially Dr. Errol Colak for their leadership. I am equally thankful to my co-lead, Dr. Naveen Subhas. This Challenge could not have been pulled together by just one team: we owe this work to over 100 radiologists, biostatisticians, and data scientists across the world who volunteered their time reviewing reports, annotating images, offering advice, and contributing data.

We hope this challenge gives learners a practical place to begin and a clearer sense of the questions that remain.

AI Can Raise the floor, But People Determine The Ceiling.

AI readiness is often treated as a technical skill, as though one could study for it, take a test, and be done. It is a pitfall particularly in education – what does it even mean to teach residents to be ready for the AI age?

Researchers at UT Austin asked 523 early-career professionals to do client-like work with an AI agent, then compared their results with an AI-only baseline. About half were “AI Amplifiers”: they did better than AI alone. A quarter were “Delegators”: their work was about as good as the AI’s. The rest, “Apprentices,” did worse, even though they were not necessarily less capable.

Yes – about a quarter of the early-career professional appeared to produce worse outputs when using AI compared to than just having it done by AI alone. (The reality is probably that the researchers were better prompt engineers than the apprentices).

Strong users framed the problem, used real domain knowledge, checked the output, and refined it over several rounds. AI was used as a collaborator that needed direction and oversight.

This feels familiar in resident education. An algorithm can produce a useful first pass. Safe clinical value still depends on whether the clinician asks the right question, spots what does not fit, and knows what to do next.

The lesson is not to teach longer prompts. It is to teach judgment in the workflow: frame, check, refine, decide.

AI can raise the floor. People still determine the ceiling.

Source: McCombs School of Business, University of Texas at Austin.

Navigating AI Decisions: Going Beyond “Buy vs Build”

Conceptual illustration of an AI buy-versus-build crossroads with governance and monitoring decisions

“Should we buy or build?” is usually the first question organizations ask about radiology AI. It is useful, but it is no longer the whole question.

Today, you can choose from cleared clinical algorithms, enterprise AI platforms, generative-AI tools, and internally developed workflows. The real challenge begins after a contract is signed or a prototype works. AI becomes another clinical system: one that can help, distract, confuse, or quietly change behavior.

Anyone who has lived through an enterprise IT rollout, PACS replacement, or hospital merger (let alone lead an aspect of it) knows the pattern. Installation is only one part of deployment. The hard work is making a tool fit the people, data, incentives, exceptions, and failure modes of the place where it will be used.

Continue reading

FDA AI Guidance and the Hard Part of Transparency

The least glamorous part of AI in radiology may turn out to be the most important: telling people what changed.

That sounds simple. It is not. A diagnostic AI tool may be trained on one dataset, validated on another, deployed inside a PACS or reporting workflow, monitored after release, and then updated when the model, threshold, input, interface, or intended environment changes. Somewhere in that chain, a radiologist is expected to decide whether to trust a box, a score, a contour, a triage flag, or a sentence.

Continue reading

AI Readiness as an Educational Obligation

I wrote a guest editorial in Academic Radiology about AI readiness as a practical educational obligation, rather than an optional informatics side quest.

The piece responds to a study of medical undergraduates and radiology trainees in China, but the issue is broader: enthusiasm and exposure do not equal competence. Radiologists need enough AI literacy to recognize workflow fit, automation bias, governance gaps, and the limits of machine suggestions. Most trainees will not become model developers.

They still need to become safe, skeptical operators who can use AI with confidence and accountability in real clinical environments, worldwide, across diverse resource settings today.

What Radiology Teaches Us About AI and the Workforce

Health systems frequently ask whether artificial intelligence will ultimately reduce the need for clinicians. Radiology provides one of the clearest real-world answers to date. AI changes the mechanics of work far more than it changes the need for expertise.

I had the privilege to speak with CNN on this topic recently. The story is out now.

In imaging operations, AI is already being used to support exam prioritization, improve image reconstruction, and reduce friction in routine workflows. These contributions are meaningful, particularly in high-volume environments. But they do not displace clinical judgment, professional accountability, or responsibility for patient outcomes. In practice, the performance gains attributed to AI are inseparable from expert oversight and careful integration into clinical teams.

We use AI as a capacity and quality multiplier, not as a substitute for our training. That line is difficult to walk and gets thinner as products improve, but the thought matters. Deploying it primarily as a justification for workforce reduction or skill substitution introduces avoidable risk to patient safety and physician trust.

Success ultimately go to those investing in workflow, governance structures, and building solution that define clear roles for both humans and machines. Radiology may be an early example, but the underlying lesson extends well beyond imaging.

The FDA has an Idea, or 10, on Good Machine Learning Practice

Good machine learning practices goes beyond the chip

The U.S. FDA, Health Canada, and the UK’s MHRA have unveiled 10 guiding principles for Good Machine Learning Practice (GMLP) in developing AI/ML medical devices. These principles aim to ensure safety, efficacy, and quality in healthcare innovation. Key focuses include leveraging multi-disciplinary expertise, implementing good software and security practices, ensuring representative clinical study participants and data sets, maintaining independence between training and test data sets, and emphasizing the performance of the human-AI team. These guidelines also highlight the importance of clear user information, robust testing, and ongoing monitoring of deployed models to manage re-training risks and maintain performance.

Read the full GMLP draft on the FDA website.

1-Minute Summary

Here are the ten principles of GMLP.

Continue reading

Artificial Intelligence: Are you a Centaur or a Cyborg?

In the rapidly evolving field of radiology, artificial intelligence (AI) is not just a tool but a collaborator, reshaping the dynamics of diagnosis and patient care. On the first order, the answer seemed clear: knowledge workers using AI outperforms those that don’t.

But the literature offers little detail on what happens after you embrace AI. Just with every tool ever existed, it really matters how you use it. As it turns out, it also matters how AI becomes part of your work.

To better understand this partnership, a group of Harvard Business School scholars published a study on business consultants who have, and who have not, opted to adopt GPT-4 in their daily work in spring 2023. There are several interesting conclusions – one of them delve into the analogy of centaurs versus cyborgs, concepts borrowed from mythology and science fiction that provide a vivid framework for the interaction between human intelligence and AI in radiology.

Continue reading

AI Regulation in Healthcare: CMS and Congressional Scrutiny

AI regulation in healthcare is coming. This article from FierceHealthcare summarizes the growing use of artificial intelligence (AI) in healthcare innovations and the increasing scrutiny from Senate lawmakers regarding AI regulation and healthcare payment. It highlights the concerns of bias in AI systems and the legislative scrutiny to ensure these technologies benefit patient care without discrimination. The discussion also covers lawsuits against major Medicare Advantage insurers for allegedly using AI to deny care, and the Centers for Medicare & Medicaid Services’ (CMS) guidance on AI use in healthcare decisions. Additionally, the need for transparency, accountability, and meaningful human review in AI applications in healthcare is emphasized, alongside calls for federal support to navigate AI’s integration into healthcare practices responsibly.

I urge you to read the full article which includes links to the first-hand sources and supplement with a short summary below for the busy professional.

1-Minute Summary

  • Federal lawmakers are actively discussing the impact of artificial intelligence (AI) in healthcare, emphasizing the need to protect patients from bias inherent in some big data systems without stifling innovation. These biases can discriminate against patients based on race, gender, sexual orientation, and disability.
  • To ensure the beneficial outcomes of AI while safeguarding patient rights, the Algorithmic Accountability Act was introduced. This act mandates healthcare systems to regularly verify that AI tools are being used as intended and are not perpetuating harmful biases, especially in federal programs like Medicare and Medicaid.
  • Major Medicare Advantage insurers, including Humana and UnitedHealthcare, are under legal scrutiny for allegedly using AI algorithms to deny care, highlighting the challenges of implementing AI in patient care decisions without exacerbating discrimination or introducing new biases.
  • The Centers for Medicare & Medicaid Services (CMS) issued guidelines prohibiting the use of AI or algorithms for making coverage decisions or denying care based solely on algorithmic predictions, emphasizing the necessity for decisions to be based on individual patient circumstances and reviewed by medical professionals.
  • Testimonies during the legislative hearings called for additional clarity on the proper use of AI in healthcare, suggesting the establishment of AI assurance labs for developing standards, and advocating for federal support to help healthcare organizations navigate the use of AI tools through investments in technical assistance, infrastructure, and training.

American College of Radiology

The emphasis on AI transparency as an answer to bias is not new. The American College of Radiology (ACR) recently kicked off its Transparent-AI initiative to advocate for openness and trust in AI. The program invites all manufacturers with FDA-cleared AI tools to participate. By offering detailed insights into an algorithm’s training, performance, and intended use, Transparent-AI not only boosts product credibility but also aids in integrating these innovations into diverse healthcare environments. Behind the scenes, the ACR has also advocated for transparency in AI with various federal agencies and lawmakers.

Disclosure: I am not involved with Transparent-AI but do sit on the ACR Commission on Informatics and chair the ACR Informatics Advisory Council and annual ACR DSI Summit. Register for the 2024 DSI Summit here!

AI+Human Better than Human in Neurodegenerative Imaging

Recent research underscores a leap in neuroimaging accuracy for Alzheimer’s disease diagnosis, emphasizing the superior performance of AI-assisted radiologists over either AI or humans alone. This collaborative approach marries the meticulous precision of AI with the nuanced understanding of human experts, potentially setting a new standard in the detection of amyloid-related imaging abnormalities. Specifically, it demonstrated superior performance in detecting amyloid-related imaging abnormalities (ARIA), crucial for amyloid-β–directed antibody therapy. This synergy enhances diagnostic precision and underscores the potential of AI-enhanced radiological diagnostics to improve patient care significantly.

How will this synergy between AI and human intelligence redefine the future of medical diagnostics? Can this model be the blueprint for addressing other complex diseases? This breakthrough prompts us to envision a healthcare landscape where technology and human expertise converge to offer unparalleled patient care.

Detailed study can be found in JAMA Network Open.