Tag Archives: Artificial Intelligence

Learning AI From Radiology’s Messy Middle

I am frequently asked by high school and college students what AI in radiology means, and by residents how to enter the technical side of imaging AI models. More importantly, how to build one from the real life of radiology?

This year’s RSNA Knee Abnormality Detection Challenge is one answer.

It is not a pristine tutorial dataset. Competitors will work from carefully pooled knee MRI images and their reports in 10 languages, sourced from 19 sites across 5 continents: the hedges, imperfect descriptions, and occasional errors that make clinical data both difficult and familiar.

The task is also not accuracy at all costs. Models must be efficient, and they must address a representative set of 12 knee abnormalities, not one model for one diagnosis.

That tension is the point. Building useful AI means confronting what radiology actually produces, then deciding what is worth measuring and improving.

As of August 17, there have been 12,853 joined users, 1,890 competitors across 1,761 active teams, and 11,456 submissions; the public leaderboard’s top, median, and last-listed scores are 0.951, 0.8745, and 0.468, respectively.

I am grateful to the RSNA staff, the Challenge Task Force, and especially Dr. Errol Colak for their leadership. I am equally thankful to my co-lead, Dr. Naveen Subhas. This Challenge could not have been pulled together by just one team: we owe this work to over 100 radiologists, biostatisticians, and data scientists across the world who volunteered their time reviewing reports, annotating images, offering advice, and contributing data.

We hope this challenge gives learners a practical place to begin and a clearer sense of the questions that remain.

Navigating AI Decisions: Going Beyond “Buy vs Build”

Conceptual illustration of an AI buy-versus-build crossroads with governance and monitoring decisions

“Should we buy or build?” is usually the first question organizations ask about radiology AI. It is useful, but it is no longer the whole question.

Today, you can choose from cleared clinical algorithms, enterprise AI platforms, generative-AI tools, and internally developed workflows. The real challenge begins after a contract is signed or a prototype works. AI becomes another clinical system: one that can help, distract, confuse, or quietly change behavior.

Anyone who has lived through an enterprise IT rollout, PACS replacement, or hospital merger (let alone lead an aspect of it) knows the pattern. Installation is only one part of deployment. The hard work is making a tool fit the people, data, incentives, exceptions, and failure modes of the place where it will be used.

Continue reading

FDA AI Guidance and the Hard Part of Transparency

The least glamorous part of AI in radiology may turn out to be the most important: telling people what changed.

That sounds simple. It is not. A diagnostic AI tool may be trained on one dataset, validated on another, deployed inside a PACS or reporting workflow, monitored after release, and then updated when the model, threshold, input, interface, or intended environment changes. Somewhere in that chain, a radiologist is expected to decide whether to trust a box, a score, a contour, a triage flag, or a sentence.

Continue reading

Your Radiology AI Briefing – May 3, 2018

In this briefing:

  • RSNA launches a new AI journal.
  • ACR makes several moves to advance the use of artificial intelligence in the future of medical imaging.
  • Researchers publish in high impact journal the successful use of AI to detect breast density.
  • ACR and MICCAI sign agreement to advance AI in medical imaging.
  • Geisinger declares successful implementation of algorithm to improve time-to-diagnosis of intracranial hemorrhages 20-fold.

Radiology AI Briefing logo graphic

Stay up to speed in 2 minutes. Radiology AI Briefing is a semi-regular series of blog posts featuring hand-picked news stories and summaries on machine learning and data science.


Continue reading

6 Elements of a Data-Driven Informatics Solution (1/3)

Big Data has become a radiology buzzword  (the others: machine learning, AI, and disruptive innovation are also up there).

However, there is a real problem with using the term Big Data – it isn’t just one set of data problems.  Big Data is a conglomerate of different data challenges: volume of data, heterogeneity of data, or the velocity of data are all important dimensions.  Machine learning and internet of things are others layers superimposed on the big data problem.

Sometimes it is helpful to step back and approach data problems with a common framework, a way to think about how and which facets of data science fit in a real-life workflow in the face of an actual problem.

Below is a 6-element framework that helps me think about data-driven informatics problems. They are generally in chronological order, but they are not “steps” because you frequently will find yourself going back and redefining many things.  However, the framework helps you maintain a big-picture outlook.  The reason any sufficiently complex data problem requires a team approach. Continue reading

The [machine learning] race is on – Don Dennison

Continue reading

Machine learning, real opportunites: Dr. Keith Dreyer’s keynote sets tone for ISC 2016

Dr. Keith Dreyer opens with a keynote during the Intersociety Summer Conference (ISC) with description of data science and overview of how machine learning have evolved over time.

He describes that machines and humans inherently see things differently. Humans are excellent at object classification, recognition of faces, understanding language, driving, and imaging diagnostics. Continue reading

Aside

The use of the phrase, “Artificial Intelligence” has exploded within the past few years as the theme of dozens of our most popular movies and television shows, magazines, books, and social media. This is despite the difficulty that many experts … Continue reading

Watson Will Replace Me? Not A Chance

Arthur C. Clark and Stanley Kubrick predicted supercomputers more intelligent than humans.  In 2001: A Space Odyssey, the HAL states, with typical human immodesty, “The 9000 series is the most reliable computer ever made… We are all, by any practical definition of the words, foolproof and incapable of error.” Forty years later, IBM’s Watson pummeled humans in Jeopardy – a distinctly human game. Continue reading

Can Dr. Watson Practice Medicine?

“Mathematical reasoning may be regarded rather schematically as the exercise of a combination of two facilities, which we may call intuition and ingenuity.” – Alan Turing

Sherlock Holmes is fictional expert in what he calls the “exact science of detection” (A Study in Scarlet). Despite his genius in deductive reasoning and intuition is unparalleled, much of the detective success relies upon the calm and composed guidance of his trusty sidekick Dr. Watson. In most of the canonical novels, Watson acts as the sanity check for Holmes’ storm of ideas and, of course, the meticulous chronicler of their adventures together.

After defeating its human opponents on Jeopardy, the supercomputer Watson by IBM will attempt to learn medicine. Despite its terabytes of storage and raw processing horsepower, Watson’s ability to make medical decisions remains unclear. Can IBM’s Watson truly understand the complex human body and make medical decisions, or will it – like Dr. Watson attempting deduction – prove to be an helpful sounding board but falling short of achieving true intuition?
Continue reading