Use the AI Canvas.
Source: A Simple Tool to Start Making Decisions with the Help of AI
In a recent Harvard Business Review article, Ajay Agrawal and coauthors shared a simple tool to think about how an AI tool may be deployed. Although the tool is no more complex than 6 boxes of free text, it does follow a number of best practices when thinking about general data and machine learning:
- Always define an end-goal – what’s the desired outcome?
- You should always make a hypothesis of what may drive this desired outcome.
- You should determine how to present the ML prediction in a way that drives action, not just the data itself.
- Your data acquisition strategy should include a feedback mechanism.
For example, this is how one might fill out the AI Canvas tool in a radiology use case:
- Prediction: Predict whether a brian MRI for a cancer patient contains increasing or new hydrocephalus
- Action: Label the examination as critical, and denote that AI has determined a critical finding. For example, create an “AI-STAT” category on worklist priority.
- Judgment: Compare the cost of interpreting this brain MRI at its usual turnaround time, versus immediately.
- Outcome: Observe whether the action taken in response to a study labeled AI-STAT is correct.
- Input: New MRIs of the brain MRI performed, and their prior studies.
- Training: Historical brain MRIs
- Feedback: Identify false positives – perhaps the prior study was from 20 years ago, or there’s been surgical resection, so that ex vacuo dilation of ventricles is not hydrocephalus. Perhaps there has been recent surgery Identify false negatives – subtle enlargement of the temporal horns missed by AI. Use this information to improve the AI.
How might you use this worksheet to brainstorm AI for your radiology practice?
Today’s world provides us with tools that make humans more capable than ever. Writers who used to make elaborate trips to exotic locations to gleam material for the next espionage thriller while talking to their book agent on the landline with expensive long distance fees can now do the research at their computers while setting up an Amazon self-publishing account. Radiologists who used to be at the mercy of transcriptionists to translate their verbal stutters into fluent medical poetry days later now generate reports within minutes using voice recognition technology.
These are empowering tools, putting the ability to affect outcome directly in the hand of those holding the highest stakes. In general terms, it makes sense that with advancements in technology, professionals can now (1) do the same amount of work with better quality, or (2) do more work at the same quality.
Unfortunately, inadvertently what happens is we tend to be expected to accomplish more and do better (occasionally one also expects to feel less tired at the end of it!) Business school professors would teach that technology advances push the entire pareto-efficient frontier forward. That is, assuming that you are already working at your absolutely most efficient way such that any improvement in speed will automatically have a quality tradeoff, then adapting a new technology may change the nature of the curve such that you can now move both “up” and “to the right.” The truth is, I am so rarely pareto-efficient in the first place that if a new technology can somehow land me onto my existing frontier, it was well worth the cost. And while technology like this, this, and this don’t literally breaking any frontiers, they do have the added benefit of putting productivity in my conscious thought and – at least temporarily – make me healthier and more productive.