‘ReXplain’ Is Transforming Radiology With AI Like Never Before

A deep dive into 'ReXplain', an AI tool that creates personalized video radiology reports to communicate findings to patients effectively

Image generated with DALL-E 3

Have you ever dealt with Radiology reports?

They look like large black-and-white sections with occasional bursts of colour unless you're a Radiologist.

I even struggle with some of them, even though I have been practising medicine for the last five years and studying it for the last six before that.

Intravenous contrast-enhanced CT scan of the human head (Source: Mikael Häggström, the Department of Radiology, Uppsala University Hospital. on Wikimedia Commons)

Worry not! It’s time for some exciting news!

A team of researchers has created an AI-driven tool to solve this problem.

In their pre-print published in ArXiv, they describe ‘ReXplain’ or ‘Radiology eXplaination’, a tool which integrates —

• LLM for Radiology report simplification

• Image segmentation model to identify and highlight different anatomic regions in Radiology images

• Avatar generator model that creates human avatars that explain the radiology findings to patients in simple lay language

All of these work together to create a video report that can accurately deliver radiology findings to a patient, simulating an easy-to-understand one-to-one consultation.

Comparison between the conventional and ReXplain-generated video radiology report (Image from the original research paper)

This is incredible!

Here’s a story where we deep dive into how ‘ReXplain’ works and how tools like this improve patient care through AI-enabled multi-modal communication like never before.

Let’s go!

---

How ‘ReXplain’ Works

ReXplain borrows insights from a clinical workflow where radiologists or clinicians walk patients through their CT radiology reports.

It is noted that there are three important steps in this process.

These are:

• Explaining essential findings to a patient in plain or lay language

• Pointing at abnormalities in the radiology images

• Telling the patient about how these images would look if they were from a healthy individual

Next, AI pipelines are created for each of these steps, resulting in a patient-friendly video report.

Let’s learn how this is done at each step.

Step 1: Converting Radiology Reports To Lay Language

GPT-4o is used at this step, where it is prompted to extract phrases that describe positive findings from the radiology reports.

After this, GPT-4o generates more information about these phrases, where it —

• Explains what these findings are

• Describes how these findings appear on a CT scan

• Describes how a CT scan would normally appear without these findings

The prompt used in this step is shown below.

Report phrase explanation prompt given to GPT-4o (Image from the original research paper)

Step 2: Connecting Extracted Findings To Radiology Images