Successful Applications of Machine Learning in Medical Imaging

Medical imaging allows health care providers to peer inside patients without the need for an operation. X-ray, ultrasound, computed tomography (CT), magnetic resonance imaging, and positron emission tomography (PET) are technologies—with different capabilities and therefore different advantages and disadvantages—for imaging the body’s internal structures. After an image has been taken, it becomes the job of a physician, often a radiologist, to interpret the image and use it to inform a diagnosis. Because image interpretation relies heavily on pattern recognition, it has long been thought that machine learning algorithms, trained on massive datasets of medical imaging scans, could perform well at this task. Though the technology is not without its faults, machine learning has been proven useful in medical imaging.

A review article by Khalifa et al.¹ provides a helpful framework through which to understand successful applications of machine learning in medical imaging. The authors identify four domains in which machine learning has exhibited the potential to improve the accuracy of medical imaging interpretation and boost its efficiency. The first is “image analysis and interpretation,” or the ability to spot complex patterns in medical images, some of which may be imperceptible to the human eye. In many types of imaging, including CT scans for lung cancer,² mammography scans for breast cancer,³ and chest x-rays for heart abnormalities,⁴ machine learning has been shown to match, or even exceed, the performance of radiologists in identifying pathological features of many conditions.

The second domain is “operational efficiency.” Machine learning algorithms can analyze medical images with great speed and without succumbing to the fatigue, and resulting slip in performance, that can impact radiologists. Some machine learning algorithms can read and diagnose x-rays in seconds, delivering a speed that can be crucial in emergency situations. Additionally, their accuracy can potentially save costs by minimizing misdiagnoses and the need to repeat imaging.

The third domain, “predictive and personalized healthcare,” refers to machine learning’s capability to make predictions for healthy patients on the basis of imaging. For instance, one study showed that an automated analysis of cardiovascular magnetic resonance imaging was highly accurate in predicting the incidence of major cardiac events,⁵ potentially enabling these patients to receive rigorous monitoring and undergo preventative care.

Finally, the fourth domain, “clinical decision support,” describes how machine learning can provide image analysis that can assist physicians during surgery and can integrate with electronic health records to provide health insights.

Although some benefits of using machine learning for medical imaging have been identified, they are not guaranteed. According to research from investigators at Harvard Medical School, different radiologists interact with machine learning in different—sometimes unpredictable—ways.⁶ In the study, years of experience and prior use of machine learning did not impact a radiologist’s performance when using machine learning tools to make a diagnosis from a chest x-ray. The research suggests that the successful application of machine learning to medical imaging may rely on factors that are not yet understood, demonstrating the need for further research on the subject.

References

1. Khalifa, M. & Albadawy, M. AI in diagnostic imaging: Revolutionising accuracy and efficiency. Computer Methods and Programs in Biomedicine Update 5, 100146 (2024), https://doi.org/10.1016/j.jdent.2022.104080

2. Abadia, A. F. et al. Diagnostic Accuracy and Performance of Artificial Intelligence in Detecting Lung Nodules in Patients With Complex Lung Disease: A Noninferiority Study. Journal of Thoracic Imaging 37, 154 (2022), DOI: 10.1097/RTI.0000000000000613

3. McKinney, S. M. et al. International evaluation of an AI system for breast cancer screening. Nature 577, 89–94 (2020), DOI: 10.1038/s41586-019-1799-6

4. Bhave, S. et al. Deep learning to detect left ventricular structural abnormalities in chest X-rays. Eur Heart J 45, 2002–2012 (2024), DOI: 10.1093/eurheartj/ehad782

5. Backhaus, S. J. et al. Artificial intelligence fully automated myocardial strain quantification for risk stratification following acute myocardial infarction. Sci Rep 12, 12220 (2022), https://doi.org/10.1038/s41598-022-16228-w

6. Yu, F. et al. Heterogeneity and predictors of the effects of AI assistance on radiologists. Nat Med 30, 837–849 (2024), https://doi.org/10.1038/s41591-024-02850-w