Reinforcement Learning for Ultrasound Image Analysis A Comprehensive Review of Advances and Applications

Over the last decade, the use of machine learning (ML) approaches in medicinal applications has increased manifold. Most of these approaches are based on deep learning, which aims to learn representations from grid data (like medical images). However, reinforcement learning (RL) applications in medicine are relatively less explored. Medical applications often involve a sequence of subtasks that form a diagnostic pipeline, and RL is uniquely suited to optimize over such sequential decision-making tasks. Ultrasound (US) image analysis is a quintessential example of such a sequential decision-making task, where the raw signal captured by the US transducer undergoes a series of signal processing and image post-processing steps, generally leading to a diagnostic suggestion. The application of RL in US remains limited. Deep Reinforcement Learning (DRL), that combines deep learning and RL, holds great promise in optimizing these pipelines by enabling intelligent and sequential decision-making. This review paper surveys the applications of RL in US over the last decade. We provide a succinct overview of the theoretic framework of RL and its application in US image processing and review existing work in each aspect of the image analysis pipeline. A comprehensive search of Scopus filtered on relevance yielded 14 papers most relevant to this topic. These papers were further categorized based on their target applications image classification, image segmentation, image enhancement, video summarization, and auto navigation and path planning. We also examined the type of RL approach used in each publication. Finally, we discuss key areas in healthcare where DRL approaches in US could be used for sequential decision-making. We analyze the opportunities, challenges, and limitations, providing insights into the future potential of DRL in US image analysis.