H-Net: A Multitask Architecture for Simultaneous 3D Force Estimation and Stereo Semantic Segmentation in Intracardiac Catheters

The success rate of catheterization procedures is closely linked to the sensory data provided to the surgeon. Vision-based deep learning models can deliver both tactile and visual information in a sensor-free manner, while also being cost-effective to produce. Given the complexity of these models for devices with limited computational resources, research has focused on force estimation and catheter segmentation separately. However, there is a lack of a comprehensive architecture capable of simultaneously segmenting the catheter from two different angles and estimating the applied forces in 3D. To bridge this gap, this work proposes a novel, lightweight, multi-input, multi-output encoder-decoder-based architecture. It is designed to segment the catheter from two points of view and concurrently measure the applied forces in the x, y, and z directions. This network processes two simultaneous X-Ray images, intended to be fed by a biplane fluoroscopy system, showing a catheter's deflection from different angles. It uses two parallel sub-networks with shared parameters to output two segmentation maps corresponding to the inputs. Additionally, it leverages stereo vision to estimate the applied forces at the catheter's tip in 3D. The architecture features two input channels, two classification heads for segmentation, and a regression head for force estimation through a single end-to-end architecture. The output of all heads was assessed and compared with the literature, demonstrating state-of-the-art performance in both segmentation and force estimation. To the best of the authors' knowledge, this is the first time such a model has been proposed

A Versatile Pseudo-Rigid Body Modeling Method

A novel semi-analytical method is proposed to develop the pseudo-rigid-body~(PRB) model of robots made of highly flexible members (HFM), such as flexures and continuum robots, with no limit on the degrees of freedom of the PRB model. The proposed method has a simple formulation yet high precision. Furthermore, it can describe HFMs with variable curvature and stiffness along their length. The method offers a semi-analytical solution for the highly coupled nonlinear constrained optimization problem of PRB modeling and can be extended to variable-length robots comprised of HFM, such as catheter and concentric tube robots. We also show that this method can obtain a PRB model of uniformly stiff HFMs, with only three parameters. The versatility of the method is investigated in various applications of HFM in continuum robots. Simulations demonstrate substantial improvement in the precision of the PRB model in general and a reduction in the complexity of the formulation.

Image-based Intraluminal Contact Force Monitoring in Robotic Vascular Navigation

Embolization, stroke, ischaemic lesion, and perforation remain significant concerns in endovascular interventions. Sensing catheter interaction inside the artery is advantageous to minimize such complications and enhances navigation safety. Intraluminal information is currently limited due to the lack of intravascular contact sensing technologies. We present monitoring of the intraluminal catheter interaction with the arterial wall using an image-based estimation approach within vascular robotic navigation. The proposed image-based method employs continuous finite element simulation of the catheter motion using imaging data to estimate multi-point forces along catheter-vessel interaction. We implemented imaging algorithms to detect and track contacts and compute catheter pose measurements. The catheter model is constructed based on the nonlinear beam element and flexural rigidity distribution. During remote cannulation of aortic arteries, intraluminal monitoring achieved tracking local contact forces, building contour map of force on the arterial wall, and estimating structural stress of catheter. Shape estimation error was within 2% range. Results suggest that high-risk intraluminal forces may happen even in low insertion forces. The presented online monitoring tool delivers insight into the intraluminal behavior of catheters and is well-suited for intraoperative visual guidance of clinicians, robotic control vascular system, and optimizing interventional device design.