A Synergistic Framework for Learning Shape Estimation and Shape-Aware Whole-Body Control Policy for Continuum Robots

In this paper, we present a novel synergistic framework for learning shape estimation and a shape-aware whole-body control policy for tendon-driven continuum robots. Our approach leverages the interaction between two Augmented Neural Ordinary Differential Equations (ANODEs) -- the Shape-NODE and Control-NODE -- to achieve continuous shape estimation and shape-aware control. The Shape-NODE integrates prior knowledge from Cosserat rod theory, allowing it to adapt and account for model mismatches, while the Control-NODE uses this shape information to optimize a whole-body control policy, trained in a Model Predictive Control (MPC) fashion. This unified framework effectively overcomes limitations of existing data-driven methods, such as poor shape awareness and challenges in capturing complex nonlinear dynamics. Extensive evaluations in both simulation and real-world environments demonstrate the framework's robust performance in shape estimation, trajectory tracking, and obstacle avoidance. The proposed method consistently outperforms state-of-the-art end-to-end, Neural-ODE, and Recurrent Neural Network (RNN) models, particularly in terms of tracking accuracy and generalization capabilities.

Towards the Feasibility Analysis and Additive Manufacturing of a Novel Flexible Pedicle Screw for Spinal Fixation Procedures

In this paper, we explore the feasibility of developing a novel flexible pedicle screw (FPS) for enhanced spinal fixation of osteoporotic vertebrae. Vital for spinal fracture treatment, pedicle screws have been around since the early 20th century and have undergone multiple iterations to enhance internal spinal fixation. However, spinal fixation treatments tend to be problematic for osteoporotic patients due to multiple inopportune variables. The inherent rigid nature of the pedicle screw, along with the forced linear trajectory of the screw path, frequently leads to the placement of these screws in highly osteoporotic regions of the bone. This results in eventual screw slippage and causing neurological and respiratory problems for the patient. To address this problem, we focus on developing a novel FPS that is structurally capable of safely bending to fit curved trajectories drilled by a steerable drilling robot and bypass highly osteoporotic regions of the vertebral body. Afterwards, we simulate its morphability capabilities using finite element analysis (FEA). We then additively manufacture the FPS using stainless steel (SS) 316L alloy through direct metal laser sintering (DMLS). Finally, the fabricated FPS is experimentally evaluated for its bending performance and compared with the FEA results for verification. Results demonstrate the feasibility of additive manufacturing of FPS using DMLS approach and agreement of the developed FEA with the experiments.

Robot-Enabled Machine Learning-Based Diagnosis of Gastric Cancer Polyps Using Partial Surface Tactile Imaging

In this paper, to collectively address the existing limitations on endoscopic diagnosis of Advanced Gastric Cancer (AGC) Tumors, for the first time, we propose (i) utilization and evaluation of our recently developed Vision-based Tactile Sensor (VTS), and (ii) a complementary Machine Learning (ML) algorithm for classifying tumors using their textural features. Leveraging a seven DoF robotic manipulator and unique custom-designed and additively-manufactured realistic AGC tumor phantoms, we demonstrated the advantages of automated data collection using the VTS addressing the problem of data scarcity and biases encountered in traditional ML-based approaches. Our synthetic-data-trained ML model was successfully evaluated and compared with traditional ML models utilizing various statistical metrics even under mixed morphological characteristics and partial sensor contact.

Towards an Autonomous Minimally Invasive Spinal Fixation Surgery Using a Concentric Tube Steerable Drilling Robot

Towards performing a realistic autonomous minimally invasive spinal fixation procedure, in this paper, we introduce a unique robotic drilling system utilizing a concentric tube steerable drilling robot (CT-SDR) integrated with a seven degree-of-freedom robotic manipulator. The CT-SDR in integration with the robotic arm enables creating precise J-shape trajectories enabling access to the areas within the vertebral body that currently are not accessible utilizing existing rigid instruments. To ensure safety and accuracy of the autonomous drilling procedure, we also performed required calibration procedures. The performance of the proposed robotic system and the calibration steps were thoroughly evaluated by performing various drilling experiments on simulated Sawbone samples.

Single-Fiber Optical Frequency Domain Reflectometry Shape Sensing of Continuum Manipulators with Planar Bending

To address the challenges associated with shape sensing of continuum manipulators (CMs) using Fiber Bragg Grating (FBG) optical fibers, we feature a unique shape sensing assembly utilizing solely a single Optical Frequency Domain Reflectometry (OFDR) fiber attached to a flat nitinol wire (NiTi). Integrating this easy-to-manufacture unique sensor with a long and soft CM with 170 mm length, we performed different experiments to evaluate its shape reconstruction ability. Results demonstrate phenomenal shape reconstruction accuracy for both C-shape (< 2 mm tip error, < 1.2 mm shape error) and J-shape (< 3.4 mm tip error, < 2.3 mm shape error) experiments.

A Patient-Specific Framework for Autonomous Spinal Fixation via a Steerable Drilling Robot

In this paper, with the goal of enhancing the minimally invasive spinal fixation procedure in osteoporotic patients, we propose a first-of-its-kind image-guided robotic framework for performing an autonomous and patient-specific procedure using a unique concentric tube steerable drilling robot (CT-SDR). Particularly, leveraging a CT-SDR, we introduce the concept of J-shape drilling based on a pre-operative trajectory planned in CT scan of a patient followed by appropriate calibration, registration, and navigation steps to safely execute this trajectory in real-time using our unique robotic setup. To thoroughly evaluate the performance of our framework, we performed several experiments on two different vertebral phantoms designed based on CT scan of real patients.

Spatial Spinal Fixation: A Transformative Approach Using a Unique Robot-Assisted Steerable Drilling System and Flexible Pedicle Screw

Spinal fixation procedures are currently limited by the rigidity of the existing instruments and pedicle screws leading to fixation failures and rigid pedicle screw pull out. Leveraging our recently developed Concentric Tube Steerable Drilling Robot (CT-SDR) in integration with a robotic manipulator, to address the aforementioned issue, here we introduce the transformative concept of Spatial Spinal Fixation (SSF) using a unique Flexible Pedicle Screw (FPS). The proposed SSF procedure enables planar and out-of-plane placement of the FPS throughout the full volume of the vertebral body. In other words, not only does our fixation system provide the option of drilling in-plane and out-of-plane trajectories, it also enables implanting the FPS inside linear (represented by an I-shape) and/or non-linear (represented by J-shape) trajectories. To thoroughly evaluate the functionality of our proposed robotic system and the SSF procedure, we have performed various experiments by drilling different I-J and J-J drilling trajectory pairs into our custom-designed L3 vertebral phantoms and analyzed the accuracy of the procedure using various metrics.

Towards Biomechanical Evaluation of a Transformative Additively Manufactured Flexible Pedicle Screw for Robotic Spinal Fixation

Vital for spinal fracture treatment, pedicle screw fixation is the gold standard for spinal fixation procedures. Nevertheless, due to the screw pullout and loosening issues, this surgery often fails to be effective for patients suffering from osteoporosis (i.e., having low bone mineral density). These failures can be attributed to the rigidity of existing drilling instruments and pedicle screws forcing clinicians to place these implants into the osteoporotic regions of the vertebral body. To address this critical issue, we have developed a steerable drilling robotic system and evaluated its performance in drilling various J- and U-shape trajectories. Complementary to this robotic system, in this paper, we propose design, additive manufacturing, and biomechanical evaluation of a transformative flexible pedicle screw (FPS) that can be placed in pre-drilled straight and curved trajectories. To evaluate the performance of the proposed flexible implant, we designed and fabricated two different types of FPSs using the direct metal laser sintering (DMLS) process. Utilizing our unique experimental setup and ASTM standards, we then performed various pullout experiments on these FPSs to evaluate and analyze their biomechanical performance implanted in straight trajectories.

Towards Design and Development of an ArUco Markers-Based Quantitative Surface Tactile Sensor

In this paper, with the goal of quantifying the qualitative image outputs of a Vision-based Tactile Sensor (VTS), we present the design, fabrication, and characterization of a novel Quantitative Surface Tactile Sensor (called QS-TS). QS-TS directly estimates the sensor's gel layer deformation in real-time enabling safe and autonomous tactile manipulation and servoing of delicate objects using robotic manipulators. The core of the proposed sensor is the utilization of miniature 1.5 mm x 1.5 mm synthetic square markers with inner binary patterns and a broad black border, called ArUco Markers. Each ArUco marker can provide real-time camera pose estimation that, in our design, is used as a quantitative measure for obtaining deformation of the QS-TS gel layer. Moreover, thanks to the use of ArUco markers, we propose a unique fabrication procedure that mitigates various challenges associated with the fabrication of the existing marker-based VTSs and offers an intuitive and less-arduous method for the construction of the VTS. Remarkably, the proposed fabrication facilitates the integration and adherence of markers with the gel layer to robustly and reliably obtain a quantitative measure of deformation in real-time regardless of the orientation of ArUco Markers. The performance and efficacy of the proposed QS-TS in estimating the deformation of the sensor's gel layer were experimentally evaluated and verified. Results demonstrate the phenomenal performance of the QS-TS in estimating the deformation of the gel layer with a relative error of <5%.

Design and Development of a Novel Soft and Inflatable Tactile Sensing Balloon for Early Diagnosis of Colorectal Cancer Polyps

In this paper, with the goal of addressing the high early-detection miss rate of colorectal cancer (CRC) polyps during a colonoscopy procedure, we propose the design and fabrication of a unique inflatable vision-based tactile sensing balloon (VTSB). The proposed soft VTSB can readily be integrated with the existing colonoscopes and provide a radiation-free, safe, and high-resolution textural mapping and morphology characterization of CRC polyps. The performance of the proposed VTSB has been thoroughly characterized and evaluated on four different types of additively manufactured CRC polyp phantoms with three different stiffness levels. Additionally, we integrated the VTSB with a colonoscope and successfully performed a simulated colonoscopic procedure inside a tube with a few CRC polyp phantoms attached to its internal surface.