SwinMTL: A Shared Architecture for Simultaneous Depth Estimation and Semantic Segmentation from Monocular Camera Images

This research paper presents an innovative multi-task learning framework that allows concurrent depth estimation and semantic segmentation using a single camera. The proposed approach is based on a shared encoder-decoder architecture, which integrates various techniques to improve the accuracy of the depth estimation and semantic segmentation task without compromising computational efficiency. Additionally, the paper incorporates an adversarial training component, employing a Wasserstein GAN framework with a critic network, to refine model's predictions. The framework is thoroughly evaluated on two datasets - the outdoor Cityscapes dataset and the indoor NYU Depth V2 dataset - and it outperforms existing state-of-the-art methods in both segmentation and depth estimation tasks. We also conducted ablation studies to analyze the contributions of different components, including pre-training strategies, the inclusion of critics, the use of logarithmic depth scaling, and advanced image augmentations, to provide a better understanding of the proposed framework. The accompanying source code is accessible at \url{https://github.com/PardisTaghavi/SwinMTL}.

Real-Time Semantic Segmentation using Hyperspectral Images for Mapping Unstructured and Unknown Environments

Autonomous navigation in unstructured off-road environments is greatly improved by semantic scene understanding. Conventional image processing algorithms are difficult to implement and lack robustness due to a lack of structure and high variability across off-road environments. The use of neural networks and machine learning can overcome the previous challenges but they require large labeled data sets for training. In our work we propose the use of hyperspectral images for real-time pixel-wise semantic classification and segmentation, without the need of any prior training data. The resulting segmented image is processed to extract, filter, and approximate objects as polygons, using a polygon approximation algorithm. The resulting polygons are then used to generate a semantic map of the environment. Using our framework. we show the capability to add new semantic classes in run-time for classification. The proposed methodology is also shown to operate in real-time and produce outputs at a frequency of 1Hz, using high resolution hyperspectral images.

Design, Modeling, and Evaluation of Separable Tendon-Driven Robotic Manipulator with Long, Passive, Flexible Proximal Section

The purpose of this work was to tackle practical issues which arise when using a tendon-driven robotic manipulator with a long, passive, flexible proximal section in medical applications. A separable robot which overcomes difficulties in actuation and sterilization is introduced, in which the body containing the electronics is reusable and the remainder is disposable. A control input which resolves the redundancy in the kinematics and a physical interpretation of this redundancy are provided. The effect of a static change in the proximal section angle on bending angle error was explored under four testing conditions for a sinusoidal input. Bending angle error increased for increasing proximal section angle for all testing conditions with an average error reduction of 41.48% for retension, 4.28% for hysteresis, and 52.35% for re-tension + hysteresis compensation relative to the baseline case. Two major sources of error in tracking the bending angle were identified: time delay from hysteresis and DC offset from the proximal section angle. Examination of these error sources revealed that the simple hysteresis compensation was most effective for removing time delay and re-tension compensation for removing DC offset, which was the primary source of increasing error. The re-tension compensation was also tested for dynamic changes in the proximal section and reduced error in the final configuration of the tip by 89.14% relative to the baseline case.

Design and validation of zero-slack separable manipulator for Intracardiac Echocardiography

Clinicians require substantial training and experience to become comfortable with steering Intracardiac echocardiography (ICE) catheter to localize and measure the area of treatment to watch for complications while device catheters are deployed in another access. Thus, it is reasonable that a robotic-assist system to hold and actively manipulate the ICE catheter could ease the workload of the physician. Existing commercially-available robotic systems and research prototypes all use existing commercially available ICE catheters based on multiple tendon-sheath mechanism (TSM). To motorize the existing TSM-based ICE catheter, the actuators interface with the outer handle knobs to manipulate four internal tendons. However, in practice, the actuators are located at a sterile, safe place far away from the ICE handle. Thus, to interface with knobs, there exist multiple coupled gear structures between two, leading to a highly nonlinear behavior (e.g. various slack, elasticity) alongside hysteresis phenomena in TSM. Since ICE catheters are designed for single use, the expensive actuators need to be located in a safe place so as to be reusable. Moreover, these actuators should interface as directly as possible with the tendons for accurate tip controls. In this paper, we introduce a separable ICE catheter robot with four tendon actuation: one part reusable and another disposable. Moreover, we propose a practical model and calibration method for our proposed mechanism so that four tendons are actuated simultaneously allowing for precise tip control and mitigating issues with conventional devices such as dead-zone and hysteresis with simple linear compensation. We consider an open-loop controller since many available ICE catheters are used without position-tracking sensors at the tip due to costs and single use

A Computational Approach for Human-like Motion Generation in Upper Limb Exoskeletons Supporting Scapulohumeral Rhythms

This paper proposes a computational approach for generation of reference path for upper-limb exoskeletons considering the scapulohumeral rhythms of the shoulder. The proposed method can be used in upper-limb exoskeletons with 3 Degrees of Freedom (DoF) in shoulder and 1 DoF in elbow, which are capable of supporting shoulder girdle. The developed computational method is based on Central Nervous System (CNS) governing rules. Existing computational reference generation methods are based on the assumption of fixed shoulder center during motions. This assumption can be considered valid for reaching movements with limited range of motion (RoM). However, most upper limb motions such as Activities of Daily Living (ADL) include large scale inward and outward reaching motions, during which the center of shoulder joint moves significantly. The proposed method generates the reference motion based on a simple model of human arm and a transformation can be used to map the developed motion for other exoskeleton with different kinematics. Comparison of the model outputs with experimental results of healthy subjects performing ADL, show that the proposed model is able to reproduce human-like motions.

Cleverarm: A Novel Exoskeleton For Rehabilitation Of Upper Limb Impairments

CLEVERarm (Compact, Low-weight, Ergonomic, Virtual and Augmented Reality Enhanced Rehabilitation arm) is a novel exoskeleton with eight degrees of freedom supporting the motion of shoulder girdle, glenohumeral joint, elbow and wrist. Of the eight degrees of freedom of the exoskeleton, six are active and the two degrees of freedom supporting the motion of wrist are passive. This paper briefly outlines the design of CLEVERarm and its control architectures.

Challenges and Opportunities in Exoskeleton-based Rehabilitation

Robotic systems are increasingly used in rehabilitation to provide high intensity training for patients with motor impairment. The results of controlled trials involving human subjects confirm the effectiveness of robot-enhanced methods and prove them to be marginally superior over standard manual therapy in some cases. Although very promising, this line of research is still in its infancy and further studies are required to fully understand the potential benefits of using robotic devices such as exoskeletons. Exoskeletons have been widely studied due to their capability in providing more control over paretic limb as well as the complexities involved in their design and control. This paper briefly discusses the main challenges in development of rehabilitation exoskeletons and elaborates more on how some of these issues are addressed in the design of CLEVERarm, a recently developed upper limb rehabilitation exoskeleton. The paper is concluded with several remarks on the current challenges in wide-spread use of exoskeletons in medical facilities, and a vision for the future of these technologies in rehabilitation medicine.

A Systematic Approach For Kinematic Design Of Upper Limb Rehabilitation Exoskeletons

Kinematic structure of an exoskeleton is the most fundamental block of its design and is determinant of many functional capabilities of it. Although numerous upper limb rehabilitation devices have been designed in the recent years, there is not a framework that can systematically guide the kinematic design procedure. Additionally, diversity of currently available devices and the many minute details incorporated to address certain design requirements hinders pinpointing the core kinematics of the available devices to compare them against each other. This makes the review of literature for identifying drawbacks of the state of the art systems a challenging and puzzling task. In fact, lack of a unifying framework makes designing rehabilitation devices an intuitive process and prone to biases from currently available designs. This research work proposes a systematic approach for kinematic design of upper limb rehabilitation exoskeletons based on conceptual design techniques. Having defined a solution neutral problem statement based on the characteristics of an ideal device, the main functionality of the system is divided into smaller functional units via the Functional Decomposition Method. Various directions for concept generation are explored and finally, it has been shown that a vast majority of the current exoskeleton designs fit within the proposed design framework and the defined functionalities.