A multitask transformer to sign language translation using motion gesture primitives

The absence of effective communication the deaf population represents the main social gap in this community. Furthermore, the sign language, main deaf communication tool, is unlettered, i.e., there is no formal written representation. In consequence, main challenge today is the automatic translation among spatiotemporal sign representation and natural text language. Recent approaches are based on encoder-decoder architectures, where the most relevant strategies integrate attention modules to enhance non-linear correspondences, besides, many of these approximations require complex training and architectural schemes to achieve reasonable predictions, because of the absence of intermediate text projections. However, they are still limited by the redundant background information of the video sequences. This work introduces a multitask transformer architecture that includes a gloss learning representation to achieve a more suitable translation. The proposed approach also includes a dense motion representation that enhances gestures and includes kinematic information, a key component in sign language. From this representation it is possible to avoid background information and exploit the geometry of the signs, in addition, it includes spatiotemporal representations that facilitate the alignment between gestures and glosses as an intermediate textual representation. The proposed approach outperforms the state-of-the-art evaluated on the CoL-SLTD dataset, achieving a BLEU-4 of 72,64% in split 1, and a BLEU-4 of 14,64% in split 2. Additionally, the strategy was validated on the RWTH-PHOENIX-Weather 2014 T dataset, achieving a competitive BLEU-4 of 11,58%.

A digital eye-fixation biomarker using a deep anomaly scheme to classify Parkisonian patterns

Oculomotor alterations constitute a promising biomarker to detect and characterize Parkinson's disease (PD), even in prodromal stages. Currently, only global and simplified eye movement trajectories are employed to approximate the complex and hidden kinematic relationships of the oculomotor function. Recent advances on machine learning and video analysis have encouraged novel characterizations of eye movement patterns to quantify PD. These schemes enable the identification of spatiotemporal segments primarily associated with PD. However, they rely on discriminative models that require large training datasets and depend on balanced class distributions. This work introduces a novel video analysis scheme to quantify Parkinsonian eye fixation patterns with an anomaly detection framework. Contrary to classical deep discriminative schemes that learn differences among labeled classes, the proposed approach is focused on one-class learning, avoiding the necessity of a significant amount of data. The proposed approach focuses only on Parkinson's representation, considering any other class sample as an anomaly of the distribution. This approach was evaluated for an ocular fixation task, in a total of 13 control subjects and 13 patients on different stages of the disease. The proposed digital biomarker achieved an average sensitivity and specificity of 0.97 and 0.63, respectively, yielding an AUC-ROC of 0.95. A statistical test shows significant differences (p < 0.05) among predicted classes, evidencing a discrimination between patients and control subjects.

APIS: A paired CT-MRI dataset for ischemic stroke segmentation challenge

Stroke is the second leading cause of mortality worldwide. Immediate attention and diagnosis play a crucial role regarding patient prognosis. The key to diagnosis consists in localizing and delineating brain lesions. Standard stroke examination protocols include the initial evaluation from a non-contrast CT scan to discriminate between hemorrhage and ischemia. However, non-contrast CTs may lack sensitivity in detecting subtle ischemic changes in the acute phase. As a result, complementary diffusion-weighted MRI studies are captured to provide valuable insights, allowing to recover and quantify stroke lesions. This work introduced APIS, the first paired public dataset with NCCT and ADC studies of acute ischemic stroke patients. APIS was presented as a challenge at the 20th IEEE International Symposium on Biomedical Imaging 2023, where researchers were invited to propose new computational strategies that leverage paired data and deal with lesion segmentation over CT sequences. Despite all the teams employing specialized deep learning tools, the results suggest that the ischemic stroke segmentation task from NCCT remains challenging. The annotated dataset remains accessible to the public upon registration, inviting the scientific community to deal with stroke characterization from NCCT but guided with paired DWI information.