An Efficient Model-Agnostic Approach for Uncertainty Estimation in Data-Restricted Pedometric Applications

This paper introduces a model-agnostic approach designed to enhance uncertainty estimation in the predictive modeling of soil properties, a crucial factor for advancing pedometrics and the practice of digital soil mapping. For addressing the typical challenge of data scarcity in soil studies, we present an improved technique for uncertainty estimation. This method is based on the transformation of regression tasks into classification problems, which not only allows for the production of reliable uncertainty estimates but also enables the application of established machine learning algorithms with competitive performance that have not yet been utilized in pedometrics. Empirical results from datasets collected from two German agricultural fields showcase the practical application of the proposed methodology. Our results and findings suggest that the proposed approach has the potential to provide better uncertainty estimation than the models commonly used in pedometrics.

Improving Nonlinear Projection Heads using Pretrained Autoencoder Embeddings

This empirical study aims at improving the effectiveness of the standard 2-layer MLP projection head $g(\cdot)$ featured in the SimCLR framework through the use of pretrained autoencoder embeddings. Given a contrastive learning task with a largely unlabeled image classification dataset, we first train a shallow autoencoder architecture and extract its compressed representations contained in the encoder's embedding layer. After freezing the weights within this pretrained layer, we use it as a drop-in replacement for the input layer of SimCLR's default projector. Additionally, we also apply further architectural changes to the projector by decreasing its width and changing its activation function. The different projection heads are then used to contrastively train and evaluate a feature extractor $f(\cdot)$ following the SimCLR protocol, while also examining the performance impact of Z-score normalized datasets. Our experiments indicate that using a pretrained autoencoder embedding in the projector can not only increase classification accuracy by up to 2.9% or 1.7% on average but can also significantly decrease the dimensionality of the projection space. Our results also suggest, that using the sigmoid and tanh activation functions within the projector can outperform ReLU in terms of peak and average classification accuracy. When applying our presented projectors, then not applying Z-score normalization to datasets often increases peak performance. In contrast, the default projection head can benefit more from normalization. All experiments involving our pretrained projectors are conducted with frozen embeddings, since our test results indicate an advantage compared to using their non-frozen counterparts.

Multivariate Time Series Regression with Graph Neural Networks

Machine learning, with its advances in Deep Learning has shown great potential in analysing time series in the past. However, in many scenarios, additional information is available that can potentially improve predictions, by incorporating it into the learning methods. This is crucial for data that arises from e.g., sensor networks that contain information about sensor locations. Then, such spatial information can be exploited by modeling it via graph structures, along with the sequential (time) information. Recent advances in adapting Deep Learning to graphs have shown promising potential in various graph-related tasks. However, these methods have not been adapted for time series related tasks to a great extent. Specifically, most attempts have essentially consolidated around Spatial-Temporal Graph Neural Networks for time series forecasting with small sequence lengths. Generally, these architectures are not suited for regression or classification tasks that contain large sequences of data. Therefore, in this work, we propose an architecture capable of processing these long sequences in a multivariate time series regression task, using the benefits of Graph Neural Networks to improve predictions. Our model is tested on two seismic datasets that contain earthquake waveforms, where the goal is to predict intensity measurements of ground shaking at a set of stations. Our findings demonstrate promising results of our approach, which are discussed in depth with an additional ablation study.

Local Exceptionality Detection in Time Series Using Subgroup Discovery

In this paper, we present a novel approach for local exceptionality detection on time series data. This method provides the ability to discover interpretable patterns in the data, which can be used to understand and predict the progression of a time series. This being an exploratory approach, the results can be used to generate hypotheses about the relationships between the variables describing a specific process and its dynamics. We detail our approach in a concrete instantiation and exemplary implementation, specifically in the field of teamwork research. Using a real-world dataset of team interactions we include results from an example data analytics application of our proposed approach, showcase novel analysis options, and discuss possible implications of the results from the perspective of teamwork research.

A Computational Framework for Modeling Complex Sensor Network Data Using Graph Signal Processing and Graph Neural Networks in Structural Health Monitoring

Complex networks lend themselves to the modeling of multidimensional data, such as relational and/or temporal data. In particular, when such complex data and their inherent relationships need to be formalized, complex network modeling and its resulting graph representations enable a wide range of powerful options. In this paper, we target this - connected to specific machine learning approaches on graphs for structural health monitoring on an analysis and predictive (maintenance) perspective. Specifically, we present a framework based on Complex Network Modeling, integrating Graph Signal Processing (GSP) and Graph Neural Network (GNN) approaches. We demonstrate this framework in our targeted application domain of Structural Health Monitoring (SHM). In particular, we focus on a prominent real-world structural health monitoring use case, i.e., modeling and analyzing sensor data (strain, vibration) of a large bridge in the Netherlands. In our experiments, we show that GSP enables the identification of the most important sensors, for which we investigate a set of search and optimization approaches. Furthermore, GSP enables the detection of specific graph signal patterns (mode shapes), capturing physical functional properties of the sensors in the applied complex network. In addition, we show the efficacy of applying GNNs for strain prediction on this kind of data.

Feature-Set-Engineering for Detecting Freezing of Gait in Parkinson's Disease using Deep Recurrent Neural Networks

Freezing of gait (FoG) is a common gait disability in Parkinson's disease, that usually appears in its advanced stage. Freeze episodes are associated with falls, injuries, and psychological consequences, negatively affecting the patients' quality of life. For detecting FoG episodes automatically, a highly accurate detection method is necessary. This paper presents an approach for detecting FoG episodes utilizing a deep recurrent neural network (RNN) on 3D-accelerometer measurements. We investigate suitable features and feature combinations extracted from the sensors' time series data. Specifically, for detecting FoG episodes, we apply a deep RNN with Long Short-Term Memory cells. In our experiments, we perform both user dependent and user independent experiments, to detect freeze episodes. Our experimental results show that the frequency domain features extracted from the trunk sensor are the most informative feature group in the subject independent method, achieving an average AUC score of 93%, Specificity of 90% and Sensitivity of 81%. Moreover, frequency and statistical features of all the sensors are identified as the best single input for the subject dependent method, achieving an average AUC score of 97%, Specificity of 96% and Sensitivity of 87%. Overall, in a comparison to state-of-the-art approaches from literature as baseline methods, our proposed approach outperforms these significantly.

Towards Generating Explanations for ASP-Based Link Analysis using Declarative Program Transformations

The explication and the generation of explanations are prominent topics in artificial intelligence and data science, in order to make methods and systems more transparent and understandable for humans. This paper investigates the problem of link analysis, specifically link prediction and anomalous link discovery in social networks using the declarative method of Answer set programming (ASP). Applying ASP for link prediction provides a powerful declarative approach, e.g., for incorporating domain knowledge for explicative prediction. In this context, we propose a novel method for generating explanations - as offline justifications - using declarative program transformations. The method itself is purely based on syntactic transformations of declarative programs, e.g., in an ASP formalism, using rule instrumentation. We demonstrate the efficacy of the proposed approach, exemplifying it in an application on link analysis in social networks, also including domain knowledge.

Adaptive kNN using Expected Accuracy for Classification of Geo-Spatial Data

The k-Nearest Neighbor (kNN) classification approach is conceptually simple - yet widely applied since it often performs well in practical applications. However, using a global constant k does not always provide an optimal solution, e.g., for datasets with an irregular density distribution of data points. This paper proposes an adaptive kNN classifier where k is chosen dynamically for each instance (point) to be classified, such that the expected accuracy of classification is maximized. We define the expected accuracy as the accuracy of a set of structurally similar observations. An arbitrary similarity function can be used to find these observations. We introduce and evaluate different similarity functions. For the evaluation, we use five different classification tasks based on geo-spatial data. Each classification task consists of (tens of) thousands of items. We demonstrate, that the presented expected accuracy measures can be a good estimator for kNN performance, and the proposed adaptive kNN classifier outperforms common kNN and previously introduced adaptive kNN algorithms. Also, we show that the range of considered k can be significantly reduced to speed up the algorithm without negative influence on classification accuracy.