Dynamic Bayesian Approach for decision-making in Ego-Things

This paper presents a novel approach to detect abnormalities in dynamic systems based on multisensory data and feature selection. The proposed method produces multiple inference models by considering several features of the observed data. This work facilitates the obtainment of the most precise features for predicting future instances and detecting abnormalities. Growing neural gas (GNG) is employed for clustering multisensory data into a set of nodes that provide a semantic interpretation of data and define local linear models for prediction purposes. Our method uses a Markov Jump particle filter (MJPF) for state estimation and abnormality detection. The proposed method can be used for selecting the optimal set features to be shared in networking operations such that state prediction, decision-making, and abnormality detection processes are favored. This work is evaluated by using a real dataset consisting of a moving vehicle performing some tasks in a controlled environment.

Continual Learning of Predictive Models in Video Sequences via Variational Autoencoders

This paper proposes a method for performing continual learning of predictive models that facilitate the inference of future frames in video sequences. For a first given experience, an initial Variational Autoencoder, together with a set of fully connected neural networks are utilized to respectively learn the appearance of video frames and their dynamics at the latent space level. By employing an adapted Markov Jump Particle Filter, the proposed method recognizes new situations and integrates them as predictive models avoiding catastrophic forgetting of previously learned tasks. For evaluating the proposed method, this article uses video sequences from a vehicle that performs different tasks in a controlled environment.

Anomaly Detection in Video Data Based on Probabilistic Latent Space Models

This paper proposes a method for detecting anomalies in video data. A Variational Autoencoder (VAE) is used for reducing the dimensionality of video frames, generating latent space information that is comparable to low-dimensional sensory data (e.g., positioning, steering angle), making feasible the development of a consistent multi-modal architecture for autonomous vehicles. An Adapted Markov Jump Particle Filter defined by discrete and continuous inference levels is employed to predict the following frames and detecting anomalies in new video sequences. Our method is evaluated on different video scenarios where a semi-autonomous vehicle performs a set of tasks in a closed environment.

Static force field representation of environments based on agents nonlinear motions

This paper presents a methodology that aims at the incremental representation of areas inside environments in terms of attractive forces. It is proposed a parametric representation of velocity fields ruling the dynamics of moving agents. It is assumed that attractive spots in the environment are responsible for modifying the motion of agents. A switching model is used to describe near and far velocity fields, which in turn are used to learn attractive characteristics of environments. The effect of such areas is considered radial over all the scene. Based on the estimation of attractive areas, a map that describes their effects in terms of their localizations, ranges of action, and intensities is derived in an online way. Information of static attractive areas is added dynamically into a set of filters that describes possible interactions between moving agents and an environment. The proposed approach is first evaluated on synthetic data; posteriorly, the method is applied on real trajectories coming from moving pedestrians in an indoor environment.

Incremental learning of environment interactive structures from trajectories of individuals

This work proposes a novel method for estimating the influence that unknown static objects might have over mobile agents. Since the motion of agents can be affected by the presence of fixed objects, it is possible use the information about trajectories deviations to infer the presence of obstacles and estimate the forces involved in a scene. Artificial neural networks are used to estimate a non-parametric function related to the velocity field influencing moving agents. The proposed method is able to incrementally learn the velocity fields due to external static objects within the monitored environment. It determines whether an object has a repulsive or an attractive influence and provides an estimation of its position and size. As stationarity is assumed, i.e., time-invariance of force fields, learned observation models can be used as prior knowledge for estimating hierarchically the properties of new objects in a scene.

Online Pedestrian Group Walking Event Detection Using Spectral Analysis of Motion Similarity Graph

A method for online identification of group of moving objects in the video is proposed in this paper. This method at each frame identifies group of tracked objects with similar local instantaneous motion pattern using spectral clustering on motion similarity graph. Then, the output of the algorithm is used to detect the event of more than two object moving together as required by PETS2015 challenge. The performance of the algorithm is evaluated on the PETS2015 dataset.

Hierarchy of GANs for learning embodied self-awareness model

In recent years several architectures have been proposed to learn embodied agents complex self-awareness models. In this paper, dynamic incremental self-awareness (SA) models are proposed that allow experiences done by an agent to be modeled in a hierarchical fashion, starting from more simple situations to more structured ones. Each situation is learned from subsets of private agent perception data as a model capable to predict normal behaviors and detect abnormalities. Hierarchical SA models have been already proposed using low dimensional sensorial inputs. In this work, a hierarchical model is introduced by means of a cross-modal Generative Adversarial Networks (GANs) processing high dimensional visual data. Different levels of the GANs are detected in a self-supervised manner using GANs discriminators decision boundaries. Real experiments on semi-autonomous ground vehicles are presented.

Learning Multi-Modal Self-Awareness Models for Autonomous Vehicles from Human Driving

This paper presents a novel approach for learning self-awareness models for autonomous vehicles. The proposed technique is based on the availability of synchronized multi-sensor dynamic data related to different maneuvering tasks performed by a human operator. It is shown that different machine learning approaches can be used to first learn single modality models using coupled Dynamic Bayesian Networks; such models are then correlated at event level to discover contextual multi-modal concepts. In the presented case, visual perception and localization are used as modalities. Cross-correlations among modalities in time is discovered from data and are described as probabilistic links connecting shared and private multi-modal DBNs at the event (discrete) level. Results are presented on experiments performed on an autonomous vehicle, highlighting potentiality of the proposed approach to allow anomaly detection and autonomous decision making based on learned self-awareness models.

A Multi-perspective Approach To Anomaly Detection For Self-aware Embodied Agents

This paper focuses on multi-sensor anomaly detection for moving cognitive agents using both external and private first-person visual observations. Both observation types are used to characterize agents' motion in a given environment. The proposed method generates locally uniform motion models by dividing a Gaussian process that approximates agents' displacements on the scene and provides a Shared Level (SL) self-awareness based on Environment Centered (EC) models. Such models are then used to train in a semi-unsupervised way a set of Generative Adversarial Networks (GANs) that produce an estimation of external and internal parameters of moving agents. Obtained results exemplify the feasibility of using multi-perspective data for predicting and analyzing trajectory information.