Probabilistic Semantic Data Association for Collaborative Human-Robot Sensing

In collaborative human-robot semantic sensing problems, e.g. for scientific exploration, robots could potentially overtrust information given by a human partner, resulting in suboptimal state estimation and poor team performance. When humans cannot be treated as oracles, robots need to update state beliefs to correctly account for possible discrepancies between human semantic observations and the actual world states which lead to those observations. This work develops strategies for rigorous online calculation of probabilistic semantic data association (PSDA) probabilities for semantic likelihoods in general settings, unlike previous work which developed naive or heuristic approximations for specific settings. The new PSDA method is incorporated into a hybrid Bayesian data fusion scheme which uses Gaussian mixture priors for object states and softmax functions for semantic human sensor observation likelihoods, and is demonstrated in Monte Carlo simulations of collaborative multi-object search missions featuring a range of relevant human sensing characteristics (e.g. false detection rate). It is shown that PSDA leads to robust estimation of observation association probabilities under a wide range of conditions whenever semantic human sensor data contain significant target reference ambiguities for autonomous object search and localization.