Monte Carlo Path Tracing and Statistical Event Detection for Event Camera Simulation

This paper presents a novel event camera simulation system fully based on physically based Monte Carlo path tracing with adaptive path sampling. The adaptive sampling performed in the proposed method is based on a statistical technique, hypothesis testing for the hypothesis whether the difference of logarithmic luminances at two distant periods is significantly larger than a predefined event threshold. To this end, our rendering system collects logarithmic luminances rather than raw luminance in contrast to the conventional rendering system imitating conventional RGB cameras. Then, based on the central limit theorem, we reasonably assume that the distribution of the population mean of logarithmic luminance can be modeled as a normal distribution, allowing us to model the distribution of the difference of logarithmic luminance as a normal distribution. Then, using Student's t-test, we can test the hypothesis and determine whether to discard the null hypothesis for event non-occurrence. When we sample a sufficiently large number of path samples to satisfy the central limit theorem and obtain a clean set of events, our method achieves significant speed up compared to a simple approach of sampling paths uniformly at every pixel. To our knowledge, we are the first to simulate the behavior of event cameras in a physically accurate manner using an adaptive sampling technique in Monte Carlo path tracing, and we believe this study will contribute to the development of computer vision applications using event cameras.

Event-based Camera Simulation using Monte Carlo Path Tracing with Adaptive Denoising

This paper proposes an algorithm for obtaining an event-based video from a noisy input video given by physics-based Monte Carlo path tracing of synthetic 3D scenes. Since the dynamic vision sensor (DVS) detects temporal brightness changes as events, the problem of efficiently rendering event-based video boils down to detecting the changes from noisy brightness values. To this end, we extend a denoising method based on a weighted local regression (WLR) to detect the brightness changes rather than applying denoising to each video frame. Specifically, we regress a WLR model only on frames where an event is detected, which significantly reduces the computational cost of the regression. We show that our efficient method is robust to noisy video frames obtained from a few path-traced samples and performs comparably to or even better than an approach that denoises every frame.