Spectral video construction from RGB video: Application to Image Guided Neurosurgery

Spectral imaging has received enormous interest in the field of medical imaging modalities. It provides a powerful tool for the analysis of different organs and non-invasive tissues. Therefore, significant amount of research has been conducted to explore the possibility of using spectral imaging in biomedical applications. To observe spectral image information in real time during surgery and monitor the temporal changes in the organs and tissues is a demanding task. Available spectral imaging devices are not sufficient to accomplish this task with an acceptable spatial and spectral resolution. A solution to this problem is to estimate the spectral video from RGB video and perform visualization with the most prominent spectral bands. In this research, we propose a framework to generate neurosurgery spectral video from RGB video. A spectral estimation technique is applied on each RGB video frames. The RGB video is captured using a digital camera connected with an operational microscope dedicated to neurosurgery. A database of neurosurgery spectral images is used to collect training data and evaluate the estimation accuracy. A searching technique is used to identify the best training set. Five different spectrum estimation techniques are experimented to indentify the best method. Although this framework is established for neurosurgery spectral video generation, however, the methodology outlined here would also be applicable to other similar research.