Hyperspectral Recovery from RGB Images
A $50,000 Camera you Already Own
Conventional cameras capture images using only three frequency bands (red, blue, green), while the full visual spectrum is a much richer representation that facilitates a wide range of additional and important applications. A new technology allows conventional cameras to increase their spectral resolution, capturing information over a wide range of wavelengths without the need for specialized equipment or controlled lighting.
Hyperspectral (HS) imaging systems are capable of collecting the complete spectral signature reflected from each point in a given scene, producing a much more spectrally detailed image than that provided by RGB cameras. Although these systems are widely used in industrial and scientific settings, they have yet to find a place in the consumer market due to their cost, size, and slow acquisition process (often requiring close to one minute to acquire a single image). HC-Vision’s story began at the Ben-Gurion University Interdisciplinary Computational Vision Laboratory where the company’s co-founders, Prof. Ohad Ben-Shahar and Boaz Arad, were studying the properties of natural hyperspectral images. To this end they began collecting what is now the largest, most detailed natural hyperspectral image collection published to date. Analysis of this database revealed that, in the case of natural images people experience in typical indoor and outdoor environments, underlying hyperspectral information can be accurately recovered from its RGB projection.
Hyperspectral images from RGB
Unlike the three channel images that RGB cameras (and human eyes) produce, hyperspectral images contain dozens or hundreds (and from an abstract theoretical point of view, infinite number) of wavelength bands. But even simpler hyperspectral cameras contain at least 31 channels, a number representing the division of the visible spectrum to spectral bands of 10nm, and so even in this case, the 31-to-3 dimensionality reduction that occurs while projecting hyperspectral information to RGB appears rather severe,. And yet, HC-Vision’s methodology is already able to recover the former from the latter with 90 percent to 95 percent accuracy over a wide variety of scenes. In addition to producing state-of-the-art results at the time of publication, this approach produced comparable results to previous methodologies which relied on hybrid HS/RGB input. This methodology often surpassed the performance of the latter (in both accuracy and computation time) despite a significant information disadvantage.