Road to Smart Perception
HSI Camera Fuses Snapshot and Linescan Benefits
The hyperspectral imaging (HSI) camera snapscan combines a good signal to noise ratio with spatial and spectral resolution advantages of linescan hyperspectral imaging technology and the ability to acquire data-sets as easily as with a snapshot camera.
Figure 1 | Full view of the datacubes (left). Zoom over a region-of-interest (middle). Blue and red spectra corresponding to the blue and red selection on the left-hand figures (right). Displayed spectra are an average over a 5×5 region with the corresponding standard deviation displayed in transparency. Datasets are a hand (a-c), a blood smear (d-f) and pills (g-l). (Bild: Imec vzw)
Intelligent robotics are bound to transform society – and smart perception is key to realizing that. By 2040, we’ll have micro-sized cameras looking at every angle, depth, mode or change in a scene, and reporting back with all (and only) relevant info. However, for smart perception to become a reality, new technologies are needed – such as mass-produced, single-chip, high-quality hyperspectral vision technology and hyperspectral cameras offering high resolution and fast image capturing and processing capabilities. Such innovations will create new opportunities for hyperspectral imaging in domains such as life sciences, medical applications and diagnostics, which are not yet served by today’s solutions. Tomorrow’s smart perceptive systems will rely on smart cameras that integrate various viewing and sensing modalities into a single compact and cheap system. Cameras, for instance, that recognize and scan a surface via radar, look underneath with an x-ray component, analyze material through a hyperspectral view, and that have the intelligence to acquire and merge various data streams into the most relevant view for the user. Hyperspectral cameras will be at the basis of this. However, today’s hyperspectral cameras are expensive and not user friendly, thus limiting their use. To overcome that, a hyperspectral imaging solution has been developed that uses optical filters on top of CMOS image sensors, resulting in a compact, cost-efficient and fast hyperspectral camera.
The best of two worlds
Until now, no hyperspectral technique was able to capture entire scenes at once with fair levels of spatial and spectral resolution. To fill that gap, Snapscan was developed: a novel, patent-pending hyperspectral camera combining the best of both linescan and snapshot imaging. Linescan hyperspectral cameras scan scenes line per line. They are ideal for applications requiring high spatial and spectral resolution – with scenes passing by the camera, such as aerial observation or food sorting. However, for inspection cameras in robots, free-flying drones, movement-activated security cameras or real-time imaging during surgery, you need an entire 3D multispectral image being sensed at once. This requirement is dealt with by snapshot hyperspectral imaging solutions, which use filters organized in a mosaic or tiled configuration, and where each tile is responsible for sensing one narrow band of the spectrum for the entire scene. However, such cameras lack linescan’s image resolution. Snapscan combines the best of both worlds, capturing high-resolution, full hyperspectral images in seconds, making it suitable for use in domains like digital microscopy for pathology and cytogenetics, medical imaging for endoscopy, wound diagnostics and guided surgery.
How it works
Snapscan combines the high-resolution capabilities of linescan sensors (up to 2,048×3,652×150 in spectral range 475 to 900nm) with the convenience of a snapshot camera – with a sensor moving behind the lens using a piezo-electric translation stage. This enables a controlled scanning of the lens image field without moving the object or requiring external camera movement. Thus, even objects that cannot be placed on a translation stage (e.g. an outdoor scene or a patient’s brain) can still be scanned. Flat signal-to-noise ratios of 200 over the full spectral range have already been demonstrated thanks to software features optimizing the reconstruction and correction of hyperspectral data cubes. Scanning takes less than two seconds. If necessary, image registration can be applied to correct small movement artifacts from the imaged target. Since the sensor board is light (<25g), a piezo-stage can easily achieve the necessary level of precision.
New applications domains
Figure 2 | The snapscan Hyperspectral Imaging camera system: prototype (left), exploded view (right) (Bild: Imec vzw)
To demonstrate Snapscan’s capabilities in life sciences, medical applications and diagnostics, three experiments have been conducted. First, Snapscan was used to image a hand with one finger tied with a rubber band. The blue spectra image corresponds to the finger with a restricted blood flow, whilst the red spectra shows a normally oxygenated one. While hardly any difference between both fingers can be noted with the naked eye, the hyperspectral curves show a clear distinction in their hyperspectral response (related to the presence of oxygenated vs. deoxygenated blood). This shows the added value of Snapscan to monitor wound healing, or in surgery (for real-time follow-up on the condition of the organs), where oxygenation is essential. Next, a blood smear was imaged by attaching Snapscan to an optical port of a microscope, detailing fine structures in the sample. This proves Snapscan’s worth in microscopy applications, and for automatic counting or identification of certain types of cells. Finally, a set of six types of medical pills was imaged. Whereas some pills could not be differentiated through RGB imaging, the hyperspectral curves in the NIR spectrum made this possible – confirming the camera’s relevance in the pharmaceutical industry for pill inspection.
