Photometric stereo is a method from the field of computational imaging. It uses multiple images of the same object or scene captured under different lighting directions. In the individual images, the surface shape creates characteristic brightness variations (shadows). For this reason, the method is also commonly referred to as Shape-from-Shading.
Time Multiplexing with Multichannel Technology
The computationally intensive processing of the captured images allows for the generation of image data with additional information content. In particular, topography (surface orientation) and texture properties (albedo) can be analysed separately. This makes it possible to visualise even subtle defects such as scratches, dents, or embossing errors with high contrast – defects that often remain hidden in conventional images. To use this method efficiently with line-scan cameras at high speeds, the four different illumination directions must be captured within a single relative motion. MSTVision achieves this through time multiplexing using its in-house developed MultiChannel technology. The complete photometric stereo computation is performed FPGA-based on a programmable frame grabber, thereby relieving the host CPU. The combination of MultiChannel acquisition, FPGA-based photometric stereo processing, and JAI’s high-speed camera Sweep SW-16000M-CXP4A enables continuous inspection of material flows at previously unattainable process speeds.
In principle, this technology can be applied equally to both area-scan and line-scan camera systems. Accurate synchronisation of the different image captures is easiest to achieve when the inspected objects are not moving during image acquisition. „Conventional photometric stereo systems are therefore often either limited in speed or highly complex in design, for example through the use of multiple cameras or external computing units,“ explains Oliver Gräff, Deputy Managing Director of MSTVision. „This makes integration into high-speed production processes considerably more difficult.“ According to Gräff, there was a clear need for compact, scalable solutions that enable photometric stereo with one or more line-scan cameras and deliver real-time results – without compromising accuracy and stability. MSTVision has developed such a solution.
JAI camera as core component
A key factor behind the enormous performance increase of MSTVision’s photometric stereo solution is the use of the Sweep SW-16000M-CXP4A line-scan camera from JAI. Its high line scan rate and data transfer speed are what allow the system to fully realise its potential in high-speed applications. „JAI cameras have impressed us overall with their high sensor quality, industrial reliability, and excellent support for high-speed line-scan camera systems,“ says Gräff. „For our application, the stable temporal characteristics and the excellent integration capability into FPGA-based processing pipelines were particularly decisive. JAI offers exactly the performance and control required for systems with synchronised multi-channel illumination and the photometric stereo method.“ According to Gräff, another decisive factor in choosing this camera was its extremely high line rate of up to 277kHz at 16k resolution. „This enables data rates of up to 4.5 GB/s and propels our photometric stereo system into speed ranges that were previously hardly achievable. Additional arguments in favor of the camera included its excellent signal-to-noise ratio, stable temporal behavior, and the CoaXPress interface.“
Synchronization is the challenge
Gräff identifies the precise and time-synchronous alignment between the line-scan camera and the integrated illumination at very high acquisition speeds as a key challenge in developing the system. „For photometric stereo systems, even the smallest timing deviations between the illumination channels are critical and directly affect the quality of the computed surface information. At the same time, the system must be capable of processing very high data rates in real time, remain economically viable, and be flexibly adaptable to different applications. This requires tight integration between camera control, lighting control, and FPGA-based image processing.“ MSTVision’s photometric stereo line-scan solution synchronises multiple illumination channels with a single line-scan camera down to the pixel level. Both the complete acquisition and the photometric stereo algorithm are implemented directly in the FPGA of the frame grabber, so that no CPU load is generated on the connected PC. „In addition to the raw images, the system therefore also delivers the computed photometric stereo results in real time,“ Gräff emphasizes.
