Photometric stereo is a method from the field of computational imaging. It uses multiple images of the same object or scene captured under different lighting directions. In the individual images, the surface shape creates characteristic brightness variations (shadows). For this reason, the method is also commonly referred to as Shape-from-Shading.
Time Multiplexing with Multichannel Technology
The computationally intensive processing of the captured images allows for the generation of image data with additional information content. In particular, topography (surface orientation) and texture properties (albedo) can be analysed separately. This makes it possible to visualise even subtle defects such as scratches, dents, or embossing errors with high contrast – defects that often remain hidden in conventional images. To use this method efficiently with line-scan cameras at high speeds, the four different illumination directions must be captured within a single relative motion. MSTVision achieves this through time multiplexing using its in-house developed MultiChannel technology. The complete photometric stereo computation is performed FPGA-based on a programmable frame grabber, thereby relieving the host CPU. The combination of MultiChannel acquisition, FPGA-based photometric stereo processing, and JAI’s high-speed camera Sweep SW-16000M-CXP4A enables continuous inspection of material flows at previously unattainable process speeds.
In principle, this technology can be applied equally to both area-scan and line-scan camera systems. Accurate synchronisation of the different image captures is easiest to achieve when the inspected objects are not moving during image acquisition. „Conventional photometric stereo systems are therefore often either limited in speed or highly complex in design, for example through the use of multiple cameras or external computing units,“ explains Oliver Gräff, Deputy Managing Director of MSTVision. „This makes integration into high-speed production processes considerably more difficult.“ According to Gräff, there was a clear need for compact, scalable solutions that enable photometric stereo with one or more line-scan cameras and deliver real-time results – without compromising accuracy and stability. MSTVision has developed such a solution.
JAI camera as core component
A key factor behind the enormous performance increase of MSTVision’s photometric stereo solution is the use of the Sweep SW-16000M-CXP4A line-scan camera from JAI. Its high line scan rate and data transfer speed are what allow the system to fully realise its potential in high-speed applications. „JAI cameras have impressed us overall with their high sensor quality, industrial reliability, and excellent support for high-speed line-scan camera systems,“ says Gräff. „For our application, the stable temporal characteristics and the excellent integration capability into FPGA-based processing pipelines were particularly decisive. JAI offers exactly the performance and control required for systems with synchronised multi-channel illumination and the photometric stereo method.“ According to Gräff, another decisive factor in choosing this camera was its extremely high line rate of up to 277kHz at 16k resolution. „This enables data rates of up to 4.5 GB/s and propels our photometric stereo system into speed ranges that were previously hardly achievable. Additional arguments in favor of the camera included its excellent signal-to-noise ratio, stable temporal behavior, and the CoaXPress interface.“
solution was its extremely high line rate of up to 277kHz at 16k resolution. – Image: JAI A/S
Synchronization is the challenge
Gräff identifies the precise and time-synchronous alignment between the line-scan camera and the integrated illumination at very high acquisition speeds as a key challenge in developing the system. „For photometric stereo systems, even the smallest timing deviations between the illumination channels are critical and directly affect the quality of the computed surface information. At the same time, the system must be capable of processing very high data rates in real time, remain economically viable, and be flexibly adaptable to different applications. This requires tight integration between camera control, lighting control, and FPGA-based image processing.“ MSTVision’s photometric stereo line-scan solution synchronises multiple illumination channels with a single line-scan camera down to the pixel level. Both the complete acquisition and the photometric stereo algorithm are implemented directly in the FPGA of the frame grabber, so that no CPU load is generated on the connected PC. „In addition to the raw images, the system therefore also delivers the computed photometric stereo results in real time,“ Gräff emphasizes.
High-Performance Line Light
What is not sufficiently illuminated cannot be optimally captured – even by the best camera. This old machine vision principle applies particularly to line-scan camera applications and thus also to MSTVision’s photometric stereo system. To meet the high demands for speed and light intensity, the company has therefore developed its own high-performance line light specifically for photometric stereo. „Our line illumination module, the Photometric Stereo Line Light MST-PSL, features integrated LED drivers and four directional channels through which light is emitted at defined oblique angles,“ explains Gräff. „Compared to other systems, this arrangement offers clear advantages in terms of irradiance and angular precision.“
In a typical setup, the MST-PSL operates with an average tilt angle of 35° and a lateral emission angle of the individual LEDs of 35° ± 8°. The individual LEDs are focused in such a way that, in strobe operation, irradiance levels of up to 6 kW/m² can be achieved, with a depth measurement range of 40 to 60mm below the system. The lighting concept is based on an integrated illumination unit with four lighting directions. This simplifies mechanical integration at a constant working distance and enables a particularly compact design that requires less than 300mm in the direction of motion. In line direction, standardised inspection widths of up to approximately 1,400mm are available. Custom lengths are also possible upon request.
