Ultra-Fast Focus Stack Capturing

3D Reconstruction by Ultra-Fast Frame-by-Frame Focusing

The TAG Zip device is the world’s fastest true volumetric imaging system. Powered by the world’s fastest focusing lens, the new system has the ability to change the focal plane for each frame of the camera sensor independently from the focal location of the previous frame. In fact, the rate limiting step is entirely dependent on the camera and data transfer rate.

Picture 1: Thanks to its speed, the TAG Zip can max-out the frame rate of any camera while having each frame be located at any focal plane. Above, the TAG Zip was used to image a robotic gripper, a beer bottle and a picture placed in the background. The TAG Zip system can then provide an EDOF image of the scene or an edge detection depth map to help position the robotic gripper as it approaches the object to pickup. (Bild: TAG Optics Inc.)

Systems based on technologies such as time-of-flight, Lidar, and structured-light are increasing in prevalence and can provide a tremendous amount of 3D-shape information about the volume. However, this level of detail comes with drawbacks delivering too much precision in areas where there are no objects and in many cases not enough precision on specific areas of interest. Moreover the limited speed of these technologies also presents challenges to a robotic assisted system Reconstructing 3D-volumes using an image stack, often called Shape From Focus (SFF) or Depth From Focus/Defocus (DFF), is another technique that has been widely used in biology and static systems for gaining multi-dimensional information. The major advantage of this technique is its high spatial accuracy and other information gained about the objects in the image volume such as geometry, 3D-map-rendering, decoding of text, QR/bar codes, true color and gradients, etc. However, this technique has been little used in dynamic environments due to the time required to focus and capture the volume.

Breaking the speed barrier

By pairing the TAG Zip system with state of the art industrial high-speed machine vision cameras, the system can achieve speeds of 300 to 500fps; with each frame natively containing full color images encoded with x, y, and z data. The system’s speed even allows for inspection of multiple arbitrary user-defined regions of interest within the 3D-space providing real-time positioning, object recognition, and measurement information. In fact, the TAG-technology has a staggering theoretical speed limitation of three million FPS. Moreover, the focusing technology underlying the TAG Lens is operated on a closed loop system ensuring reliability, repeatability, and traceability; key elements for mission critical applications such as robotic guidance and go/no-go part inspection. The versatility enabled by ultra-fast frame-by-frame focusing not only unlocks the possibility to run multiple processing algorithms such as DFF and object recognition on live volumetric image stacks, but also allows new smart image capture processes. Specifically, after performing a first rough, but uniform coverage of a specified volume, the data contained can quickly be interpreted and used for further on-the-fly targeting. Higher density image collections can quickly be performed at targeted locations for detail extraction – all while continuously running the rough scan to ensure progressive and continuous monitoring of any changes in the 3D-field-of-view. A further advantage is the progressive resolution capability of the system. When a robotic arm needs to select a particular component, it must first locate the part in 3D. However the accuracy reqirements change throughout the process: as the arm approaches its target, the precision level must increase and eventually reach mm and even sub-mm resolution. The TAG Zip system can extract highly precise 3D information about objects near the imaging lens versus objects located meters away due to the native behavior of the depth-of-field by being closer to the imaging lens. This can be done while still analyzing the full 3D environment to detect moving or stationary objects and prevent possible obstruction or accidents.

Resume

Picture 2: Following a preliminary rough pass through the scene, the TAG Zip can then hone-in on any detected objects to perform higher density image collections and generate further 3D-information. On the fly processing can then be used to extract additional metadata about the particular focal locations where objects are detected such as edge detections and 3D-axia-locations. (Bild: TAG Optics Inc.)

Smart factories must be able to adapt to ever changing complex and demanding environments due to moving parts and multiple processes involved in manufacturing and assembly. As computational technology evolves, sensing systems will become significant limitations for the ever changing smart factory production lines. The advantages of a continuous 3D-volumetric systems based on high-speed capturing of images at different focal locations will result in the generation of critical information about parts of interest and their surroundings. Moreover, thanks to the high FPS bandwidth associated with the TAG Zip’s underlying technology, new development in both chip and sensor can be seamlessly incorporated and will insure that the TAG Zip 3D-volumetric-imaging-system will continue to be at the cutting edge of speed and reliability.