From CPU to GPU
The main idea for the solution was to transfer most of the computations from the central processor (CPU) to the graphics processor (GPU). To solve that task, Fastvideo developed the Fastvideo SDK, which works on Nvidia GPU. This approach has accelerated the necessary algorithms and simplified the software architecture, because computations in this case no longer interfere with system functions based on the CPU. Let’s look at the advantages of image processing on a GPU instead of a CPU:
- • A graphics card is a more specialized device than a CPU, and due to its architecture, it can perform many tasks much faster.
- • A graphics processor can offer significantly faster memory access than a central processor.
- • The algorithms can be run in parallel at graphics cards.
Transferring computations to the graphics card does not mean that the CPU is completely free. The CPU is responsible for I/O and system control. The proposed solution is heterogeneous, since it uses all the available resources of both the CPU and GPU for image processing, which in turn leads to high performance. In addition to increasing the speed of image processing, using a graphics processor has allowed Fastvideo to implement more complicated algorithms to increase the image quality and color reproduction. The workflow is similar to that used in filmmaking, where the colors in the frame are given special attention.
Image 3 | The wind turbine inspection drone from Alerion is intended to fully automate the inspection of wind turbines for damage. The time spent on inspection of one turbine was reduced from 2h to 10min. (Image: Alerion)
Walking Excavator And Drones
One of the best examples where the Fastvideo SDK this solution can be applied is image processing for Ximea cameras. The cameras are used, for example, in the Menzi Muck remote-controlled walking excavator. For this particular project, the Fastvideo SDK solution allowed up to 60fps of synchronized image acquisition from each of the two 3.1MP cameras; real-time processing with H.264/H.265 encoding and streaming (including black level, white balance, demosaicing, auto exposure, etc.) and glass-to-glass video latency over 4G/5G network with nearly 50ms. Another project using Ximea cameras is the wind turbine inspection drone from Alerion. This drone is intended to fully automate the inspection of wind turbines for damage. For this task, it is important to ensure good quality of images, based on which a 3D model is subsequently built. Using the cameras in conjunction with the GPU image processing solution made it possible to achieve the required image quality and high processing speed, which in turn made it possible to automate the inspection process. As a result, the time spent on inspection of one turbine was reduced from 2 to 3h to 10min. Here, of course, process automation played a big role. However, this would not have been possible without high processing speed and excellent image quality that allows even very small damage to be noticed. The Fastvideo SDK for GPU image processing can work on both stationary and mobile graphics cards.
