Could you explain what challenge in machine vision inspired the development of Genicam twenty years ago?
Fritz Dierks: GenIcam was created, when there was a discussion about using GigE as an interface for cameras, because we had two problems. The technical problem was that it was common to use a fixed register layout for camera standards like iDC or 1394 FireWire cameras, but it was extremely inflexible. Nobody who uses a standard driver could use a custom feature and there was another problem with fixed register layouts because you couldn’t change implementation. Then there was a ‚political‘ problem because all major companies fought for their own register layout standard. At that point, I remember that Basler had a configuration tool for their cameras, which wouldn’t fix the registers. You had a configuration layer where you could design features by names, type and meaning to configure which registers you have to set and get in order to touch these features. I propose that this might be a way out of the problem. We don’t agree on all the details of the register layout, but we standardise names, types and meanings and also standardised an XML format, where you can put in the mapping between the features and the details of the camera. This could be specific for every camera and every vendor, so everybody would be free to add their own register layout. If I had known how much work that would be for me, I’m not sure I would have proposed it. But in the end, it worked and that was the birth of GenIcam.
What was the biggest hurdle in the beginning?
Dierks: Making people agree on a totally different way of dealing with camera features because everybody was aware how fixed register layout worked. GenIcam was a completely new idea and people were a bit reluctant to adopt it. We were in a desperate situation because the market expected the GigE vision standard to become live at 2006 and we were at the end of 2004. Furthermore we had the GenAPI mechanism, which is the XML file, and the first version of the standard implementation should be shown at Vision 2005. People said we need a list of standard feature names and types, but we didn’t have, because we were focused on getting the standard mechanism running. So there was an emergency meeting beginning of 2006 hosted by National Instruments. Within one week we agreed on the basic set of the new SNFC (Standard Feature Naming Convention), which is still there today.
How does GenIcam balance standardisation with flexibility for manufacturers to offer their own unique features?
Michael Schmidt: Most devices for machine vision applications like cameras have a rather simple register interface. This was even more true twenty years ago. It´s like a register interface to an assembly language: It’s compact and efficient, but each processor model brings its own instruction. Today we have high level languages like C++ that allow you to describe your program in a processor independent way. The compiler translates your source code to machine code and takes care of many other things like type checking… GenAPI is a bit like the compiler. It provides a high level programming interface and translates it to the low level register interface nobody really wants to work with. You can write anything with a C++ compiler, but all programs that operate under a certain operating system (Linux or Windows), must agree on the same interface to open a main window. SFNC defines the features for standard functions and how they work. That makes it easier to write software that works with devices from multiple vendors. But you can still add features that are not standardised, e.g. your own version of exposure time control.
What was the first GenIcam module to be ratified in 2006 and what are the main developments since then?
Dierks: You need three modules to cover the key things in order to deal with a camera. First one is the configuration mechanics, which is the GenAPI module, the 2nd one is SNFC to get standardised names for the features. But there is a 3rd one which is the transport layer driver access. Because if you want to use drivers from 3rd parties, you need a standardised way of dealing with them. The reason why this wasn’t released in the beginning was mainly about IEEE 1394 and GigE Vision. For those interfaces, the camera manufacturers have already their own drivers. So the business needs for an interface dealing with drivers was not that much. But it changed with CoaXPress (CXP) because it is a frame grabber based interface. And the frame grabbers used to come from different vendors than the cameras.
Schmidt: Another important module is PFNC (Pixel Format Naming Convention), which standardises the names and formats of pixels. If someone uses, e.g. an 8bit Bayer format, they know how data formatting goes. But if you have a frame grabber the software has no idea how the frame grabber will provide the serial interface to the camera. Later came GenCP (Genric Control Protocol) a standardised protocol for the control channel that was used in USB3 vision. The idea behind it was: you do it once and then can use it for different transport layers. The latest edition was GenDC (Generic Data Container), a format to combine several image components in one container and provide a generic way to transport this container.
How might AI and embedded vision influence future GenIcam developments?
Schmidt: AI processing is usually done on a host system, like a GPU. Embedded systems sometimes provide accelerators that can be used inside a camera. Even if they typically act and feel like a regular Linux system, with one or more cameras attached to it, what we have is a standard system that looks like a camera. GenIcam has all that’s necessary, to connect a camera to an AI or embedded system.
Dierks: There has been quite some attempt to make GenIcam useful for embedded, but we were not really successful. The reason for that are two technical and a political challenge. The technical challenge is that embedded use is implemented on video for Linux. People are using that functionality for cameras which are implemented in kernel mode drivers. It is not really suitable to be run in kernel mode because it’s quite a large thing. The other challenge is that in the embedded world, people are used to program using Gstreamer, a different programming model than the one we’ve chosen for GenIcam. The political challenge is that a lot of embedded systems are running under Linux and the Linux community is strongly insisting on Open Source software. But the reference implementation is currently not Open Source, because we wanted to make sure that it isn’t forked. When you are a camera manufacturer and you have to test a new product, you test it against the reference implementation and this is something every software vendor is using. Then you get away with much lower testing effort than if you needed to get the implementations from every software vendors. The same argument goes the other way around. If you’re a software vendor and you want to test with a camera, you can rely on. This is why GenIcam is not Open Source. The approach to solve the kernel mode problem is typically that you run the GenIcam reference implementation as a background service (daemon) and as a normal application software (userland). If the kernel mode gets kind of a call set, then you just switch to userland and ask the daemon what kind of register you have to set. Then you can set it from kernel mode. There are prototypes from that model which works pretty well. The GStreamer problem can also be solved: There are several implementations around where you can just make a GStreamer pipeline and put in the pipeline something like ‚gain minus ten‘, and you can seamlessly put that into the GStreamer syntax. The EMVA, which is hosting GenIcam, is currently discussing whether to make the reference implementation Open Source.
How do you expect GenIcam to evolve by 2036?
Schmidt: One major extension we are already working on is a standardised feature interface API (working title GenFire). The current GenAPI defines the format of the XML description, but the reference implementation to interpret the XML file is technically not a standard. It has many more functions than needed for feature access. Our idea is to specify the functions and application needs to enable different and simpler implementations. This will allow us to to bring devices up under the GenIcam umbrella that couldn´t be integrated so far. For example, devices, where you don’t have an XML description, but an API. Another possible extension is that the CCNs should gain more attraction. There might be a need to standardise the way these AI networks are configured.
Dierks: If you look at the way software interfaces are made today, you see that AI is changing things drastically. Today the standard is a GUI interface. Everything you can do with a software, you can see in the GUI and you have tons and tons of menus and stuff. The problem with that approach is that software is becoming so feature rich that the GUI is getting too complex and it has too many features, so you can´t really deal with it any more. The next step is that you just get a chat interface, with a little GUI to see the status, but in the end, you tell your software what you want it to do, and then it just do it. Something similar might happen to the way cameras are configured. In the old days, we had a fixed register layout. Today it’s GenIcam. But even with GenIcam, you have the problem that there are too many features today. My vision is that you get a kind of MCP (Model Context Protocol) style interface based on a SDK. You just take your AI, connect it to the camera and the camera not only exposes the features, but also certain traits on how the image will be treated. You will be able to tell the camera: Can you please increase the contrast, because is looks awful. Then the camera will just look at a histogram, find out what to do, change it, and show a live image. You say OK and then everything is done. We already have done prototypes for that. Maybe some of these ideas may even make it to GenIcam one day.
M. Schmidt: We built an interface ready for modern AI twenty years ago. SFNC works great as an input to large language models like ChatGPT, because it’s a collection of the common wisdom of the machine vision industry. It’s a perfect instruction, how to use a machine vision camera. The XML district description can provide most of the necessary information for a LLM to understand and use the custom features of a camera that are not in the SFNC because the XML description has documentation fields which are a good guide for an AI to configure and to compose systems. Generative AI is nothing I see in machine vision devices, in the future. It will have its place, when it comes to creating and optimising machine vision systems. LLMs should be able to propose the right components and tell you how to connect them and then write the application for it. But I don’t see it in the image processing loop because it’s just too slow and too unpredictable for most applications. GenIcam has a good future in combination with AI systems and will look very similar to what we have today in ten years.
