In a typical product lifecycle there are four main phases: design, manufacture, operational life, and disposal. Within each of these phases there are many lower-level processes take place and form the detail of the product lifecycle. Products start with an idea or concept. Requirements are specified, before detailed design and simulation is completed. With a design complete, manufacturing is implemented with sourcing of materials, physical manufacturing, product testing, and packaging the product ready for sale.
Products are then shipped and distributed, and enter their operational life, during which the product may undergo many service processes such as user training, upgrades, and services. Finally, the product reaches the end of its useful life and should be recycled before final disposal.
A key point is that each of these processes can create data and information relevant to the product, and each process can be informed, automated, or improved by access to information from other stages of the product lifecycle. This is the main concept and rationale for what many companies are referring to as the ‚Digital Thread‘; the desire to create a unified, connected, digital record of a product across all stages of its lifecycle, where any user can access any relevant data, anywhere across the ecosystem of suppliers, manufacturers, distributors, and end-users. The strategic drivers for the creation of digital threads are based on the benefits which can be realized by users and the value which can be captured by suppliers of the underlying technology. Digital threads offer increased productivity, traceability, flexible new product introductions, feedback and closed loop quality processes, and additional insights how products perform, are used, and how they fail.
All these benefits rely on ready access to unified data across a product lifecycle, and it is no surprise that many larger companies are such as Siemens, Cisco, and Microsoft are seeking to play an active role in the digital lifecycle to capture part of the value which can be extracted from this unified data. Although vision tech is not a single technology which underpins the realization of a digital thread, there are many opportunities for the use of vision within the digital lifecycle concept. Vision remains one of the most strategic topics in automated information extraction because of its extremely wide applicability and ability to capture large amounts of information. Many companies are applying vision to enable parts of the digital lifecycle, typically either using or providing information from or to a digital thread.
CAD, Metrology, & Vision
The design phase is the start of the digital thread and often captures the ground truth of a product in a CAD model. The scope of CAD has increased dramatically over the past years, moving from being a capture of the predominantly 3D mechanical aspects of a product, to cover materials and product simulation, design-for-manufacture & design-for-quality guidelines, as well as including marketing aspects such as product branding, product renders, and product promotion. One area where CAD models have been used for many years is in industrial metrology. The model provides the fundamental design intent and defines critical dimensions and tolerances for a part, which are then tested during production by quality engineers using a variety of coordinate measurement machines. A developing theme is the evolution from offline metrology, often taking place in separate measurement rooms, toward at-line and in-line systems that offer reduced materials handling, higher throughput, greater inspection coverage, and automation. This move toward the production line is being accompanied by an increased use of optical sensing as this is more amenable to 100% measurement. Here vision tech is the primary approach, often making heavy use of 3D vision and robotic motion control.
Another developing theme is to define directly in the CAD model those features of a part are critical to a design and should be inspected or measured in production, and have this information already present in the CAD package to be consumed by a vision based metrology or inspection system. Examples of these types of system are already being developed by companies such as Zeiss and Kitov Systems, with products that are intended to directly consume CAD files containing relevant inspection or measurement data. This information is then used to automatically generate test plans before completing production inspection and collating and analyzing the results. These types of solutions are ambitious in their scope of automation, but also provide a clear strategic direction to create a link between design-time intent and automated production quality. Interestingly, there is no fundamental difference between metrology operations and vision inspections in the concept of these type of automated solutions. Both metrology and inspection can be included to provide a more unified part or product quality process.
This point also highlights an ongoing strategic evolution of the machine vision market. With the increased importance of in-process measurements and the focus on optical and vision-based measurement by traditional metrology companies, the target application space of in-production quality control and quality assurance is identical to that of traditional vision automation suppliers. This development suggests that manufacturing will see an increased convergence between vision inspection and metrology solutions and a greater overlap between metrology companies and automation suppliers.
QIF (Quality Information Framework)
A final topic related to metrology and the digital lifecycle is the relatively recent standard Quality Information Framework. QIF is a global standard that defines a CAD agnostic, XML-based format to define and organise quality information for production. QIF can provide a unified approach to communicating and using measurement plans, measurement results, measurement rules, part geometries, product manufacturing information, as well as statistical analyses. An interesting aspect of the QIF development is that it directly addresses the issue of siloed data from industrial quality processes and seeks to communicate information and results which can be consumed by multiple stakeholders downstream from an original product design, but also to communicate and use the results of quality measurements as they are added to the product record. QIF was first adopted as an ANSI standard in 2013, and recently was converted to an ISO standard ISO23592 in 2020. The standard is hosted by the Digital Metrology Standards Consortium who include many leading metrology and automation companies, suggesting a strong strategic interest in the potential of the standard.
Synthetic Data
The topic of AI is ever present in today’s media and the use of AI/machine learning in vision is a topic with very high visibility. Almost every company in the sector recognizes the importance of AI and are including this into their strategic planning. The conceptual process of a vision AI system uses representative data to train a model, often a neural network, before using this model to provide inferences on unseen data during a run-time inspection. An important aspect of this process is the availability and quality of training data. Without sufficiently representative training data, there is the risk that the model will contain bias or is unable to produce correct inferences in all cases. This problem is exacerbated in a manufacturing context, as image data is often very similar, arising from well controlled processes, and may not contain many examples of defects. Using synthetic data is one approach to addressing the issue of lack of training data and has several attractive aspects. As well as avoiding the need to collect and annotate large amounts of real image data, by directly synthesizing data from 3D scenes, perfect localization and 100% labelling of objects by class or instance can be achieved.
Various toolchains can be used to support this approach, for example the Blender 3D suite is being used by many companies. Blender’s combination of high-quality rendering engines and open Python API in a freely available package is an attractive means to develop a synthetic data approach, particularly for early-stage companies. Under a paid model, the well-established game engine provider Unity, have actively expanded into the industrial arena, providing several products for AR/VR applications, as well as a dedicated synthetic data generation service based on their photo-realistic render engine.
Directly within the industrial sector, Siemens Digital Industries have recently marketed a new service which is intended to directly provide a user with a trained AI vision model based on the user uploading either a CAD model or 3D scan of the part. The solution uses the 3D CAD model to render many synthetic images in the cloud, which are then used to train a dedicated neural network model. Once trained, the model is then provided back to the user for runtime inference at the edge. As the provider of one of the mainstream CAD suites Siemens also has a large exposure to the CAD users and is well placed to develop additional services based on use of CAD models.
