Beyond Imaging

The traditional boundaries between machine vision and embedded vision systems are quickly disappearing. Industrial vision systems have historically relied on PC-hosted processing and external camera interfaces, while tightly integrated embedded vision systems have incorporated image sensors, processors and interfaces. The increasing availability of cost-optimized embedded processors that support powerful integrated ISPs is leading developers to reevaluate their legacy vision system architectures. Coupled with the advent of integrated edge AI capabilities into the latest generation of embedded processors, this may lead many developers to move toward new embedded vision system architectures that support powerful new capabilities. Next-generation vision systems demand low latency, a minimal footprint and maximum energy efficiency, combined with native multimodal sensor support powered by AI-assisted ISPs. To address these requirements, MIPI Alliance is adding new capabilities to the MIPI Camera Serial Interface 2 (MIPI CSI-2) and is developing a new MIPI Kinematic Streaming Format (MIPI-KSF) specification to extend standardized transport to kinematic perception sensors such as radar and lidar.

Existing CSI-2 Capabilities for Machine Vision

Originally developed to embed camera sensors into mobile phones, MIPI CSI-2 has expanded considerably in its capabilities and applications to become ubiquitous across many industries. Current CSI-2 v4.2 capabilities relevant to machine vision applications include:

• Advanced Compression: Differential pulse code modulation (DPCM) and multi-pixel compression (MPC) for multi-pixel colour filter arrays (CFAs) reduce the volume of image data that must be transported, while preserving the signal-to-noise ratio (SNR).
• Always-On Sentinel Conduit (AOSC): Supports ultra-low-power environmental monitoring using a dedicated vision digital signal processor (VDSP), allowing the primary host processor to remain inactive until predefined trigger events are detected.
• End-to-End Security: The MIPI Camera Security Framework provides authentication and encryption capabilities for protecting CSI-2 data streams.
• Event Sensing and Processing (ESP): Adds support for event-based image sensors that transmit dynamic pixel changes rather than complete image frames, enabling applications such as high-speed edge tracking to operate within limited interface bandwidths.
• High Dynamic Range Depths: RAW28 encoding supports the transport of high-bit-depth image data for advanced computational imaging and machine learning applications.
• Latency Reduction & Transport Efficiency (LRTE): Reduces the amount of data transmitted across the interface and minimizes electromagnetic emissions, improving transport efficiency and enabling data from more sensors to be carried over fewer wires.
• Optimized PHY support for MIPI A-/C-/D-PHY: Supporting 2-wire D-PHY embedded clock mode and 3-wire C-PHY ternary encoding helps to reduce wire count, pin count and routing complexity.
• Smart ROI: Enables selected regions of an image to be streamed to downstream processors, reducing the cost of transferring complete frames when only specific scene information is required.
• Virtual Channels: CSI-2 currently supports up to 32 virtual channels, allowing multiple sensor streams to share a physical interface and reduce pin count.

• 

• 

• 

Extending CSI-2 Beyond Optical Imaging

Machine vision systems are becoming increasingly dependent on heterogeneous sensor networks, which combine optical cameras with radar, lidar and other types of image sensors. Historically, the data from kinematic sensors (which include radar, lidar and other types of non-optical sensors) has lacked a standardized streaming representation, requiring the development and use of proprietary data formats. To address this gap, a new MIPI Kinematics Working Group is developing the MIPI KSF specification, with member release planned for Q1-Q2 2027. The new specification aims to standardize a streaming format for raw kinematic sensor data and support technology-agnostic kinematic sensors, including frequency-modulated continuous-wave (FMCW) and time-of-flight (ToF) sensing. The KSF specification, in combination with CSI-2, will enable radar and lidar data to be transported to embedded processors, reducing the signal processing burden within resource-constrained sensor nodes. It will also simplify sensor fusion by enabling both optical and kinematic sensor data to be transported within the same CSI-2 framework, simplifying the integration and synchronization of multiple heterogeneous sensor streams for multi-modal perception.

MIPI is continually working to enhance CSI-2 capabilities so that it continues to support future machine vision architec-tures. In the release of the CSI-2 v4.3 specification, MIPI will add support for multi-drop bus/multi-point network topologies, enabling multiple sensors to share a common high-speed bus, and RAW32 encoding to support the transport of kinematic sensor data. Additional technical capabilities under consideration for inclusion in future CSI-2 releases include: commercial and industrial long-reach transport support; standardized ISP interface to ease integration with image signal processors; plesiochronous reference clock provisions for managing clock drift across distributed asynchronous sensor nodes; and expansion of virtual channel support to 256 channels to support increasingly complex heterogeneous sensor configurations.

www.mipi.org