Image 4 | OnBoard integrates the new 3rd generation VGA sensor camera module with MIPI CSI interface, into a powerful reference vision system ARM-based Quad Core platform. (Image: Prophesee)
Event-Based Vision Sensors
The advantage of treating dynamic visual information this way does not end at the sensing stage. In order to fully unlock the potential of these event-based vision sensors, also the paradigms of vision processing need to be fundamentally rethought. First of all, the notion of a frame at the basis of vision processing is to be abandoned altogether. As the sensors encode visual dynamics into highly resolved spatio-temporal patterns of events, representing the relevant features of scene dynamics (such as moving object contours, trajectories, velocity, etc.), processing algorithms now work on continuous time events and features instead of on discrete static images. The mathematics that describe these features in space and time are simple and elegant and yield highly efficient algorithms and computational rules that allow for real-time operation of sensory-processing systems while minimizing demand on computing power. The materialization of the research effort led to the launch of the most advanced event-based reference system: OnBoard integrates the new 3rd generation VGA sensor camera module with MIPI CSI interface, into a powerful reference vision system ARMbased Quad Core platform. Very high dynamic range, >120dB, can be achieved without the needs of multiple measurements, like in conventional HDR techniques, thanks to the time-based encoding of the illumination information,and the circuitry governing each pixel. It provides comprehensive connectivity including Ethernet, USB, HDMI, Wi-Fi, operating under a Linux OS. The embedded system runs dedicated computer vision software. It offers a tracking algorithm to detect motion, segment data into groups of spatio-temporal events and track over time (taking two out of four available cores). The application layer comprises area monitoring, high-speed counting, vibration measurement and real-time inspection.
