Computer vision algorithms are often burdened for embedded implementation due to integration time and system complexity. Many commercial systems prevent low-level image processing customization and hardware optimization due to the largely proprietary nature of the algorithms and architectures, hindering research development by the larger community. This work presents DevCAM- an open-source Multi-Processor System on Chip (MPSoC) based system, targeted at hardware-software research for vision systems, specifically for co-located sensor processor systems. The objective being to facilitate the integration of multiple latest generation sensors, abstracting interfacing difficulties to high-bandwidth sensors, enable user defined hybrid processing architectures on FPGA, CPU and GPU, and to unite multi-module systems with 40Gb/s networking and 12Gb/s off-chip NVMe storage.  The system can accommodate up to six 4-lane MIPI sensor modules which are electronically synchronized to an embedded RTK-GPS receiver and 9-axis IMU. We demonstrate a number of available configurations that can be achieved for stereo, 360, and light-field image acquisition tasks. The development framework includes mechanical, PCB, FPGA and software components for the rapid integration into any system. System capabilities are demonstrated with the focus on opening new research frontiers such as distributed edge processing, inter system synchronization sensor synchronization using GPSDO, and hybrid hardware acceleration of image processing tasks.

Paper: Electronic Imaging 2021 Conference, 3D Imaging and Applications, “DevCAM: An open-source multi-camera development system for embedded vision”, D. E. Meyer, A. M. Birlangi, F. Kuester

The supporting software repository is available here.