With FSM:GO and the Sony IMX678 sensor of the Sony Starvis 2 series, you will lose no time building your embedded vision application. The FSM:GO, together with our adapter ecosystem, allows you to get your application ready for production efficiently, quickly, and reliably without any significant prior knowledge.
Our demonstrator shows how easy it is:
You develop your application as usual with a developer board of your choice and the corresponding tool chain (for our demonstrator, we use the NVIDIA® Jetson AGX Orin™).
The Jetson AGX Orin™ is connected to the optical sensor modules using our standard FPA with a highly flexible micro coax cable, ideally suited for routing inside a tight product enclosure. The 4-lane MIPI CSI-2 interface with 2.5 Gbps/lane enables both sensors to operate at full bandwidth and deliver 4K/60 FPS. The sensor pair is installed on a mounting bracket at the factory and calibrated for the stitching application to achieve the best optical performance and for easy assembly in the customer’s production line.
The application layer generates a panoramic live image with the aid of these two 4K image sensors and is realized with two standard packages of our partner RidgeRun: Their highly performant and CUDA optimized “CUDA Stitcher” merges the two images after the lens distortion correction has been performed on the two 90° FOV images using the “CUDA Undistort” module.
Since we have already tuned the image signal processor (ISP) of NVIDIA’s high-end System on Module (SoM) for the best image results with the given sensor lens combination, you can get started immediately with your core competency – the user application!
How we support your solution:
- Two FRAMOS Sensor Modules with Sony IMX678, factory assembled to dual sensor mounting bracket
- Factory focused lenses with 90° Horizontal Field of View (HFOV)
- PixelMate™ to Micro-Coax adapter with 300 mm cable
- High-performance development kit NVIDIA® Jetson AGX Orin™
- Driver and ISP tuning for NVIDIA® Jetson AGX Orin™
- RidgeRun “Cuda Stitcher” and “Cuda Undistort” modules
That’s why FSM:GO with IMX678:
This FRAMOS Sensor Module (FSM) provides the Sony IMX678 sensor on a very compact 26.5 mm x 26.5 mm module. The rolling shutter sensor has a native resolution of 8.3 MP and an optical format of 1/1.8 at a pixel size of 2 x 2 µm. The Module has a MIPI CSI-2 interface with up to 4-data lanes. As part of the FRAMOS Sensor Module Ecosystem, it is part of a sophisticated connectivity framework and leverages flexible compatibility to several processing platforms, accessing a comprehensive lineup of adapters. Optional FRAMOS Sensor Adapters (FSA), therefore, establish the standard interface to a line-up of FRAMOS Processor Adapters (FPA), allowing them to connect to a variety of proprietary processing platforms. The customer-specific implementation is further supported by standardized software and drivers that include reference drivers for selected platforms, starting with the NVIDIA® Jetson™ Family or NXP® i.MX 8M Plus™ development boards.
Application
With the help of this setup, a wide field can be observed ranging from applications such as panoramic image creations to smart AI-based observation of crowds. In industrial contexts, it is possible to gather information on factory sites or within production areas, building the foundation for optimizing work processes.
Areas of application:
- Streaming Solutions & Sports Analytics
- 360° and Virtual Reality
- Security & Surveillance: Observation of crowds
- Digital Twin & Property Mapping
Build your embedded vision system with the FSM:GO IMX678 Development Kit: FSM-IMX678 Development Kit | FRAMOS
