Vision Algorithms for Embedded Vision
Most computer vision algorithms were developed on general-purpose computer systems with software written in a high-level language
Most computer vision algorithms were developed on general-purpose computer systems with software written in a high-level language. Some of the pixel-processing operations (ex: spatial filtering) have changed very little in the decades since they were first implemented on mainframes. With today’s broader embedded vision implementations, existing high-level algorithms may not fit within the system constraints, requiring new innovation to achieve the desired results.
Some of this innovation may involve replacing a general-purpose algorithm with a hardware-optimized equivalent. With such a broad range of processors for embedded vision, algorithm analysis will likely focus on ways to maximize pixel-level processing within system constraints.
This section refers to both general-purpose operations (ex: edge detection) and hardware-optimized versions (ex: parallel adaptive filtering in an FPGA). Many sources exist for general-purpose algorithms. The Embedded Vision Alliance is one of the best industry resources for learning about algorithms that map to specific hardware, since Alliance Members will share this information directly with the vision community.
General-purpose computer vision algorithms
One of the most-popular sources of computer vision algorithms is the OpenCV Library. OpenCV is open-source and currently written in C, with a C++ version under development. For more information, see the Alliance’s interview with OpenCV Foundation President and CEO Gary Bradski, along with other OpenCV-related materials on the Alliance website.
Hardware-optimized computer vision algorithms
Several programmable device vendors have created optimized versions of off-the-shelf computer vision libraries. NVIDIA works closely with the OpenCV community, for example, and has created algorithms that are accelerated by GPGPUs. MathWorks provides MATLAB functions/objects and Simulink blocks for many computer vision algorithms within its Vision System Toolbox, while also allowing vendors to create their own libraries of functions that are optimized for a specific programmable architecture. National Instruments offers its LabView Vision module library. And Xilinx is another example of a vendor with an optimized computer vision library that it provides to customers as Plug and Play IP cores for creating hardware-accelerated vision algorithms in an FPGA.
Other vision libraries
- Halcon
- Matrox Imaging Library (MIL)
- Cognex VisionPro
- VXL
- CImg
- Filters
Sensor Cortek Demonstration of SmarterRoad Running on Synopsys ARC NPX6 NPU IP
Fahed Hassanhat, head of engineering at Sensor Cortek, demonstrates the company’s latest edge AI and vision technologies and products in Synopsys’ booth at the 2024 Embedded Vision Summit. Specifically, Hassanhat demonstrates his company’s latest ADAS neural network (NN) model, SmarterRoad, combining lane detection and open space detection. SmarterRoad is a light integrated convolutional network that
Annual Computer Vision and Perceptual AI Developer Survey Now Open
Every year we survey developers to understand their requirements and pain points around computer vision and perceptual AI. This survey is now in its 11th year because of people like you, who contribute their real-world insights. We share the results from the survey at Alliance events and in white papers and presentations made available throughout
Build VLM-powered Visual AI Agents Using NVIDIA NIM and NVIDIA VIA Microservices
This blog post was originally published at NVIDIA’s website. It is reprinted here with the permission of NVIDIA. Traditional video analytics applications and their development workflow are typically built on fixed-function, limited models that are designed to detect and identify only a select set of predefined objects. With generative AI, NVIDIA NIM microservices, and foundation
Quantization: Unlocking Scalability for Large Language Models
This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm Find out how LLM quantization solves the challenges of making AI work on device In the rapidly evolving world of artificial intelligence (AI), the growth of large language models (LLMs) has been nothing short of astounding. These
Ambarella and Plus Announce High Performance Transformer-based AD Perception Software Stack, PlusVision, for CV3-AD AI Domain Controller Family With Industry-leading Power Efficiency
Birds-Eye-View Vision Technology Enables OEMs to Offer L2+/L3 Autonomy Across Vehicle Models With Uniform Perception Software SANTA CLARA, Calif., July 31, 2024 — Ambarella, Inc. (NASDAQ: AMBA), an edge AI semiconductor company, and Plus, an AI-based driver assist and autonomous driving (AD) solutions provider, today announced that Plus’s PlusVision™—a high-performance transformer-based AD perception software stack
Enhance Multi-camera Tracking Accuracy by Fine-tuning AI Models with Synthetic Data
This article was originally published at NVIDIA’s website. It is reprinted here with the permission of NVIDIA. Large-scale, use–case-specific synthetic data has become increasingly important in real-world computer vision and AI workflows. That’s because digital twins are a powerful way to create physics-based virtual replicas of factories, retail spaces, and other assets, enabling precise simulations
Super-Safety and Self-Steering: Exploring Autonomous Vehicles
With more than 90% of road traffic accidents coming down to human error, the importance of autonomy and its safety benefits cannot be overlooked. As the industry moves into a new era of autonomous driving technologies, safety benchmarks and standards are evolving, forcing car companies to adopt new technologies to keep their cars competitive. IDTechEx‘s
Nota AI Demonstration of Transforming Edge AI with the LaunchX Converter and Benchmarker
Tae-Ho Kim, CTO and Co-founder of Nota AI, demonstrates the company’s latest edge AI and vision technologies and products at the 2024 Embedded Vision Summit. Specifically, Kim demonstrates his company’s LaunchX platform, featuring its powerful Converter and Benchmarker. LaunchX optimizes AI models for edge devices, reducing latency and boosting performance. Practical applications of the Converter
Nota AI Demonstration of Elevating Traffic Safety with Vision Language Models
Tae-Ho Kim, CTO and Co-founder of Nota AI, demonstrates the company’s latest edge AI and vision technologies and products at the 2024 Embedded Vision Summit. Specifically, Kim demonstrates his company’s Vision Language Model (VLM) solution, designed to elevate vehicle safety. Advanced models analyze and interpret visual data to prevent accidents and enhance driving experiences. The
Free Webinar Explores Synthetic Data for Deep Learning Model Training
On September 26, 2024 at 9 am PT (noon ET), Jakub Pietrzak, Chief Technology Officer for SKY ENGINE AI, will present the free hour webinar “Leveraging Synthetic Data for Real-time Visual Human Behavior Analysis Using the SKY ENGINE AI Platform,” organized by the Edge AI and Vision Alliance. Here’s the description, from the event registration
Nota AI Demonstration of Revolutionizing Driver Monitoring Systems
Tae-Ho Kim, CTO and Co-founder of Nota AI, demonstrates the company’s latest edge AI and vision technologies and products at the 2024 Embedded Vision Summit. Specifically, Kim demonstrates Nota DMS, his company’s state-of-the-art driver monitoring system. The solution enhances driver safety by monitoring attention and detecting drowsiness in real-time. Cutting-edge AI techniques make Nota DMS
Steering a Revolution: Optimized Automated Driving with Heterogeneous Compute
This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm Qualcomm Technologies’ latest whitepaper navigates the advantages of Snapdragon Ride Solutions based on heterogeneous compute SoCs. As the automotive industry continues to progress toward automated driving, advanced driver assistance systems (ADAS) are in high demand. These systems
Develop Generative AI-powered Visual AI Agents for the Edge
This blog post was originally published at NVIDIA’s website. It is reprinted here with the permission of NVIDIA. An exciting breakthrough in AI technology—Vision Language Models (VLMs)—offers a more dynamic and flexible method for video analysis. VLMs enable users to interact with image and video input using natural language, making the technology more accessible and
DEEPX Demonstration of Its DX-M1 AIoT Booster
Taisik Won, the President of DEEPX USA, demonstrates the company’s latest edge AI and vision technologies and products at the 2024 Embedded Vision Summit. Specifically, Won demonstrates the company’s AIoT Booster, the DX-M1. The DX-M1 is DEEPX’s flagship AI chip, meticulously engineered for seamless integration into any AIoT application. This cutting-edge chip can simultaneously process
DEEPX Demonstration of Its DX-V1 and DX-V3 AI Vision Processors
Aiden Song, PR Manager at DEEPX, demonstrates the company’s latest edge AI and vision technologies and products at the 2024 Embedded Vision Summit. Specifically, Song demonstrates the company’s DX-V1 and DX-V3 AI vision processors. The DX-V1 and DX-V3 are AI enabler chips for vision systems. The DX-V1 is a standalone edge AI chip that can
What’s Next in On-device Generative AI?
This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm. Upcoming generative AI trends and Qualcomm Technologies’ role in enabling the next wave of innovation on-device The generative artificial intelligence (AI) era has begun. Generative AI innovations continue at a rapid pace and are being woven into
