Vision Algorithms for Embedded Vision
Most computer vision algorithms were developed on general-purpose computer systems with software written in a high-level language
Most computer vision algorithms were developed on general-purpose computer systems with software written in a high-level language. Some of the pixel-processing operations (ex: spatial filtering) have changed very little in the decades since they were first implemented on mainframes. With today’s broader embedded vision implementations, existing high-level algorithms may not fit within the system constraints, requiring new innovation to achieve the desired results.
Some of this innovation may involve replacing a general-purpose algorithm with a hardware-optimized equivalent. With such a broad range of processors for embedded vision, algorithm analysis will likely focus on ways to maximize pixel-level processing within system constraints.
This section refers to both general-purpose operations (ex: edge detection) and hardware-optimized versions (ex: parallel adaptive filtering in an FPGA). Many sources exist for general-purpose algorithms. The Embedded Vision Alliance is one of the best industry resources for learning about algorithms that map to specific hardware, since Alliance Members will share this information directly with the vision community.
General-purpose computer vision algorithms
One of the most-popular sources of computer vision algorithms is the OpenCV Library. OpenCV is open-source and currently written in C, with a C++ version under development. For more information, see the Alliance’s interview with OpenCV Foundation President and CEO Gary Bradski, along with other OpenCV-related materials on the Alliance website.
Hardware-optimized computer vision algorithms
Several programmable device vendors have created optimized versions of off-the-shelf computer vision libraries. NVIDIA works closely with the OpenCV community, for example, and has created algorithms that are accelerated by GPGPUs. MathWorks provides MATLAB functions/objects and Simulink blocks for many computer vision algorithms within its Vision System Toolbox, while also allowing vendors to create their own libraries of functions that are optimized for a specific programmable architecture. National Instruments offers its LabView Vision module library. And Xilinx is another example of a vendor with an optimized computer vision library that it provides to customers as Plug and Play IP cores for creating hardware-accelerated vision algorithms in an FPGA.
Other vision libraries
- Halcon
- Matrox Imaging Library (MIL)
- Cognex VisionPro
- VXL
- CImg
- Filters
Open-source Physics Engine and OpenUSD Advance Robot Learning
This blog post was originally published at NVIDIA’s website. It is reprinted here with the permission of NVIDIA. The Newton physics engine and enhanced NVIDIA Isaac GR00T models enable developers to accelerate robot learning through unified OpenUSD simulation workflows. Editor’s note: This blog is a part of Into the Omniverse, a series focused on how
“Introduction to DNN Training: Fundamentals, Process and Best Practices,” a Presentation from Think Circuits
Kevin Weekly, CEO of Think Circuits, presents the “Introduction to DNN Training: Fundamentals, Process and Best Practices” tutorial at the May 2025 Embedded Vision Summit. Training a model is a crucial step in machine learning, but it can be overwhelming for beginners. In this talk, Weekly provides a comprehensive introduction… “Introduction to DNN Training: Fundamentals,
Upcoming Workshop Explores Deploying PyTorch on the Edge
On October 21, 2022 at noon PT, Alliance Member company Qualcomm, along with Amazon, will deliver the free (advance registration required) half-day in-person workshop “PyTorch on the Edge: Amazon SageMaker x Qualcomm® AI Hub” at the InterContinental Hotel in San Francisco, California. From the event page: Seamless cloud-to-edge AI development experience: Train on AWS, deploy
D3 Embedded, HTEC, Texas Instruments and Tobii Pioneer the Integration of Single-camera and Radar Interior Sensor Fusion for In-cabin Sensing
The companies joined forces to develop sensor fusion based interior sensing for enhanced vehicle safety, launching at the InCabin Europe conference on October 7-9. Rochester, NY – October 6, 2025 – Tobii, with its automotive interior sensing branch Tobii Autosense, together with D3 Embedded, and HTEC today announced the development of an interior sensing solution
Breaking Free from the CUDA Lock-in
This blog post was originally published at SiMa.ai’s website. It is reprinted here with the permission of SiMa.ai. The AI hardware landscape is dominated by one uncomfortable truth: most teams feel trapped by CUDA. You trained your models on NVIDIA GPUs, deployed them with TensorRT, and now the thought of switching hardware feels like rewriting
Upcoming Seminar Explores the Latest Innovations in Mobile Robotics
On October 22, 2022 at 9:00 am PT, Alliance Member company NXP Semiconductors, along with Avnet, will deliver a free (advance registration required) half-day in-person robotics seminar at NXP’s office in San Jose, California. From the event page: Join us for a free in-depth seminar exploring the latest innovations in mobile robotics with a focus
Free Webinar Explores Edge AI-enabled Microcontroller Capabilities and Trends
On November 18, 2025 at 9 am PT (noon ET), the Yole Group’s Tom Hackenberg, principal analyst for computing, will present the free hour webinar “How AI-enabled Microcontrollers Are Expanding Edge AI Opportunities,” organized by the Edge AI and Vision Alliance. Here’s the description, from the event registration page: Running AI inference at the edge,
“Lessons Learned Building and Deploying a Weed-killing Robot,” a Presentation from Tensorfield Agriculture
Xiong Chang, CEO and Co-founder of Tensorfield Agriculture, presents the “Lessons Learned Building and Deploying a Weed-Killing Robot” tutorial at the May 2025 Embedded Vision Summit. Agriculture today faces chronic labor shortages and growing challenges around herbicide resistance, as well as consumer backlash to chemical inputs. Smarter, more sustainable approaches… “Lessons Learned Building and Deploying
Using the Qualcomm AI Inference Suite from Google Colab
This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm. Building off of the blog post here, which shows how easy it is to call the Cirrascale AI Inference Cloud using the Qualcomm AI Inference Suite, we’ll use Google Colab to show the same scenario. In the previous blog
“Transformer Networks: How They Work and Why They Matter,” a Presentation from Synthpop AI
Rakshit Agrawal, Principal AI Scientist at Synthpop AI, presents the “Transformer Networks: How They Work and Why They Matter” tutorial at the May 2025 Embedded Vision Summit. Transformer neural networks have revolutionized artificial intelligence by introducing an architecture built around self-attention mechanisms. This has enabled unprecedented advances in understanding sequential… “Transformer Networks: How They Work
Deploy High-performance AI Models in Windows Applications on NVIDIA RTX AI PCs
This blog post was originally published at NVIDIA’s website. It is reprinted here with the permission of NVIDIA. Microsoft is now making Windows ML available to developers. Windows ML enables C#, C++ and Python developers to optimally run AI models locally across PC hardware from CPU, NPU and GPUs. On NVIDIA RTX GPUs, it utilizes
“Understanding Human Activity from Visual Data,” a Presentation from Sportlogiq
Mehrsan Javan, Chief Technology Officer at Sportlogiq, presents the “Understanding Human Activity from Visual Data” tutorial at the May 2025 Embedded Vision Summit. Activity detection and recognition are crucial tasks in various industries, including surveillance and sports analytics. In this talk, Javan provides an in-depth exploration of human activity understanding,… “Understanding Human Activity from Visual
“Multimodal Enterprise-scale Applications in the Generative AI Era,” a Presentation from Skyworks Solutions
Mumtaz Vauhkonen, Senior Director of AI at Skyworks Solutions, presents the “Multimodal Enterprise-scale Applications in the Generative AI Era” tutorial at the May 2025 Embedded Vision Summit. As artificial intelligence is making rapid strides in use of large language models, the need for multimodality arises in multiple application scenarios. Similar… “Multimodal Enterprise-scale Applications in the
How CLIKA’s Automated Hardware-aware AI Compression Toolkit Efficiently Enables Scalable Deployment of AI on Any Target Hardware
This blog post was originally published at CLIKA’s website. It is reprinted here with the permission of CLIKA. Organisations are making a shift from experimental spending on AI to long-term investments in this new technology but there are challenges involved. Here’s how CLIKA can help. With democratising AI and greater access to open-source AI models,
Using the Qualcomm AI Inference Suite Directly from a Web Page
This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm. Applying the Qualcomm AI Inference Suite directly from a web page using JavaScript makes it easy to create and understand how AI inference works in in web solutions. Qualcomm Technologies in collaboration with Cirrascale has a free-to-try
“Developing a GStreamer-based Custom Camera System for Long-range Biometric Data Collection,” a Presentation from Oak Ridge National Laboratory
Gavin Jager, Researcher and Lab Space Manager at Oak Ridge National Laboratory, presents the “Developing a GStreamer-based Custom Camera System for Long-range Biometric Data Collection” tutorial at the May 2025 Embedded Vision Summit. In this presentation, Jager describes Oak Ridge National Laboratory’s work developing software for a custom camera system… “Developing a GStreamer-based Custom Camera
