Processors for Embedded Vision
THIS TECHNOLOGY CATEGORY INCLUDES ANY DEVICE THAT EXECUTES VISION ALGORITHMS OR VISION SYSTEM CONTROL SOFTWARE
This technology category includes any device that executes vision algorithms or vision system control software. The following diagram shows a typical computer vision pipeline; processors are often optimized for the compute-intensive portions of the software workload.
The following examples represent distinctly different types of processor architectures for embedded vision, and each has advantages and trade-offs that depend on the workload. For this reason, many devices combine multiple processor types into a heterogeneous computing environment, often integrated into a single semiconductor component. In addition, a processor can be accelerated by dedicated hardware that improves performance on computer vision algorithms.
While computer vision algorithms can run on most general-purpose CPUs, desktop processors may not meet the design constraints of some systems. However, x86 processors and system boards can leverage the PC infrastructure for low-cost hardware and broadly-supported software development tools. Several Alliance Member companies also offer devices that integrate a RISC CPU core. A general-purpose CPU is best suited for heuristics, complex decision-making, network access, user interface, storage management, and overall control. A general purpose CPU may be paired with a vision-specialized device for better performance on pixel-level processing.
Graphics Processing Units
High-performance GPUs deliver massive amounts of parallel computing potential, and graphics processors can be used to accelerate the portions of the computer vision pipeline that perform parallel processing on pixel data. While General Purpose GPUs (GPGPUs) have primarily been used for high-performance computing (HPC), even mobile graphics processors and integrated graphics cores are gaining GPGPU capability—meeting the power constraints for a wider range of vision applications. In designs that require 3D processing in addition to embedded vision, a GPU will already be part of the system and can be used to assist a general-purpose CPU with many computer vision algorithms. Many examples exist of x86-based embedded systems with discrete GPGPUs.
Digital Signal Processors
DSPs are very efficient for processing streaming data, since the bus and memory architecture are optimized to process high-speed data as it traverses the system. This architecture makes DSPs an excellent solution for processing image pixel data as it streams from a sensor source. Many DSPs for vision have been enhanced with coprocessors that are optimized for processing video inputs and accelerating computer vision algorithms. The specialized nature of DSPs makes these devices inefficient for processing general-purpose software workloads, so DSPs are usually paired with a RISC processor to create a heterogeneous computing environment that offers the best of both worlds.
Field Programmable Gate Arrays (FPGAs)
Instead of incurring the high cost and long lead-times for a custom ASIC to accelerate computer vision systems, designers can implement an FPGA to offer a reprogrammable solution for hardware acceleration. With millions of programmable gates, hundreds of I/O pins, and compute performance in the trillions of multiply-accumulates/sec (tera-MACs), high-end FPGAs offer the potential for highest performance in a vision system. Unlike a CPU, which has to time-slice or multi-thread tasks as they compete for compute resources, an FPGA has the advantage of being able to simultaneously accelerate multiple portions of a computer vision pipeline. Since the parallel nature of FPGAs offers so much advantage for accelerating computer vision, many of the algorithms are available as optimized libraries from semiconductor vendors. These computer vision libraries also include preconfigured interface blocks for connecting to other vision devices, such as IP cameras.
Vision-Specific Processors and Cores
Application-specific standard products (ASSPs) are specialized, highly integrated chips tailored for specific applications or application sets. ASSPs may incorporate a CPU, or use a separate CPU chip. By virtue of their specialization, ASSPs for vision processing typically deliver superior cost- and energy-efficiency compared with other types of processing solutions. Among other techniques, ASSPs deliver this efficiency through the use of specialized coprocessors and accelerators. And, because ASSPs are by definition focused on a specific application, they are usually provided with extensive associated software. This same specialization, however, means that an ASSP designed for vision is typically not suitable for other applications. ASSPs’ unique architectures can also make programming them more difficult than with other kinds of processors; some ASSPs are not user-programmable.
Sean Connelly, CEO of Luxonis partner Noroc, demonstrates the company’s latest edge AI and vision technologies and products at the 2021 Embedded Vision Summit. Specifically, Connelly demonstrates from-behind collision detection for bicycle riders. Connelly shows an example of localizing and estimating the trajectories of vehicles behind you to know if they are on a trajectory
Martin Peterlin, Chief Technology Officer at Luxonis, demonstrates the company’s latest edge AI and vision technologies and products at the 2021 Embedded Vision Summit. Specifically, Peterlin demonstrates neural inference-controlled crop, zoom and H.265 encode. Peterlin shows how to use a neural network to guide what portion of a high-resolution (12 Mpixel) image sensor is output
New MP10304 PCIe card delivers up to 100 TOPS of performance at under 25W Redwood City, Calif. – Nov. 16, 2021 – Mythic, the pioneering AI processor company with breakthrough analog compute-in-memory technology, today unveiled its new MP10304 PCIe card which features four M1076 Mythic Analog Matrix Processors (AMPs), delivering up to 100 TOPs of
Luxonis Demonstration of Spatial Artificial Intelligence and Computer Vision for Human/Machine Safety
Brandon Gilles, CEO of Luxonis, demonstrates the company’s latest edge AI and vision technologies and products at the 2021 Embedded Vision Summit. Specifically, Gilles demonstrates spatial artificial intelligence (AI) and computer vision (CV) for human/machine safety. In situations where people need to interact with potentially dangerous equipment, it is often not possible to physically guard
Lattice Semiconductor partner Ignitarium demonstrates the company’s latest edge AI and vision technologies and products at the 2021 Embedded Vision Summit. Specifically, Ignitarium demonstrates AI-based unit inspection using Lattice Semiconductor’s ECP5 FPGA. This demonstration was developed by Lattice Semiconductor partner Ignitarium, experts in applied AI technology. It shows how you can implement efficient machine learning/AI
Lattice Semiconductor partner Helion-Vision demonstrates the company’s latest edge AI and vision technologies and products at the 2021 Embedded Vision Summit. Specifically, Helion-Vision demonstrates its IONOS ISP running on Lattice Semiconductor’s ECP5 FPGA. Lattice Semiconductor has partnered with Helion-Vision to bring its industry-leading IONOS image signal processing (ISP) IP portfolio to the Lattice ECP5 FPGA.
Deepak Boppana, Senior Director of Segments and Solutions Marketing at Lattice Semiconductor, demonstrates the company’s latest edge AI and vision technologies and products at the 2021 Embedded Vision Summit. Specifically, Boppana demonstrates a development board equipped with a CrossLink-NX FPGA, based on the Lattice Semiconductor Nexus platform. The FPGA is programmed to perform human presence
Sean Hehir is a proven technology executive with strong commercial skills and significant experience in driving explosive growth and strategic alliances. Sydney – 15 November, 2021 – BrainChip Holdings Ltd (ASX: BRN), (OTCQX: BCHPY), a leading provider of ultra-low power, high performance artificial intelligence technology and the world’s first commercial producer of neuromorphic AI chips,
Nadav Eden, Technical Account Manager at Hailo, demonstrates the company’s latest edge AI and vision technologies and products at the 2021 Embedded Vision Summit. Specifically, Eden demonstrates state-of-the-art object detection on the edge with the Hailo-8 AI processor. YoloV5m is a high-end neural network model for object detection, a fundamental computer vision task. The Hailo-8
Yaniv Sulkes, Vice President of Product Marketing and Automotive at Hailo, demonstrates the company’s latest edge AI and vision technologies and products at the 2021 Embedded Vision Summit. Specifically, Sulkes demonstrates how to leverage the high throughput of the Hailo-8 AI processor in order to improve large-scale object detection on the edge. The high throughput
This blog post was originally published at NVIDIA’s website. It is reprinted here with the permission of NVIDIA. DeepStream SDK 6.0 is now available for download. A powerful AI streaming analytics toolkit, DeepStream helps developers build high-performance, low-latency, complex, video analytics applications, and services. Graph Composer This release introduces Graph Composer, a new low-code programming
Turn-key solution for companies developing Smart City AI vision systems Enables developers to rapidly train AIs with synthetic data on novel use-cases including pedestrian and traffic management Sheffield, UK, 11 November – Mindtech Global, developer of the world’s leading end-to-end synthetic data creation platform for training AI vision systems – has released a new application
Amol Borkar, Senior Product Marketing Manager for Vision and AI DSPs at Cadence, demonstrates the company’s latest edge AI and vision technologies and products at the 2021 Embedded Vision Summit. Specifically, Borkar demonstrates various vision and AI applications, including real-time examples of driver monitoring, automotive perception, 6DoF SLAM and semantic segmentation, running on Tensilica processor-
Doug Watt, Director of Field Applications Engineering at Blaize, demonstrates the company’s latest edge AI and vision technologies and products at the 2021 Embedded Vision Summit. Specifically, Watt demonstrates sensor fusion and semantic segmentation processing on the Blaize Pathfinder P1600. The use case fuses data received from full HD cameras and from lidar and radar
Latest Version of Solution Stack Offers New Features For Power-Efficient AI/ML Inferencing HILLSBORO, Ore. – Nov. 10, 2021 – Lattice Semiconductor Corporation (NASDAQ: LSCC), the low power programmable leader, today announced a roadmap of low power, AI/ML-enabled solutions that improve battery life and enable innovative user experiences in Edge applications such as Client Compute devices.
Rajesh Anantharaman, Senior Director of Products at Blaize, demonstrates the company’s latest edge AI and vision technologies and products at the 2021 Embedded Vision Summit. Specifically, Anantharaman demonstrates both people and pose detection and key point tracking using the Blaize Picasso SDK. A high resolution, multi-neural network and multi-function graph-native application built on the Blaize
Shawn Holiday, Senior Director for Customer Success at Blaize, demonstrates the company’s latest edge AI and vision technologies and products at the 2021 Embedded Vision Summit. Specifically, Holiday demonstrates multi-camera object detection on the Blaize Xplorer X1600E. The use case processes 5 independent HD video streams using 5 independent YoloV3 networks with less than 100
NVIDIA Sets Path for Future of Edge AI and Autonomous Machines With New Jetson AGX Orin Robotics Computer
NVIDIA Ampere Architecture Comes to Jetson, Delivering up to 200 TOPS and 6x Performance Gain Tuesday, November 9, 2021—GTC—NVIDIA today introduced NVIDIA Jetson AGX Orin™, the world’s smallest, most powerful and energy-efficient AI supercomputer for robotics, autonomous machines, medical devices and other forms of embedded computing at the edge. Built on the NVIDIA Ampere architecture,
NVIDIA Announces Major Updates to Triton Inference Server as 25,000+ Companies Worldwide Deploy NVIDIA AI Inference
Capital One, Microsoft, Samsung Medison, Siemens Energy, Snap Among Industry Leaders Worldwide Using Platform Tuesday, November 9, 2021—GTC—NVIDIA today announced major updates to its AI inference platform, which is now being used by Capital One, Microsoft, Samsung Medison, Siemens Energy and Snap, among its 25,000+ customers. The updates include new capabilities in the open source
Umair Hashim, Principal Solutions Engineer at Arm, demonstrates the company’s latest edge AI and vision technologies and products at the 2021 Embedded Vision Summit. Specifically, Hashim demonstrates an open source approach To cloud-native vision workload deployment on the Arm architecture. Hashim showcases how to use open source lightweight Kubernetes-based orchestration for deploying and managing vision