Processors for Embedded Vision
THIS TECHNOLOGY CATEGORY INCLUDES ANY DEVICE THAT EXECUTES VISION ALGORITHMS OR VISION SYSTEM CONTROL SOFTWARE
This technology category includes any device that executes vision algorithms or vision system control software. The following diagram shows a typical computer vision pipeline; processors are often optimized for the compute-intensive portions of the software workload.
The following examples represent distinctly different types of processor architectures for embedded vision, and each has advantages and trade-offs that depend on the workload. For this reason, many devices combine multiple processor types into a heterogeneous computing environment, often integrated into a single semiconductor component. In addition, a processor can be accelerated by dedicated hardware that improves performance on computer vision algorithms.
While computer vision algorithms can run on most general-purpose CPUs, desktop processors may not meet the design constraints of some systems. However, x86 processors and system boards can leverage the PC infrastructure for low-cost hardware and broadly-supported software development tools. Several Alliance Member companies also offer devices that integrate a RISC CPU core. A general-purpose CPU is best suited for heuristics, complex decision-making, network access, user interface, storage management, and overall control. A general purpose CPU may be paired with a vision-specialized device for better performance on pixel-level processing.
Graphics Processing Units
High-performance GPUs deliver massive amounts of parallel computing potential, and graphics processors can be used to accelerate the portions of the computer vision pipeline that perform parallel processing on pixel data. While General Purpose GPUs (GPGPUs) have primarily been used for high-performance computing (HPC), even mobile graphics processors and integrated graphics cores are gaining GPGPU capability—meeting the power constraints for a wider range of vision applications. In designs that require 3D processing in addition to embedded vision, a GPU will already be part of the system and can be used to assist a general-purpose CPU with many computer vision algorithms. Many examples exist of x86-based embedded systems with discrete GPGPUs.
Digital Signal Processors
DSPs are very efficient for processing streaming data, since the bus and memory architecture are optimized to process high-speed data as it traverses the system. This architecture makes DSPs an excellent solution for processing image pixel data as it streams from a sensor source. Many DSPs for vision have been enhanced with coprocessors that are optimized for processing video inputs and accelerating computer vision algorithms. The specialized nature of DSPs makes these devices inefficient for processing general-purpose software workloads, so DSPs are usually paired with a RISC processor to create a heterogeneous computing environment that offers the best of both worlds.
Field Programmable Gate Arrays (FPGAs)
Instead of incurring the high cost and long lead-times for a custom ASIC to accelerate computer vision systems, designers can implement an FPGA to offer a reprogrammable solution for hardware acceleration. With millions of programmable gates, hundreds of I/O pins, and compute performance in the trillions of multiply-accumulates/sec (tera-MACs), high-end FPGAs offer the potential for highest performance in a vision system. Unlike a CPU, which has to time-slice or multi-thread tasks as they compete for compute resources, an FPGA has the advantage of being able to simultaneously accelerate multiple portions of a computer vision pipeline. Since the parallel nature of FPGAs offers so much advantage for accelerating computer vision, many of the algorithms are available as optimized libraries from semiconductor vendors. These computer vision libraries also include preconfigured interface blocks for connecting to other vision devices, such as IP cameras.
Vision-Specific Processors and Cores
Application-specific standard products (ASSPs) are specialized, highly integrated chips tailored for specific applications or application sets. ASSPs may incorporate a CPU, or use a separate CPU chip. By virtue of their specialization, ASSPs for vision processing typically deliver superior cost- and energy-efficiency compared with other types of processing solutions. Among other techniques, ASSPs deliver this efficiency through the use of specialized coprocessors and accelerators. And, because ASSPs are by definition focused on a specific application, they are usually provided with extensive associated software. This same specialization, however, means that an ASSP designed for vision is typically not suitable for other applications. ASSPs’ unique architectures can also make programming them more difficult than with other kinds of processors; some ASSPs are not user-programmable.
STMicroelectronics Extends STM32Cube.AI Development Tool with Support for Deeply Quantized Neural Networks
21 Jul 2022 | Geneva | STMicroelectronics has released STM32Cube.AI version 7.2.0, the first artificial-intelligence (AI) development tool by an MCU (microcontroller) vendor to support ultra-efficient deeply quantized neural networks. STM32Cube.AI converts pretrained neural networks into optimized C code for STM32 microcontrollers (MCUs). It is an essential tool for developing cutting-edge AI solutions that make
Daniel Firu, Co-founder and CPO of Quadric, presents the “Accelerate All Your Algorithms with the quadric q16 Processor,” tutorial at the May 2022 Embedded Vision Summit. As edge and embedded vision applications increasingly incorporate neural networks, developers are looking to add neural network accelerator functionality to their systems. There is… “Accelerate All Your Algorithms with
“Natural Intelligence Outperforms Artificial Intelligence for Autonomy and Vision,” a Presentation from Opteran Technologies
James Marshall, Chief Scientific Officer at Opteran Technologies, presents the “Natural Intelligence Outperforms Artificial Intelligence for Autonomy and Vision” tutorial at the May 2022 Embedded Vision Summit. Mainstream approaches to AI for autonomy and computer vision make use of data-, energy- and compute-intensive techniques such as deep learning, which struggle… “Natural Intelligence Outperforms Artificial Intelligence
World’s First AV1 Decoder Silicon IP with Support for 12-bit Pixel Size and 4:4:4 Chroma Sub-Sampling Released by Allegro DVT
Grenoble, France – June 16th 2022 – Allegro DVT, the leading provider of video processing silicon IPs and video compliance streams, has announced that its D310 AV1 Decoder silicon IP now supports 12-bit sample size and 4:4:4 chroma sub-sampling. Emerging applications such as cloud-gaming, automotive and screen mirroring will directedly benefit from these new features to provide a superior
This blog post was originally published at Tryolabs’ website. It is reprinted here with the permission of Tryolabs. At Tryolabs, we are passionate about building computer vision solutions that have real-world impact. Very often, this means integrating them to run on edge devices. In our journey to master AI on the edge, we have developed
“A Novel Packet-based Accelerator for Resource-Constrained Edge Devices,” a Presentation from Expedera
Sharad Chole, Chief Scientist and Co-founder of Expedera, presents the “Novel Packet-based Accelerator for Resource-Constrained Edge Devices” tutorial at the May 2022 Embedded Vision Summit. As AI applications move from cloud platforms into edge devices, processor designers are increasingly incorporating hardware accelerators for real-time AI processing. These accelerators must meet… “A Novel Packet-based Accelerator for
“NeuPro-M: A Highly Scalable, Heterogeneous and Secure Processor for High-performance AI/ML in Smart Edge Devices,” a Presentation from CEVA
Yair Siegel, Senior Director of Business Development at CEVA, presents the “NeuPro-M: A Highly Scalable, Heterogeneous and Secure Processor for High-performance AI/ML in Smart Edge Devices” tutorial at the May 2022 Embedded Vision Summit. AI capabilities have become an integral part of many products across a wide range of markets… “NeuPro-M: A Highly Scalable, Heterogeneous
This market research report was originally published at Woodside Capital Partners’ website. It is reprinted here with the permission of Woodside Capital Partners. Woodside Capital Partners (WCP) is pleased to share our Industry Report on the AI Semiconductor Market 2021, authored by Managing Director Shusaku Sumida. This report covers the following aspects of the Semiconductor
Free Webinar Explores How to Use Edge Impulse’s FOMO Algorithm on Resource-constrained Microcontrollers
On August 30, 2022 at 9 am PT (noon ET), Jenny Plunkett, Senior Developer Relations Engineer at Edge Impulse, and Armaghan Ebrahimi, Partner Solutions Engineer at Sony Electronics Professional Solutions Americas, will present the free hour webinar “Edge Impulse’s FOMO Technology and Sony’s Computer Vision Platform: A Compelling Combination,” organized by the Edge AI and
EdgeCortix Collaborates with Renesas to Deliver Enhanced Feature-Rich Compiler for the Renesas DRP-AI AI-Accelerator
TOKYO, July 12, 2022 /PRNewswire/ — EdgeCortix® Inc., an innovative fabless semiconductor design company with a software first approach, focused on delivering class-leading compute efficiency and latency for edge artificial intelligence (AI) inference, announced today a collaboration with Renesas Electronics Corporation (Renesas). Through this collaboration, EdgeCortix has taken its industry leading heterogeneous platform-based compiler framework
This blog post was originally published at Syntiant’s website. It is reprinted here with the permission of Syntiant. In our last blog post, we gave insight for modelers into the vital performance statistics of edge silicon. In this blog post, we will show how modelers can leverage the true power and throughput performance of edge
“How to Enhance Edge AI Vision with the Katana SoC Using Multi-Modal Sensing,” a Presentation from Synaptics
Shay Kamin Braun, Director of Low-power AI Marketing at Synaptics, presents the “How to Enhance Edge AI Vision with the Katana SoC Using Multi-Modal Sensing” tutorial at the May 2022 Embedded Vision Summit. Machine learning (ML)-based vision edge AI has wide applicability across a variety of segments, including consumer electronics,… “How to Enhance Edge AI
This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm. To run neural networks efficiently at the edge on mobile, IoT, and other embedded devices, developers strive to optimize their machine learning (ML) models’ size and complexity while taking advantage of hardware acceleration for inference. For these
Erik Kokalj, Director of Applications Engineering at Luxonis, presents the “Enable Spatial Understanding for Embedded/Edge Devices with DepthAI” tutorial at the May 2022 Embedded Vision Summit. Many systems need to understand not only what objects are nearby, but also where those objects are in the physical world. This is “spatial… “Enable Spatial Understanding for Embedded/Edge
Laguna Hills, Calif. – June 14, 2022 – BrainChip Holdings Ltd (ASX: BRN, OTCQX: BRCHF, ADR: BCHPY), the world’s first commercial producer of neuromorphic AI IP, and Prophesee, the inventor of the world’s most advanced neuromorphic vision systems, today announced a technology partnership that delivers next-generation platforms for OEMs looking to integrate event-based vision systems
Hussein Osman, Segment Marketing Director at Lattice Semiconductor, presents the “Accelerate Tomorrow’s Models with Lattice FPGAs” tutorial at the May 2022 Embedded Vision Summit. Deep learning models are advancing at a dizzying pace, creating difficult dilemmas for system developers. When you begin developing an edge AI system, you select the… “Accelerate Tomorrow’s Models with Lattice
This blog post was originally published at Lanner Electronics’ website. It is reprinted here with the permission of Lanner Electronics. In dense urban environments where traffic flow is heavy, keeping up with traffic control and maintaining road safety at intersections with the heaviest traffic is often tricky as vehicles and pedestrians’ behaviors are erratic and
Avi Baum, CTO and Co-founder of Hailo, presents the “Taking Intelligent Video Analytics to the Next Level,” tutorial at the May 2022 Embedded Vision Summit. In this presentation, Baum examines how powerful edge AI improves the value proposition of video analytics across market segments such as physical security, access control,… “Taking Intelligent Video Analytics to
e-con Systems Launches Multi-camera Solution for NVIDIA Jetson AGX Orin Based on Sony STARVIS IMX485
NVIDIA Jetson AGX Orin | Sony STARVIS IMX485 | 1/1.2” sensor | Large pixel size | 4K resolution | Multi-camera solution San Jose and Chennai (June 21, 2022) – e-con Systems, a leading embedded OEM camera company and an NVIDIA Elite Partner, today launched the e-CAM82_CUOAGX – a 4K ultra-low-light camera powered by the NVIDIA
This blog post was originally published at Syntiant’s website. It is reprinted here with the permission of Syntiant. Academic machine learning requires extensive toolsets for model training and evaluation, but the vast majority of academic work is more grounded in model capacity than looking at the energy cost of deploying those models. This puts modelers