Processors for Embedded Vision
THIS TECHNOLOGY CATEGORY INCLUDES ANY DEVICE THAT EXECUTES VISION ALGORITHMS OR VISION SYSTEM CONTROL SOFTWARE
This technology category includes any device that executes vision algorithms or vision system control software. The following diagram shows a typical computer vision pipeline; processors are often optimized for the compute-intensive portions of the software workload.
The following examples represent distinctly different types of processor architectures for embedded vision, and each has advantages and trade-offs that depend on the workload. For this reason, many devices combine multiple processor types into a heterogeneous computing environment, often integrated into a single semiconductor component. In addition, a processor can be accelerated by dedicated hardware that improves performance on computer vision algorithms.
While computer vision algorithms can run on most general-purpose CPUs, desktop processors may not meet the design constraints of some systems. However, x86 processors and system boards can leverage the PC infrastructure for low-cost hardware and broadly-supported software development tools. Several Alliance Member companies also offer devices that integrate a RISC CPU core. A general-purpose CPU is best suited for heuristics, complex decision-making, network access, user interface, storage management, and overall control. A general purpose CPU may be paired with a vision-specialized device for better performance on pixel-level processing.
Graphics Processing Units
High-performance GPUs deliver massive amounts of parallel computing potential, and graphics processors can be used to accelerate the portions of the computer vision pipeline that perform parallel processing on pixel data. While General Purpose GPUs (GPGPUs) have primarily been used for high-performance computing (HPC), even mobile graphics processors and integrated graphics cores are gaining GPGPU capability—meeting the power constraints for a wider range of vision applications. In designs that require 3D processing in addition to embedded vision, a GPU will already be part of the system and can be used to assist a general-purpose CPU with many computer vision algorithms. Many examples exist of x86-based embedded systems with discrete GPGPUs.
Digital Signal Processors
DSPs are very efficient for processing streaming data, since the bus and memory architecture are optimized to process high-speed data as it traverses the system. This architecture makes DSPs an excellent solution for processing image pixel data as it streams from a sensor source. Many DSPs for vision have been enhanced with coprocessors that are optimized for processing video inputs and accelerating computer vision algorithms. The specialized nature of DSPs makes these devices inefficient for processing general-purpose software workloads, so DSPs are usually paired with a RISC processor to create a heterogeneous computing environment that offers the best of both worlds.
Field Programmable Gate Arrays (FPGAs)
Instead of incurring the high cost and long lead-times for a custom ASIC to accelerate computer vision systems, designers can implement an FPGA to offer a reprogrammable solution for hardware acceleration. With millions of programmable gates, hundreds of I/O pins, and compute performance in the trillions of multiply-accumulates/sec (tera-MACs), high-end FPGAs offer the potential for highest performance in a vision system. Unlike a CPU, which has to time-slice or multi-thread tasks as they compete for compute resources, an FPGA has the advantage of being able to simultaneously accelerate multiple portions of a computer vision pipeline. Since the parallel nature of FPGAs offers so much advantage for accelerating computer vision, many of the algorithms are available as optimized libraries from semiconductor vendors. These computer vision libraries also include preconfigured interface blocks for connecting to other vision devices, such as IP cameras.
Vision-Specific Processors and Cores
Application-specific standard products (ASSPs) are specialized, highly integrated chips tailored for specific applications or application sets. ASSPs may incorporate a CPU, or use a separate CPU chip. By virtue of their specialization, ASSPs for vision processing typically deliver superior cost- and energy-efficiency compared with other types of processing solutions. Among other techniques, ASSPs deliver this efficiency through the use of specialized coprocessors and accelerators. And, because ASSPs are by definition focused on a specific application, they are usually provided with extensive associated software. This same specialization, however, means that an ASSP designed for vision is typically not suitable for other applications. ASSPs’ unique architectures can also make programming them more difficult than with other kinds of processors; some ASSPs are not user-programmable.
Independent consultant Oliver Gunasekara and Shweta Shrivastava, chief product officer at Nauto, present the “Lessons From the Start-up Trenches” tutorial at the September 2020 Embedded Vision Summit. This interview with Oliver Gunasekara, founder and CEO of NGCodec, explores lessons learned during the journey of NGCodec—a video codec developer—from its founding… “Lessons From the Start-up Trenches,”
OUTLINE: Market forecasts: Yole Développement (Yole) expects the server computing market to reach US$33 billion in 2025. AI and HPC dedicated products combined will reach 29Munits and US$8 billion in 2025. Configurable SoC dedicated to AI inference will be more used following AI’s hype. The CPU revenue will reach US$21 billion in 2025, driven by
“Safer and More Efficient Intersections with Computer Vision,” a Presentation from Cubic | GRIDSMART
Jeff Price, Vice President and General Manager at Cubic | GRIDSMART, presents the “Safer and More Efficient Intersections with Computer Vision” tutorial at the September 2020 Embedded Vision Summit. GRIDSMART is an edge computer vision system that uses omnidirectional imaging to detect and track objects through intersections to optimize traffic… “Safer and More Efficient Intersections
Chips&Media Adds Extraordinary Features in Next-Generation Video Codec IP: Introducing the WAVE6 Series
SEOUL, Jan 14, 2021 – Chips&Media, a global leading semiconductor HW IP provider that specializes in video codec, image signal processing, and computer vision, today shared its rolled-out product list for the next-generation video codec IPs, the WAVE6 series. Chips&Media is supposedly planning to boost supported architecture and features by changing from frame pipeline to
Augmenting its leading technology roadmap to redefine computing performance Accelerating successful partnerships with OS providers for next-generation computing devices Enabling Samsung and the mobile ecosystem to redefine premium performance for smartphones Leading CPU talent to join Qualcomm’s best in class innovation culture Jan 13, 2021 – SAN DIEGO – Qualcomm Incorporated (NASDAQ: QCOM) today announced
Independent AI architect Siddha Ganju presents the “Deep Learning on Mobile Devices” tutorial at the September 2020 Embedded Vision Summit. Over the last few years, convolutional neural networks (CNNs) have grown enormously in popularity, especially for computer vision. Many applications running on smartphones and wearable devices could benefit from the… “Deep Learning on Mobile Devices,”
News highlights: Arm’s next-generation mobile solution is powering the new Samsung Exynos 2100 SoC, an early adopter of the new Arm Cortex-X1 CPU in a tri-cluster, octa-core configuration, and optimized for 5nm Delivering over 30% multi-tasking performance improvement and exceptional battery life to take premium smartphone experiences to the next level Partnership highlights technology innovation
GeForce RTX 30 Series Laptop GPUs Offer Up to 2x Efficiency, 3rd Gen Max-Q Technology, Start at $999 Tuesday, January 12, 2021 – A new era of laptops begins today featuring the NVIDIA Ampere architecture, with the launch of 70+ models powered by GeForce® RTX™ 30 Series Laptop GPUs. These next-gen laptops, which start at
Powered by NVIDIA Ampere Architecture, Delivers Up to 10x the Ray-Tracing Performance of GTX 1060; Starting at $329 Tuesday, January 12, 2021 – NVIDIA today announced that it is bringing the NVIDIA Ampere architecture to millions more PC gamers with the new GeForce® RTX™ 3060 GPU. With its efficient, high-performance architecture and the second generation
AMD Ryzen Threadripper PRO Processors, designed for the most demanding professional workloads, coming to retail channel SANTA CLARA, Calif. – 01/12/2021 – Today, AMD (NASDAQ: AMD) announced the full portfolio of AMD Ryzen™ 5000 Series Mobile Processors, bringing the highly-efficient and extremely powerful “Zen 3” core architecture to the laptop market. New AMD Ryzen 5000
CEVA Extends its Leadership in High Performance Scalable Sensor Hub DSPs with 2nd Generation SensPro Family
SensPro2™ delivers up to 6X DSP performance improvement in computer vision, 2X AI inferencing improvement and 20% lower power compared to 1st generation SensPro, at the same process node New, low power, entry-level SensPro2 DSPs deliver up to 10X performance improvement versus CEVA-BX2 DSP for AI networks used for voice assistants, natural language processing and
Delivering the Best Business, Education, Mobile and Gaming Computing Platforms for More Than 500 New PC Designs Coming in 2021 NEWS HIGHLIGHTS At its CES news conference, Intel highlighted how it is driving technology leadership with the introduction of more than 50 processors, resulting in more than 500 new designs for laptops and desktops coming
Progress Includes Automated Crowdsourced Mapping, New Lidar SoC, Software-Defined Radar and AV Test Vehicles in Four New Countries NEWS HIGHLIGHTS Automated, worldwide autonomous vehicle (AV) mapping capability allows Mobileye to expand its AV test fleets; new vehicles expected in Detroit, Tokyo, Shanghai, Paris and (pending regulation) New York City early this year. Intel brings its
Ambarella Introduces CV5 High Performance AI Vision Processor for Single 8K and Multi-Imager AI Cameras
Ambarella’s new 5 nm CV5 AI vision CVflow® processor sets a new standard for power efficiency with 8K video recording in under 2 watts SANTA CLARA, Calif.—January 11, 2021—Ambarella, Inc. (Nasdaq: AMBA), an AI vision silicon company, today unveiled CV5, an artificial intelligence (AI) vision processor capable of recording 8K video or four 4K video streams.
On February 9, 2021 at 9 am PT (noon ET), the Edge AI and Vision Alliance’s Founder, Jeff Bier, and Vice President of Special Projects, Phil Lapsley, will present the free hour webinar “Adding Visual AI or Computer Vision to Your Embedded System: Key Things Every Engineering Manager Should Know,” in partnership with Aspencore. Here’s
New June Oven Designed with MediaTek’s i500 AIoT Platform Delivers a Smart Cooking Experience to Consumers
MediaTek’s i500 brings powerful computer vision, advanced multimedia and fast Wi-Fi to the third-generation June Oven HSINCHU, Taiwan – December 14, 2020 –MediaTek today announced that the company’s i500 AIoT chipset platform is the main processor inside the new June Oven, a twelve-in-one countertop convection oven. MediaTek’s i500 has a dedicated AI Processing Unit (APU)
“Video Activity Recognition with Limited Data for Smart Home Applications,” a Presentation from Comcast
Hongcheng Wang, Director of Technical Research at Comcast, presents the “Video Activity Recognition with Limited Data for Smart Home Applications” tutorial at the September 2020 Embedded Vision Summit. Comcast’s Xfinity Home connects millions of home smart cameras and IoT devices to improve its customers’ safety and security. The company’s teams use computer vision and deep
Qualcomm Expands 5G Capabilities to Mobile Devices Powered by New Snapdragon 480 5G Mobile Platform, a First in the Snapdragon 4-Series
Snapdragon 480 Accelerates Global 5G Commercialization and Delivers Series-Defying Features to Mass-Volume Smartphone Segment Jan 4, 2021–SAN DIEGO–Qualcomm Technologies, Inc. announced the Qualcomm® Snapdragon™ 480 5G Mobile Platform, the first 4-series mobile platform equipped with 5G. Snapdragon 480 continues to help drive further proliferation of 5G, allowing users access to truly global 5G connectivity and
Commercial partnership under new DARPA Toolbox initiative provides DARPA researchers with access to CEVA’s portfolio of wireless connectivity and smart sensing IPs Rockville, MD., – January 05, 2021 – CEVA, Inc. (NASDAQ: CEVA), the leading licensor of wireless connectivity and smart sensing technologies, today announced an open licensing agreement with the U.S. Defense Advanced Research Projects
“Deploying AI Software to Embedded Devices Using Open Standards,” a Presentation from Codeplay Software
Andrew Richards, Co-founder and CEO of Codeplay Software, presents the “Deploying AI Software to Embedded Devices Using Open Standards” tutorial at the September 2020 Embedded Vision Summit. AI software developers need to deploy diverse classes of algorithms in embedded devices, including deep learning, machine vision and sensor fusion. Adapting these… “Deploying AI Software to Embedded