System on Chip versus Discrete AI Chipset: Bringing Artificial Intelligence Beyond Mobile Devices

Subscribe To Download This Insight

4Q 2019 | IN-5694

Moving Artificial Intelligence (AI) to the edge mitigates potential vulnerability and risks such as unreliable connectivity, data loss, and delayed responses. However, the jury is still out there in terms of the type of AI chipset implementation. This insight reviews the pros and cons of a System-on-Chip (SoC) approach versus a discrete AI chipset approach and evaluates the market opportunities for both.

Registered users can unlock up to five pieces of premium content each month.

Log in or register to unlock this Insight.


Qualcomm Beefed up Its Qualcomm AI Engine


Qualcomm has recently upped its game in the Artificial Intelligence (AI) domain through fifth generation Qualcomm AI Engine, which is composed of Qualcomm Kyro Central Processing Unit (CPU), Adreno Graphics Processing Unit (GPU), and, most importantly, Hexagon Tensor Accelerator (HTA). Integrated in Qualcomm’s latest 5G chipsets, namely Snapdragon 865 and 765, Qualcomm AI Engine will be making their way into majority of the premium and mid-tier smartphones, bringing with it an AI computation workload of 15 Trillion Operations Per Second (TOPS), INT8 and INT16 optimization and enhanced lossless compression for deep learning bandwidth. Developers can take advantage of either CPU, GPU, or HTA in the AI Engine to carry out their AI workloads.

In addition to the hardware performance, Qualcomm also introduces Qualcomm Neural Processing Software Development Kit (SDK) and Hexagon NN Direct to facilitate the quantization and deployment of AI models directly on Hexagon 698 Processor. This enables smartphone app developers to leverage HTA directly for their AI applications, such as facial and landmark detection, natural language translation, and facial feature replacement, bringing better resource optimization and efficiency.

While Qualcomm continues to optimize its AI capabilities on its System-on-Chip (SoC), the company is facing two significant challenges, namely an intensifying competitive landscape and an increasingly plateauing global smartphone market. In the past few years, smartphone giants such as Apple, Huawei, and Samsung have launched their own SoC with integrated AI capabilities, with other chipset vendors, MediaTek and Unisoc, following suit. A quickly saturated premium-tier smartphone market has also prompted Qualcomm to move its AI capabilities into mid-tier smartphones, but such decision is unlikely to arrest the slowdown of global smartphone adoption rate.

The Strengths and Challenges for SoC


As such, it became paramount for mobile SoC vendors to expand into other markets. Huawei and MediaTek incorporate their SoCs into Internet of Things (IoT) gateways and home entertainment, and Xilinx finds its niche in machine vision through its Versal ACAP SoC. Qualcomm moves first into automotive then into robotics, Personal Computers (PCs) and Extended Reality (XR). This move is in line with NVIDIA, the largest AI chipset vendor in the robotics space. Based on its GPU architecture, NVIDIA Jetson AGX platform is a high performance SoC that features GPU, ARM-based CPU, deep learning accelerators and image signal processors. Not necessarily a direct competitor, Qualcomm RB3 Platform, which features the company’s previous generation premium chipset Snapdragon 845, offers high speed cellular connectivity which NVIDIA platform lacks. The newly-launched Snapdragon 865 is likely to find its way into the next generation robotics platform, bringing both 5G and AI capabilities to mobile robots. In PC, Qualcomm 8cx compute platform features an AI Engine that has 5 TOPS of performance which supports better image quality, enhanced security and better voice recognition in PC devices.

Beyond these sectors, however, the opportunity for SoC dries up quickly. For many edge-based AI inference workloads in automotive and industrial manufacturing, SoCs are deemed too power hungry, complicated in architecture, and costly for the AI computational workloads that they are responsible for. Edge devices in these sectors place high premium on good balance between performance, power consumption, and cost. This leads to the adoption of a discrete edge AI chipset that is uncoupled with the SoC. Scalable AI chipset architecture may also be preferred as more than one chipset may be deployed in parallel to handle higher AI workloads, without the additional cost of CPU and GPU. A discrete AI chipset also favors future upgradability with minimal hardware and software processes redesign.

Diversity is Key


According to ABI Research’s Artificial Intelligence and Machine Learning (MD-AIML-104) Market Data, the market size for Application Specific Integrated Circuit (ASIC) responsible for edge inference is expected to reach US$4.3 billion by 2024. This include all SoCs with integrated AI chipset, discrete ASIC, and hardware accelerators. While 90% of the market is dominated by mobile devices, pockets of opportunities have emerged in automotive, industrial manufacturing, oil and gas, transportation, and logistics and retail, which prefers discrete AI chipsets over power-hungry and costly SoC solutions. Even in the smartphone industry, vendors like Google and LG have either introduced or are planning to integrate discrete AI processors into their smartphones.

Therefore, even though the debate between SoC and discrete AI chipset is not settled yet, it is a foregone conclusion that a successful AI chipset company needs to have both SoC and discrete AI chipset to compete in all these fronts. To address these aforementioned use cases, Qualcomm has decided to bring its Cloud AI 100 chipset to edge applications. Originally designed for inference workloads in the data center, Qualcomm now intends to introduce Cloud AI 100 in 5G network infrastructure and Advanced Driver-Assistance Systems (ADAS) in autonomous driving and edge computing. Nonetheless, there is no doubt that Qualcomm will be facing fierce competition in these domains. Many of these sectors are leaning toward Intel x86 compute architecture, and Intel will have an upper hand with its oneAPI platform that unifies heterogenous compute architectures offered by the company. At the same time, there are several dozens of AI startups that Qualcomm will be competing against in these sectors, such as Hailo, Horizon Robotics, Gyrfalcon Technology, and Greenwaves Technologies.