2021 Is a Promising Year for Ultra Low Power Machine Learning

At CES 2021, devices powered by Artificial Intelligence (AI) continue to be the main attraction of the annual tech show. From L’Oreal’s smart lipsticks to Samsung’s butler robots, consumer devices have shown their capabilities to sense the environment and to react to stimuli in near real time. More importantly, the innovation has trickled down to the ever-improving true wireless headsets and earbuds where ML-based adaptive noise cancellation, app-based equalization, and noise-reduction customization are increasingly common and popular.

In the past, these devices would gather information from various sensors—such as acoustic, gyroscope, light, pressure, and temperature sensors—and process them in a centralized manner at a gateway or in data center. The centralized analytics model would in turn generate a response based on the inputs. However, this process would incur throughput and latency penalty as the response would need some time and ideal connectivity to be sent from the gateway or the data center to the edge device, in turn causing a lagged reaction and poor user experience.

The emergence of TinyML has changed this. By programming ML models to operate in a hardware environment with an extreme power and space constraint, particularly at the sensor level, devices can filter through all the relevant input and generate a timely response without needing constant connectivity to a gateway or a data center. This dramatically reduces the latency, creating a much better response and user experience.

The consumer device market has certainly taken note of the importance of TinyML. A slew of recent announcements demonstrates high level of activities in this space.

• CEVA: CEVA released the SensPro 2 with seven self-contained cores. Out of the seven cores, the SP100 and SP50 offer the smallest die size and deliver a significant improvement for speech recognition neural network compared with the CEVA-BX2 scalar digital signal processor and are ideal for audio AI workloads, such as conversational assistants, sound analytics, and natural language processing.
• Syntiant: A TinyML startup based in Irvine, California, the company has recently announced its 10 millionth shipment of its previous generation chipsets. At the same time, the company has also launched a new chipset, NDP120. Instead of focusing on a single application, NDP120 can now support multiple deep-learning models simultaneously at under one milliwatt, including echo cancellation, beamforming, noise suppression, speech enhancement, speaker identification, keyword spotting, multiple wake words, event detection, and local commands recognition.
• Bosch: Bosch Sensortec, a subsidiary of Robert Bosch GmbH, launched the BHI260AP, a self-learning motion sensor dedicated for wearables and hearables. This smart sensor is equipped with a six-axis inertial measurement unit and a microcontroller with embedded ML models. The sensor can be used in applications that involve orientation tracking, position tracking, and swimming motion. More importantly, the BHI260AP is customizable and programmed via a software development kit.
• Nordic Semiconductor and Edge Impulse: The Bluetooth chipset manufacturer has partnered with Edge Impulse to launch a series of Bluetooth Low Energy development boards that incorporate the Edge Impulse’s Edge Optimized Neural (EON) compiler that optimizes computer processing and memory use for TinyML applications. Nordic Semiconductor is targeting applications such as perimeter security, predictive and preventative maintenance, and utilities. As of December 2020, the Edge Impulse has forged similar partnership with Himax Technologies, a TinyML chipset supplier for computer vision.

According to ABI Research’s latest global market data on AI and ML, more than 800 million consumer devices will be shipped with a TinyML chipset in 2025, predominantly in the smartphone, wearable, and smart home sectors. New domains such as robotics and augmented reality glasses have shown a lot of promise, too.

Aside from consumer devices, TinyML chipsets are expected to make their ways into industrial and manufacturing, healthcare, smart cities, and transport and logistics. In 2025, ABI Research estimates a total of 1.2 billion edge devices to be shipped with a TinyML chipset. Vendors need more than cost-efficient hardware to achieve this market size; they need to create user-friendly development platforms and control interfaces to entice the developer community to quickly embrace this innovation. Traditionally, developing embedded ML applications required a high level of mathematical and computer programming expertise along with knowledge of professional software. Nowadays, the popularization of high-level AI frameworks (such as TensorFlow Lite for Microcontroller), the emergence of software compilers (such as EON from Edge Impulse), and edge AI onboarding and deployment platforms like Laneyes will greatly facilitate the deployment process. Nonetheless, the wide range of applications at the edge means that heavy fragmentation is bound to happen. More partnerships and standardization efforts need to materialize to ensure wide-scale rollout.