AI Chips in Smartphones Key to Boosting Machine Vision Adoption and Applications

Machine vision refers to a number of computer- and embedded-vision technologies, where visual systems scan, analyze, and interpret images or video. It encompasses a variety of technologies and software, hardware, and integrated systems. The increasing number of smartphones with more advanced features, such as Huawei’s P20 Pro, Apple’s iPhone X, and Google’s Pixel 3, are helping to drive this machine vision market, making it possible for the technology to be used directly on the device. Machine vision offers a range of applications in smartphones; some are experiencing an increase in application adoption due to the enablement of advanced cameras, image sensors, 3D sensing, security, and Artificial Intelligence (AI) chips.

The AI chips, such as Apple’s A12 Bionic and Huawei’s HiSilicon Kirin 980, are helping to bring machine vision to the device and are offering the required processing power to run the technology. For devices without an AI chip, machine vision is still possible, provided that they have a camera and connectivity to the cloud. These devices can upload images and video to the cloud; this allows the AI analysis to occur in the cloud and therefore offers a much wider market of devices to utilize machine vision. It is, however, a slower process because it is not done instantaneously; it is often less secure as it involves sending the data elsewhere; and it requires connectivity to the cloud. This may not be as much of an issue moving forward when 5G is introduced.

In most cases, embedded machine vision in smartphones is being used for image recognition and provides the user with more information about an object. For example, a photo of a building could be used to provide information about its history; a photo of a piece of clothing could be used to show the user where it (or a similar item) can be bought; and a photo of a flower could be used to provide information about what type of flower it is. The machine vision technology, powered by the AI chip, uses in-built image recognition algorithms to identify the object within the image. For devices without a chip, this process is done via the cloud, offering a slower, more limited form of machine vision.

However, there are now a number of smartphones that offer on-device machine vision capabilities, and this is possible through the use of AI chips and cameras. For example, Huawei’s P20 Pro has a HiSilicon Kirin 970 chip that incorporates a neural Network Processing Unit (NPU) for AI acceleration. It is also used to recognize and identify faces, scenes, and objects to provide personalized recommendations and allows the camera to be automatically set up to accurately capture the image and organize photo albums. It is further aided by the use of Leica’s three-lens camera system that provides high quality images. Similarly, Apple’s Bionic A11 in its iPhone X smartphone includes a neural network for AI processes that recognize faces and objects in an image. It is further aided by the TrueDepth camera that provides the machine vision system with 3D images, and this offers more points of recognition. Companies such as Qualcomm and MediaTek also offer AI mobile platforms or chips to smartphone makers such as Asus, LG, Samsung, Sony, and Vivo so that they can provide AI processing abilities.

A number of image recognition apps that allow devices without an AI chip to offer machine vision capabilities via the cloud and the device’s camera are also available. Google’s Lens image recognition app allows users to search for information about an object, text, product, landmark, plant, retail product, or animal using their smartphone’s camera. It can be used by devices with Android 6.0 and higher and in iOS devices with Google Photos, provided that Internet connectivity and a camera are available. Samsung’s Bixby Vision image recognition applied to the cameras on the Galaxy S8 allows users to translate foreign languages, learn more about an object or building, and use augmented reality functions to try the products before buying them. In addition, Amazon’s Rekognition application programming interface allows companies to add image and video analytics that can recognize objects, people, text, scenes, and activities and that remove inappropriate content into smartphone applications.

There are a number of other consumer, commercial, and industrial applications that are either currently in use or being investigated for smartphone machine vision, including:

• Agriculture: Machine vision aids farmers with the monitoring of crops using custom devices to scan a plant for moisture, nutrient, and chlorophyll levels to determine the effects of chemicals and identify disease. This could be done via a smartphone, preventing the farmer from needing a separate device and offering a lightweight, portable option.
• Healthcare:Smartphones are being used to provide medical professionals with detailed wound information by using the device’s camera to take an image of the area and by using machine vision to provide information such as wound size and depth as well as tissue analytics. This can help determine the correct course of treatment without requiring a hands-on examination.
• Industry: Machine vision in the industry is generally done via specific, bulky stationary systems to aid with the maintenance and upkeep of machines as well as for stock keeping. Smartphones are increasingly replacing these systems, allowing workers to use the smartphone’s camera to provide the machine vision process.
• Sports: Machine vision in smartphones is being used to analyze athletes’ movements in sporting activities. The technology recognizes the shape of an individual, tracks their movement, and provides an analysis of how to improve their style to prevent future injuries.

Using smartphones in an industrial setting over permanently installed hardware for machine vision has the advantage of being easier to replace and being able to respond faster to different requirements. It also allows for more mobility by offering the user the freedom to move around and to use the smartphone for other purposes, thereby reducing the number of required devices.

Machine vision in smartphones is still in its early stages, with most devices relying on the cloud for the technology. While this is a viable method for machine vision processes, it does involve more time and requires connectivity to the cloud. With on-device machine vision, the process is immediate and removes the reliance on the network for communications. Currently, few devices have an AI chip. To move machine vision onto the device, a greater number of smartphone vendors should consider adding an AI chip to their devices. This will help to speed up the machine vision process, remove the reliability on the cloud, and keep the user’s data more secure.

Machine vision is most commonly being used in smartphones to provide users with more information about what is in their images or video or to identify a particular object or individual. This improves the overall user experience; machine vision learns over time what the user captures with their camera. In the future, a greater number of smartphone machine vision applications and use cases will be developed and refined, resulting in a larger number of smartphones being shipped with embedded AI chips and an increasing number of companies offering cloud-based apps.

Developers and implementers of smartphone machine vision technology also need to ensure that they have a good understanding of the complexity of the technology that they are trying to deploy. The required computer resources must also be available or obtainable to ensure that the device can power the machine vision technology. Any company considering the implementation of machine vision technology in smartphones should look into partnering with or acquiring a machine vision supplier to ensure that the technology functions seamlessly with the device.

Another important consideration when deploying machine vision technology to a smartphone is privacy. For example, in Europe, companies must remain General Data Protection Regulation (GDPR) compliant, limiting the application of some forms of machine vision, such as facial recognition. Those developing and purchasing machine vision systems need to maintain awareness of the changing legal landscape related to privacy in this area, as other countries may enact such legislation.

Ensuring that accurate data is acquired is also important. Large, high-quality data sets are required to train machine vision models. This is particularly important with aspects such as facial recognition—where the algorithms will be defined by the available demographics and the country—and can be difficult to deploy in other countries. The problem of bias in data sets is something that companies must be aware of to limit the skewing of results in a model. Companies looking into machine vision must ensure that the required quantity and quality of data is available for successful machine vision applications.

The most important aspect to consider is the AI chip. Without one, a device cannot provide its own machine vision capabilities. ABI Research forecasts that machine vision in smartphones will grow; nearly 585 million smartphones with embedded machine vision will be expected to ship in 2023, and this will be aided by several major smartphone manufacturers’ addition of embedded vision products in at least one model line via an AI chip. This is unlikely to trickle down to lower-end devices; however, if it does, the forecast will increase at a faster rate. As smartphone vendors continue to compete to offer the best features—such as improved cameras, security, and AI capabilities—the number of vendors offering machine vision is set to increase.