The On-device AI Market for IoT Applications 1st Edition

Internet of Things (IoT) is continually evolving and expanding into new domains. Among the mostrecent developments is the integration of artificial intelligence (AI) capabilities directly onto IoTdevices to unlock a new generation of applications. Devices that integrate AI have numerousbenefits over traditional rule-based or manually programmed methods, particularly forapplications that require object detection, speech recognition, predictive maintenance, anomalydetection, dynamic resource optimisation and autonomous decision-making. Running AIalgorithms directly on the device – known as edge AI or on-device AI – brings numerousadvantages, such as real-time responsiveness, reduced data transfer, enhanced privacy andimproved resilience. While cloud processing remains effective for many IoT use cases, a growingnumber of emerging use cases now demand the capabilities of on-device AI.

The market for on-device AI solutions is characterised by a high degree of heterogeneity in bothtechnologies and applications, in contrast to cloud-based AI where the hardware is typicallydesigned around predefined use cases and centralised infrastructure. Embedded AI processingcan be architected in numerous ways depending on the end use case, and it can be integratedinto an almost limitless range of devices across consumer, industrial and automotive domains.This leads to a differentiated market landscape, with unique design constraints, performancerequirements and optimisation strategies. However, the overarching objective is typically thesame for all vendors – to achieve the highest possible performance per watt for the intended usecase.

Berg Insight has identified 40 key companies that shape the on-device AI landscape. The market can broadly be divided into two layers. The first encompasses hardware categories such as AIsystem-on-chips (SoCs) or system-on-modules (SoMs), AI accelerators and AI microcontrollerunits (MCUs), each optimised for different levels of performance, power efficiency andintegration. AI SoCs typically integrate components such as general-purpose and specialised AIcompute cores, on-chip memory and connectivity on a single chip, while SoMs extend thisdesign by including external system memory, storage and interface components on a largerboard, targeting more advanced use cases. AI accelerators are specialised chips or modulesdesigned to enhance AI inference efficiency in existing systems, typically working alongside aseparate host processor in embedded applications. AI MCUs serve lower-power devices bybringing neural network capabilities to sensors, wearables and IoT endpoints where energyefficiency and cost are most critical. The second layer consists of on-device AI platforms thatcombine hardware, software and developer tools to simplify model deployment andoptimisation.

Over the past decade, the on-device AI market has been driven primarily by traditional machinelearning use cases such as computer vision and anomaly detection, which have seen steadyannual growth of around the 10 percent range. In recent years, the market has reached aninflexion point as emerging technologies and applications in generative AI, robotics andautonomous driving have opened up new dimensions of growth. These developments areexpected to accelerate market growth and give rise to entirely new use cases and productcategories. Berg Insight estimates that the revenue generated by on-device AI solutions reachedUS$ 10.1 billion in 2024, an increase of around 22 percent from 2023. This figure includes AISoCs/SoMs, AI accelerators, AI MCUs and specialised on-device AI software and platforms, butexcludes revenues generated by non-IoT applications such as smartphones, tablets andpersonal computers. The market is expected to grow to US$ 30.6 billion in 2029, representing acompound annual growth rate (CAGR) of 25 percent.