An Easier Way to Build Powerful Edge AI Applications

Designing a simple sensor-based device for the Internet of Things (IoT) isn’t the hardest thing you’ll ever do, but building a connected device able to handle machine learning (ML) at the edge is another story altogether. A processor family, development board, and associated software from NXP Semiconductors are designed to overcome the key challenges in functionality, performance, and development, enabling faster deployment of sophisticated edge artificial intelligence (AI) capabilities in industrial and IoT applications.

Designers have already begun to take advantage of edge AI solutions able to perform ML inference on low-power devices without relying on cloud-based resources. Functions such as wake word detection, sensor data pattern analysis, and basic object detection typically can be handled by energy-efficient processors running ML models built with model-optimizing tools and frameworks. Problems arise when you attempt to stretch processor resources to tackle more complex problems, particularly those requiring a real-time or near-real-time response.

How a multicore processor can significantly accelerate ML inference

NXP Semiconductors addresses the functionality and performance challenges of these emerging real-time edge AI designs with its i.MX 93 applications processor family (Figure 1).

Figure 1: The i.MX 93 applications processors provide the foundation for edge AI designs with their combination of processing resources, security system, memory, and comprehensive set of clocks, timers, connectivity options, and interfaces. (Image source: NXP Semiconductors)

These devices integrate a rich set of capabilities, including multimedia, memory, interfaces, and connectivity options, and feature a compelling combination of processing resources:

• Up to two high-performance Arm Cortex-A55 application processor cores for Linux-based application processing
• An ultra-low-power Arm Cortex-M33 platform for low-latency real-time control processing
• An Arm Ethos-U65 microNPU neural processing unit (NPU) for efficient ML inference execution
• NXP’s integrated EdgeLock secure enclave (ESE), providing a root of trust for secure boot and key management, on-the-fly cryptographic functions, and other capabilities needed to protect edge applications

Using the capabilities of these processors, you can partition large edge AI applications into manageable pieces: rather than stealing cycles from application code running on the Cortex-A55 cores, the NPU offloads the execution of compute-intensive neural network algorithms. Separately, the Cortex-M33 remains dedicated to latency-sensitive tasks, such as sensor data acquisition or process control, while the ESE maintains the security of the system, software code, and critical data. As you’ll see below, the NPU’s ability to offload ML inference from the Cortex-A55 cores is a key enabler for edge AI applications that require near real-time responsiveness.

How a hardware board and software speed application development

Although processor functionality and performance are key factors, effective development of edge AI applications depends on the ability to quickly get up to speed on processor capabilities and rapidly build effective software. The combination of NXP’s FRDM-IMX93 development board (Figure 2) and associated software development resources gives you all you need to begin creating applications.

Figure 2: The FRDM-IMX93 development board integrates an i.MX 93 applications processor and multiple support chips to provide a complete system for development. (Image source: NXP Semiconductors)

The FRDM-IMX93 board is designed to simplify the development of applications using the i.MX 93 processor family. Measuring only 105 × 65 millimeters (mm), the board includes two gigabytes (Gbytes) of low-power double data rate 4X (LPDDR4X) dynamic random access memory (DRAM), a 32 Gbyte embedded multimedia card 5.1 (eMMC5.1), a power management integrated circuit (PMIC), a radio transceiver module providing Wi-Fi 6, Bluetooth 5.4, and an 802.15.4 wireless personal network, and other required support chips. The board’s headers, connectors, and debug interface let you easily add expansion boards to debug your system.

The board offers several configuration options, but it is ready to use in its default mode. After you connect a display and mouse and provide power to the board using the included USB cable, the board boots from the eMMC, which comes pre-flashed with an NXP Linux binary demo image.

Pre-built applications accelerate edge AI deployment

The true strength of the FRDM-IMX 93 development board lies in its accompanying software and associated tools. Included in the default demo image, NXP’s GoPoint for i.MX Applications Processors software package provides a set of example applications accessible through the GoPoint graphical user interface (GUI). When you launch an application, GoPoint walks you through any required configuration steps, such as specifying a camera video source.

Perhaps just as interesting to many of us, NXP’s example applications repository contains the complete Python source code for each GoPoint application. Additionally, the documentation for each application includes a description of the software architecture, the ML models used, and the launch setup. For example, the documentation for the GoPoint i.MX 93 low power machine learning application describes the execution flow between the Cortex-A55 and Cortex-M33 cores, showing how the Cortex-M33 performs the heavy lifting of wake word detection, until finally waking up Linux on the Cortex-A33 to notify the user.

NXP’s GoPoint driver monitoring system (DMS) application demonstrates another important concept: how to interweave the execution of multiple models to detect several different states in a more complex application. After invoking this application, select the camera source and inference processor (application processor or NPU, for example) and then run it. As it executes, the current status of the camera subject (driver) is displayed, indicating whether the subject is distracted, drowsy, yawning, smoking, or using a cell phone (Figure 3).

Figure 3: Running on an i.MX 93 application processor, a sample GoPoint application performs inference on multiple ML models to detect different driver states with only milliseconds of latency. (Image source: NXP Semiconductors)

A compelling piece of information in each application's repository is the performance data, which shows the NPU’s ability to enable more effective edge AI applications. In the DMS application, for example, NXP found that running CPU-based smoking/calling detection inference with one thread takes 407.963 milliseconds (ms). Running with two threads drops inference time to 303.159 ms, but the real payoff lies in employing the NPU; inference on the NPU slashes the time to only 23.008 ms.

Conclusion

The ability of a low-power i.MX 93 application processor to perform substantial inference tasks with near real-time latency opens numerous possibilities for edge AI solutions in industrial and IoT applications. By using an associated development board and software, you can quickly and effectively build those applications.