Bench Talk

(Source: Claudine/stock.adobe.com); generated with AI

As artificial intelligence (AI) applications grow in complexity, energy efficiency becomes increasingly vital.

The environmental, economic, and operational impacts are already significant. According to Jesse Dodge, senior research scientist at the Allen Institute for AI, a single query to ChatGPT uses roughly as much electricity as lighting a single light bulb for twenty minutes.^[1] What's more, data centers are expected to see their energy demands double by 2030, owing mainly to the evolution and proliferation of AI technology.^[2]

AI computations, particularly at scale, have an immense carbon footprint. As demand for the technology grows, energy efficiency is crucial to support scaling without escalating power consumption. Moreover, optimizing the technology's energy use not only contributes to sustainability but also translates to reduced operational costs.

Energy-efficient AI is more viable in large-scale or continuous deployments, while energy use is also an important consideration in edge devices, where power is limited. Energy-efficient AI can extend battery life, minimize downtime, and improve reliability for these systems. Data center carbon and energy reporting regulations have also grown stricter in recent years.

In this blog, we will explore several innovations, including low-power AI chips, new architectures, graph neural networks, and edge AI processing to control costs, protect the environment, adhere to regulatory requirements, and support the continued evolution of AI technology.

The Rise of Low-Power AI Chips

Over the past several years, hardware manufacturers and designers have been exploring new ways to balance power efficiency with performance.

As the dominant provider of processor cores for mobile phones, Arm^® has a long history of focusing on power efficiency. Today, that obsession with efficiency is being brought to bear on AI—and not just in phones. For example, the company says its Neoverse CPU is the most power-efficient processor for cloud data centers. According to Arm, when deployed in the Amazon Web Services (AWS) Graviton systems, the Neoverse CPU achieved 60 percent better efficiency than competing architectures.^[3]

MediaTek AI processors, meanwhile, support energy-efficient AI acceleration, particularly in smartphones, smart home devices, wearable technology, and autonomous vehicles. According to the organization, its NPU offers 27 times better power efficiency than a typical CPU and 15 times better than a typical GPU.^[4] The company launched its seventh-generation NPU in 2023, supported by both a software development kit and a suite of development tools.

Lastly, Intel^® Movidius™ Vision Processing Units (VPUs) are designed explicitly for computer vision applications. The hardware combines parallel programmable compute with workload-specific AI hardware acceleration to minimize data movement.^[5] Intel offers both a VPU for PC workloads and specialized Movidius X VPUs for cameras and embedded systems.

This energy-efficient hardware can also help reduce reliance on cloud servers by enabling edge AI processing. Rather than having to manage energy-intensive data transmissions, embedded systems can leverage their own hardware. In addition to contributing to a more sustainable approach to AI, this simultaneously supports real-time operations and extends battery life for smart devices and wearables.

Emerging Technologies in Energy-Efficient AI Processing

Neural network machine learning models have long been essential in AI. Neuromorphic chips take the same approach to hardware design. Their architecture borrows heavily from neuroscience and biology,^[6] resulting in something akin to an artificial brain with greater adaptability, performance, energy efficiency, and support for parallel processing compared to traditional silicon chips.

Completely new approaches to hardware are on the horizon. One promising recent development was announced this past summer by a group of Chinese researchers: a tensor processing unit (TPU) based on carbon nanotubes.^[7] The chip’s unique architecture consists of 3,000 carbon nanotube field-effect transistors (FETs) arranged into a tightly coupled 3×3 grid of nine processing units. Each unit receives data from its upstream neighbors, calculates part of it, and passes it to its downstream neighbors.

“Based on our carbon-based tensor processor chip, we built a five-layer convolutional neural network that can perform image recognition tasks with an accuracy rate of up to 88 percent and a power consumption of only 295μW, which is the lowest power consumption among all new convolutional acceleration hardware technologies,” explains researcher Zhiyong Zhang. "The system simulation results show that the carbon-based transistor using the 180nm technology node can reach 850MHz and the energy efficiency exceeds 1TOPS/W, which shows obvious advantages over other device technologies at the same technology node.”

Graph Neural Networks for Energy Efficiency

It’s not just through new architecture that AI can become more energy efficient. A graph neural network (GNN), for example, can greatly improve the energy efficiency of electrical power grids. Compared to alternative machine learning models, GNNs are better suited to the unique structure and complexities of grid management and monitoring,^[8] providing warning of component failure and enabling more accurate forecasting.

GNNs can also assist in renewable energy management, using data from both solar and wind sources to optimize energy distribution and minimize loss.

Conclusion

Energy-efficient AI is essential for reducing environmental impact, lowering overhead, improving device longevity, and scaling sustainably. As AI grows more sophisticated and its resource requirements more intensive, we’ll need to find new ways to reduce energy consumption. Innovations such as low-power AI chips, GNNs, edge AI, and graph neural networks are all steps in the right direction—and together, they’re likely to support even further evolution.

Sources

[1] https://www.npr.org/2024/07/12/g-s1-9545/ai-brings-soaring-emissions-for-google-and-microsoft-a-major-contributor-to-climate-change
[2] https://www.forbes.com/sites/arielcohen/2024/05/23/ai-is-pushing-the-world-towards-an-energy-crisis/
[3] https://newsroom.arm.com/blog/driving-ai-datacenter-compute-efficiency
[4] https://i.mediatek.com/ai
[5] https://www.intel.com/content/www/us/en/products/details/processors/movidius-vpu.html
[6] https://www.ibm.com/think/topics/neuromorphic-computing
[7] https://techxplore.com/news/2024-08-tensor-processor-chip-based-carbon.html
[8] https://www.svk.se/siteassets/5.jobba-har/dokument-exjobb/graph-neural-network-forecasting-in-electric-power-systems.pdf