Every AI Feature Has an Energy Cost

Every AI feature added to an application comes with an energy cost that is often overlooked in initial discussions. The energy required to run processors, move information through memory, cool systems, and transfer data in data centers can add up quickly. While a single AI request may seem trivial, the impact escalates when an application handles millions of such requests.

This does not imply that developers should avoid using AI; instead, energy efficiency should be integrated into product design from the outset. It's not always necessary to use a larger, more powerful model for a simple task; a smaller model, traditional search, or regular application logic could suffice. Caching frequently requested results instead of generating them repeatedly can also reduce energy consumption.

The design of the feature itself plays a crucial role. Sending a request after every minor user action can generate unnecessary work; a better interface might collect relevant information beforehand and send a single comprehensive request. Background jobs could be grouped and processed collectively rather than initiated independently.

Similar principles apply to stored data; retaining every intermediate result indefinitely necessitates more storage, backups, and infrastructure. Developers typically have no control over the electricity source or cooling methods used in data centers; however, they can manage how often their applications call a model, the amount of information they transmit, and the tool they select for each task.

As AI becomes an integral part of software, energy efficiency could become a metric for good engineering. The most efficient AI request might be the one an application discovers it never needed to make in the first place.