AI and energy: why computing power and power consumption matter
The rapid rise of generative artificial intelligence comes with a tangible physical price: energy. While software previously ran relatively 'invisibly', the current generation of Large Language Models (LLMs) requires an industrial scale of power and cooling. But why is this hunger for electricity so great, and how is it developing?
The two phases of AI energy consumption
The energy demand of AI can be roughly split into two intensive processes:
- Training phase: Building a new model requires months of calculations by thousands of specialized GPUs (graphics cards) running continuously at full capacity.
- Inference (usage): Every time a user enters a prompt and the model generates text or images, it requires significantly more computing power than a traditional search query. An average AI search query is estimated to consume 10 times as much energy as a standard Google search [to be verified].
Efficiency trends: Hardware is catching up with software
The sector is painfully aware of these scalability issues. There is currently a strong push for efficiency. On the software side, we see the rise of smaller, highly specialized models (Small Language Models) and optimized frameworks such as Retrieval-Augmented Generation (RAG), which retrieve specific data without requiring heavy computation from the entire model.
On the hardware side, it is all about heat management and efficiency. Data centers are increasingly experimenting with liquid cooling. We also see that waste heat from these giant server farms is increasingly being recovered, for example via large industrial heat pumps that heat nearby neighborhoods.
What this means for the user and developer
The dependence on power-hungry cloud infrastructure forces developers and entrepreneurs to look for alternatives. Central data centers are hitting the limits of the power grid (grid congestion). This opens the door for decentralized solutions.
We see a cautious trend towards local High-Performance Computing (HPC) setups. By running these at home or locally in the office, whether or not combined with a home battery to absorb peaks in electricity prices (dynamic tariffs) or to efficiently use self-generated solar energy, AI computing power can be commercially leased or deployed locally. The model is thus slowly shifting from pure cloud dependency to a hybrid, more distributed ecosystem.
The future of AI is determined not only by who writes the smartest model, but also by who manages the underlying energy infrastructure most intelligently.