AI isn’t magic. It’s math. And behind the curtain of every...

Originally published on Tumblr.

AI isn’t magic. It’s math. And behind the curtain of every “revolutionary” AI model lies a staggering computational cost that often goes unnoticed. Let’s break it down.

Transformers, the backbone of many state-of-the-art AI systems, are computational beasts. The forward and backward passes of these models are governed by the complexity O(n²d + nd²), where n is the sequence length and d is the model dimension. This isn’t just a theoretical exercise—it’s a real-world constraint. Each floating-point operation (FLOP) contributes to the overall energy consumption, and when scaled to the massive datasets and models used today, the numbers become astronomical.

Consider the power consumption of TPU and GPU tensor cores. These specialized processors are designed for efficiency, yet the energy required for mixed-precision matrix multiplications is non-trivial. As AI models grow, so does their appetite for power, leading to increased demand on data centers. The Power Usage Effectiveness (PUE) metric, which measures the energy efficiency of these facilities, becomes crucial. A PUE of 1.2, for instance, indicates that for every watt used by computing equipment, an additional 0.2 watts are consumed by cooling and other overheads.

But energy isn’t the only concern. The carbon footprint of AI is tied to the regional grid’s carbon intensity. In areas reliant on coal, the CO₂ emissions per kWh are significantly higher than those using renewable sources. This means that the same AI model can have vastly different environmental impacts depending on where it’s run.

Water usage for cooling is another hidden cost. On average, data centers consume about 1.8 liters of water per kWh. When scaled to the petaFLOP-days required for training large models, the water usage becomes a significant environmental consideration. It’s a sobering reminder of the physical resources underpinning digital progress.

And let’s not forget the thermodynamic limits imposed by Landauer’s principle. This principle states that erasing a single bit of information requires a minimum energy of kT ln 2, where k is the Boltzmann constant and T is the temperature in Kelvin. While current technology operates far from this limit, it serves as a theoretical boundary that underscores the inefficiencies inherent in irreversible computation.

In the rush to fund and deploy AI, as seen in recent stories of inflated valuations and failed projects, it’s crucial to remember that these systems are not without cost. The social and environmental impacts of AI should be at the forefront of our considerations. After all, a strong economy is built on a foundation of sustainable practices that prioritize the wellbeing of society over short-term gains. Let’s ensure that our pursuit of AI advancements doesn’t come at the expense of the planet.