The Challenges of Powering AI Infrastructure

Impact Reflection

Artificial intelligence entered the public domain in a dramatic way when OpenAI released ChatGPT as a tool for language composition and analysis in late 2022. Exciting and a little scary in equal measure, it engaged a broad swath of the public in interacting with technology in a way that mimicked human interaction. Technically, the term “artificial intelligence” refers to both “the theory and development of computer systems [that are] able to perform tasks that normally require human intelligence.” Visual perception, speech recognition, decision-making, and translation are all examples of what AI can do. 

In the short time since ChatGPT arrived, it’s become clear that AI will transform both our society and our economy. Conversations about the exciting potential growth of AI and hyperscale computing capability must also address the logistical and environmental challenges of expanding infrastructure to handle the dramatic increase in electricity demanded by powerful chips. The AI transformation is energizing investors and policy makers to better understand the challenges of building the most productive and sustainable foundation possible. 

The revolution in graphics chips, for which Nvidia has been the standard-bearer, began in 1999 with their invention of the graphics processing unit (GPU). These chips provided faster and more detailed graphics that could simulate live-action more and more effectively. Nvidia has stayed at the leading edge of GPU innovation supporting increasingly sophisticated gaming uses and also the background AI functions we have all become accustomed to including text autocorrect, map navigation, facial and fingerprint recognition and customized internet searches.

One key technological advance that Nvidia and other GPU manufacturers have made is to create ever smaller but still very precise circuitry on computer chips. In 1965, Intel founder Gordon Moore made the observation (later dubbed “Moore’s Law”) that the number of transistors on an integrated circuit would double every two years, also doubling computing capacity. It further suggested that the associated financial cost of computing would simultaneously decline. This has generally held true over the last sixty years.

In the last decade or two, chipmakers have reached the limits of atomic-scale circuitry, and innovation to accelerate capacity has blossomed in other directions including not only hardware but software. Nvidia’s innovation in GPUs has now fueled an even greater growth rate in computing, named “Huang’s Law” by the Wall Street Journal after Nvidia’s CEO and co-founder Jensen Huang.¹ Accelerated development supported by GPUs and advanced AI-specific hardware has now made generative AI like ChatGPT possible.

The expansion of AI to multiple industries is forcing computing infrastructure to grow to data-center scale and beyond, dramatically increasing the energy needed to continue to evolve the complexity of AI applications. Hyperscale data centers are 10 to 20 times as large as conventional data centers and are needed to supply the computing power that generative AI requires.²

Researchers argue that these increases in energy use will be offset by increasing efficiency.³ Historically, that hasn’t been the case. Making a resource cheaper and more efficient has led to a persistent increase in the use of the new technology and an increase in its power use on an absolute basis. More powerful technology also changes expectations and, in doing so, raises the baseline of power demand to meet those expectations.

Today, there are few requirements for measurement that specifically focus on AI uses, making it hard to understand what sort of regulation (or further innovation) makes sense. Regulators in the EU have now engaged on this issue, and in February 2024, passed the “AI Act” to take effect in 2025 that requires high-intensity AI systems to report their energy usage, resource use, and other impacts through their systems’ lifecycles.⁴

Impact-oriented investors and asset managers are pushing hard to define the metrics needed to control the new baseline demand that is virtually certain to come. The dramatic increase in power needs – and the need for water for cooling – could dovetail productively with efforts to reduce greenhouse gas emissions and push forward the transition to renewable energy generation, but only if we can measure and manage the metrics of energy demand.

Nvidia’s Blackwell chip was unveiled this spring, with computing speeds that are 30x faster while also reducing energy use by 25x. This chip needs more electricity on an absolute basis to produce those amazing effects, though, using 1200 watts of electricity per chip. That’s about the amount of energy needed to power the average home in the US.

Some companies are building their own “behind the meter” or off-grid power generation to handle the increased density of chips and increased electricity demand. That strategy risks outages with no ability to resort to grid power. Supply growth is also taking the form of utility company-based growth with guaranteed power purchase agreements already signed by the largest users supporting the development of AI such as Google, Amazon, and Microsoft.

Concern over the increase in electricity demand goes beyond encouraging the growth of solar farms and other new renewable energy sources. The US electricity grid has limits to its ability to flex power transmission as needs change, and while the rapid development of utility company facilities is adding capacity, it is also adding complexity to management of the grid. Rolling blackouts and brown-outs have been the consequence of asymmetrically added demand and supply amid grid limitations for decades. The development of AI is predicted to run into real world energy capacity limitations before 2030 in a way that short-circuits its expected growth trajectory. Jeff Jakubiak, an energy regulatory attorney, notes that “there has never been such potential for imbalance between supply and demand in the electric grid as we are seeing coming down the track with AI.”⁵

Impact investing has seen its strategy and philosophy evolve over the decades. In the early 1970s, it began with a focus on excluding harmful companies from investment strategies, then expanded to include a “best in class” approach that sought to duplicate market diversification while minimizing negative impact. In the most recent decade, there has been a steady ascendence of an approach based on positive impact. This approach emphasizes more active engagement with company management teams to help guide them towards more positively impactful practices while staying profitable as well.

The momentum of AI is increasing and thus poses a particular challenge in assessing impact. The basics remain – you can’t manage what you can’t measure – but to engage with this developing technology, impact analysis has to go “real time” as well. We need to work to improve the environmental impacts of AI, even as we are still trying to measure them, and proceed in dialogue among all stakeholders. Protecting the environment while also avoiding a breakdown of our electrical infrastructure – that is the real-time challenge of AI.

Resources

¹ Mims, Christopher, “ Huang’s Law is the New Moore’s Law, and Explains Why Nvidia Wants Arm,”Wall Street Journal, September 19, 2020.

² Berreby, David, “As the Use of A.I. Soars, So Does the Energy and Water It Requires,” e360.yale.edu, February 6, 2024.

³ In the US, the AI Accountability Act and the Blueprint for an AI Bill of Rights are two elements of an effort underway for more than a year to convene national (and international) conversation about a different impact of AI, addressing threats to privacy and concerns hard-coding of bias and discrimination.

⁴ Berreby, ibid.

⁵ Green, Peter, “AI’s Thirst for Power,” Quartz, June 8, 2024.