It is common knowledge that machine learning consumes a batch of energy. All those ai models powering email digests, technology-67012224″>regicidal chatbots, and videos of Homer Simpson singing nu-metal are racking up a hefty server bill measured in megawatts per hour. But apparently no one, not even the companies behind the technology, can say exactly what the cost is.
Estimates exist, but experts say those figures are partial and contingent, offering only a snapshot of ai's total energy use. This is because machine learning models are incredibly variable and can be configured in ways that drastically alter their energy consumption. Furthermore, the organizations best placed to introduce a bill (companies like Meta, Microsoft, and OpenAI) simply do not share the relevant information. (Judy Priest, CTO of cloud operations and innovations at Microsoft, said in an email that the company is currently “investing in developing methodologies to quantify the energy use and carbon impact of ai while working on ways to make large systems more efficient, both in training and application.” OpenAI and Meta did not respond to requests for comment).
One important factor we can identify is the difference between training a model for the first time and deploying it to users. Training, in particular, is energy intensive and consumes much more electricity than traditional data center activities. Training a large language model like GPT-3, for example, is My dear use just under 1,300 megawatt hours (MWh) of electricity; approximately as much power as consumed annually by 130 American households. To put this in context, streaming an hour of Netflix requires about 0.8 kWh (0.0008 MWh) of electricity. That means you would have to watch 1,625,000 hours to consume the same amount of energy as it takes to train GPT-3.
But it's hard to say how a figure like this applies to today's most modern systems. Power consumption could be higher, because ai models have been trending upward in size for years and larger models require more power. On the other hand, companies could be using some of the proven methods make these systems more energy efficient, which would curb the upward trend in energy costs.
The challenge with making updated estimates, says Sasha Luccioni, a researcher at French-American ai firm Hugging Face, is that companies have become more secretive as ai has become profitable. If we went back a few years, companies like OpenAI would be publishing details of their training regimens: what hardware and for how long. But the same information simply doesn't exist for the latest models, like ChatGPT and GPT-4, Luccioni says.
“With ChatGPT we don't know how big it is, we don't know how many parameters the underlying model has, we don't know where it runs… It could be three raccoons in a trench coat because we just don't know what's under the hood.”
“It could be three raccoons in a trench coat because you just don't know what's under the hood.”
Luccioni, author of several articles examining the energy use of ai, suggests that this secrecy is partly due to competition between companies, but is also an attempt to deflect criticism. Statistics on ai energy use (especially in its more frivolous use cases) naturally invite comparisons with the wastefulness of cryptocurrencies. “There is a growing awareness that all this is not free,” he says.
Training a model is only part of the picture. Once a system is created, it is deployed to consumers who use it to generate results, a process known as “inference.” Last December, Luccioni and his colleagues at Hugging Face and Carnegie Mellon University published an article (currently awaiting peer review) which contained the first estimates of the inference energy usage of several ai models.
Luccioni and his colleagues ran tests on 88 different models covering a variety of use cases, from answering questions to identifying objects and generating images. In each case, they ran the task 1000 times and estimated the energy cost. Most of the tasks they tested use a small amount of energy, such as 0.002 kWh to classify written samples and 0.047 kWh to generate text. If we use our hour of Netflix streaming as a comparison, these are equivalent to the energy consumed watching nine seconds or 3.5 minutes, respectively. (Remember: that's the cost of doing each task 1,000 times.) The numbers were notably higher for the imaging models, which used an average of 2,907 kWh per 1,000 inferences. As the article points out, the average smartphone uses 0.012 kWh to charge, so generating an image using ai can Uses almost as much energy as charging your smartphone.
However, the emphasis is on “can”, as these figures do not necessarily generalize across all use cases. Luccioni and his colleagues tested ten different systems, from small models that produced tiny 64 x 64 pixel images to larger models that produced 4K images, and this resulted in a huge range of values. The researchers also standardized the hardware used to better compare different ai models. This does not necessarily reflect real-world implementation, where software and hardware are often optimized for energy efficiency.
“This is definitely not representative of everyone's use cases, but now at least we have some numbers,” Luccioni says. “I wanted to put a flag in the ground, saying 'Let's start from here.'”
“The generative ai revolution has a planetary cost that we are completely unaware of.”
The study therefore provides useful relative data, although not absolute figures. It shows, for example, that ai models require more power to generate results than when classifying inputs. It also shows that anything involving images consumes more energy than text. Luccioni says that while the contingent nature of this data can be frustrating, it tells a story in itself. “The generative ai revolution has a planetary cost that we are completely unaware of and, for me, its spread is especially indicative,” he says. “The thing is, we just don't know.”
So trying to determine the energy cost of generating a single Balenciaga potato is complicated due to the tangle of variables. But if we want to better understand the planetary cost, there are other paths to follow. What if, instead of focusing on model inference, we zoom out?
This is the approach of Alex de Vries, a PhD candidate at VU Amsterdam who cut his teeth calculating bitcoin's energy expenditure for his blog. Digiconomistand who has used Nvidia GPUs (the gold standard of ai hardware) to estimate the sector's overall energy use. As de Vries explains in the comment published in Joule Last year, Nvidia accounted for about 95 percent of sales in the ai market. The company also publishes power specifications for its hardware and sales projections.
Combining this data, de Vries estimates that by 2027 the ai sector could consume between 85 and 134 terawatt hours each year. This is roughly the same as the annual energy demand of De Vries' home country, the Netherlands.
“We're talking about ai electricity consumption potentially accounting for half a percent of global electricity consumption by 2027,” says de Vries. The edge. “I think it's a pretty significant number.”
A recent report from the International Energy Agency offered similar estimates, suggesting that electricity use in data centers will increase significantly in the near future thanks to the demands of artificial intelligence and cryptocurrencies. The agency says current data center energy use is around 460 terawatt hours in 2022 and could rise to between 620 and 1,050 TWh in 2026, equivalent to the energy demands of Sweden or Germany, respectively.
But de Vries says it's important to put these figures in context. He notes that between 2010 and 2018, data center energy use has been fairly stable, accounting for about 1 to 2 percent of global consumption. (And when we say “data centers” here we mean everything that makes up the “Internet”: from corporations' internal servers to all the apps you can't use offline on your smartphone.) Demand certainly increased during this period, de Vries says, but hardware became more efficient, thus offsetting the increase.
Their fear is that things may be different for ai precisely because of companies' tendency to simply throw bigger models and more data at any task. “That's a really lethal dynamic for efficiency,” de Vries says. “Because it creates a natural incentive for people to keep adding more computational resources, and as soon as the models or the hardware become more efficient, people will make those models even bigger than before.”
It is impossible to answer the question of whether efficiency gains will offset increased demand and usage. Like Luccioni, de Vries laments the lack of available data, but says the world cannot simply ignore the situation. “It's been a little tricky to figure out which direction this is going and it's certainly not a perfect number,” he says. “But it is sufficient basis to give a little warning.”
Some companies involved in ai say the technology itself could help with these problems. Priest, speaking on behalf of Microsoft, said ai “will be a powerful tool in advancing sustainability solutions” and emphasized that Microsoft was working to achieve “sustainability goals of being carbon negative, water positive and zero waste by 2030.”
But a company's objectives can never encompass all the demand of the entire industry. Other approaches may be needed.
Luccioni says he would like to see companies introduce energy star ratings for ai models, allowing consumers to compare energy efficiency in the same way they would with household appliances. For de Vries, our approach should be more fundamental: do we even need to use ai for particular tasks? “Because given all the limitations that ai has, it's probably not going to be the right solution in a lot of places, and we're going to waste a lot of time and resources finding out the hard way,” he says.
