For reference, global energy consumption is about 180,000 TWh[1]. So while the numbers in this article are large, they're not a significant fraction of the total. Traveling and buying things are probably a much bigger part of your carbon footprint. For example:
- 25 LLM queries: ~8.5 Wh
- driving one mile: ~250-1000 Wh
- one glass bottle: ~1000 Wh [2]
- a new laptop: ~600,000 Wh [3]
- round-trip flight from LA to Tokyo: ~1,000,000 Wh
For reference, this is based on figures given by Sam Altman, which are worth as much as going to random.org and asking it for a number between 0 and 100 to use as Wh per LLM query.
Scratch that, the latter is probably more reliable.
Without knowing the cumulative amount of energy consumption it is not a fair comparison. If there are one billion llm sessions every day, it is still a lot of energy.
No, not really. A billion people (15% of the population) drive more than a mile a day. Well over 100 million laptops are sold every year. These are easy numbers to look up.
Just look at your own life and see how much of each you would use.
I think the point is that we all need to use less energy, we need to avoid flights from LA to Tokyo where possible, not using the energy use as an excuse to use even more energy.
If you want to meaningfully cut your energy usage, you need to identify its biggest sinks. 8 Wh per day is about as much as an idle charger you don't bother to remove from the outlet. I've yet to hear about anyone evaporating lakes with a charger, yet we almost all leave them plugged it.
It would be better to not use this energy, but it won't move the needle either way.
We need cheaper and cleaner forms of energy. More efficient uses of energy.
I do not agree that we "all" need to use less energy overall. Energy use tracks wealth pretty closely, and manufacturing/creating things tends to be energy intensive.
The more cheap clean energy we make available, the more novel uses will be found for it.
> We need cheaper and cleaner forms of energy. More efficient uses of energy.
Yep that's the dream, but it's not what I have coming out of my wall right now.
> Energy use tracks wealth pretty closely,
I'm guessing the majority of users on this site are in the 1% globally so it seems reasonable to consider what's produced/manufactured for us and what services like these that we're using
> The more cheap clean energy we make available, the more novel uses will be found for it.
That will be a brilliant future but it's not the reality today.
Yes quoting energy use per query isn't the full picture, though it is still a useful benchmark for understanding the relative impact of one's use as an individual. As for cumulative impact, the ieee article gives an estimate of 347 TWh per year by 2030, which is still a very small fraction of global energy consumption today.
Model usage seems quite small compared to training. I can run models on my phone which took millions of hours of GPU training time to create. Although this might change with the new AI slop tiktok apps every company is rushing to create now.
One thing it's doing is jacking up electricity rates for US States that are part of the [PJM Interconnection grid](https://en.wikipedia.org/wiki/PJM_Interconnection). It's a capacity auction price that is used to guarantee standby availability and it is [up significantly](https://www.toledochamber.com/blog/watts-up-why-ohios-electr...) at $270.43 per MW/day, which is far above prior years (~$29–58/MW/day) and this is translating to significantly higher consumer prices.
Yeah. We have been turning off old plants and not bringing on-line new ones the entire time I've been alive now. At best we've perpetually been renewing licenses to grant operation of old plants well beyond their original design lifetimes. Anything new is fought tooth and nail by practically every local community. Even solar and wind brings out the NIMBYs in force.
Every recent "datacenters are evil" news segment/article these days has a section quoting a few local NIMBYs talking about how they are opposing more transmission lines in their area for various reasons. Then these same folks (usually literally the same person) is quoted as saying that they are "for" investing into the grid and understands America needs more capacity - just not here.
It's pretty frustrating to watch. There are actually large problems with the way many local communities are approving datacenter deals - but people cannot seem to put two and two together why we are where we are. If everyone vetos new electrical infrastructure in their community, it simply doesn't get built.
Are they paying for electricity used by server farms. Or are they just paying more profits for owners of electricity producers? Do server farms get electricity below market price?
Ofc, possible long term contracts and options are involved in some of these markets. But there the option sellers would bear the cost.
This is a recurrent question and not just for servers.
In Europe it is constantly
>"why does the households of half of Europe pay for German unwillingness to have a good power mix? Why should anyone want more cross country or long range interconnects if it drives up local prices?"
Say Norway with abundant hydropower, they should by all right have cheap power. But reality is not so in half of the country because they're sufficiently interconnected to end up on a common bidders euro market and end up paying blood money for the poor political choices of countries they don't even share a border with.
Addition: this also creates perverse incentives. A good solution for many of the interconnected flat euro countries would love enormous hydropower overcapacity to be built in Norway at the cost of the local nature. This is great for whoever sells the hydropower. This is great for whoever is a politician that can show off their green all-hydro power mix in a country as hilly as a neutron star. But this is not great for whoever gets their backyard hiking trails reduced to a hydro reservoir.
But hey we do it with everything else too, "open pit mines are too destructive to have in our country, so we'll buy it from china and pretend we're making green choice. Globalism in a nutshell: Export your responsibility.
It's a supply-demand gap, but since the reasons for it are very apparent, it's completely reasonable to describe it as "consumers paying for [the existence of] datacenters".
I don't see how? It's much more reasonable to state "all electrical consumers are paying a proportionate amount to operate the grid based on their usage rates". This is typically spelled out by the rate commissions and designed to make sure one power consumer is not "subsidizing" the other.
In the case of your quoted article - taking it at face value - this means "everyone" is paying .02/khw more on their bill. A datacenter is going to be paying thousands of times more than your average household as they should.
I don't see a problem with this at all. Cheap electricity is required to have any sort of industrial base in any country. Paying a proportionate amount of what it costs the grid to serve you seems about as fair of a model as I can come up with.
If you need to subsidize some households, then having subsidized rates for usage under the average household consumption level for the area might make sense?
I don't really blame the last watt added to the grid for incremental uptick in costs. It was coming either way due to our severe lack of investment in dispatchable power generation and transmission capacity - datacenters simply brought the timeline forward a few years.
There are plenty of actual problematic things going into these datacenter deals. Them exposing how fragile our grid is due to severe lack of investment for 50 years is about the least interesting one to me. I'd start with local (and state) tax credits/abatements myself.
No, it's a lie. Consumers paying more because of data centers raising demand could be true, but that's not equivalent to them paying for the data centers' usage. The data centers also have to pay an increased rate when prices go up.
Data centers get commercial or maybe even industrial rates depending on their grid hookup and utilities love predictable loads. Those are lower than residential rates. If you're dishonest and don't understand the cost of operating a grid, you could say that's users paying for data centers. But then you'd need to apply it to every commercial/industrial user.
If the regular users were paying for data centers usage, why are so many of them going off-grid with turbines or at least partially on-prem generation?
In India, we have different energy consumption bands like 0-200kWh, 200-400kWh and so on. People whose consumption is in 0-200kWh pay less as compared to 200-400kWh and so on.
The lack of investment in energy infrastructure - especially dispatchable power sources and grid transmission - is finally coming to bite us.
Datacenters are simply the final straw/tipping point, and make a convenient scapegoat.
At some point you run out of the prior generation's (no pun intended) energy investments. Efficiency gains only get you so far, eventually you need capital investment into actually building things.
Yep, this is no different from any other consumption market showing up for power. The US has talked about 'brining manufacturing back' and while I don't see it happening, what did we expect that was going to do to the power grid.
One thing we should be careful about regarding calculations related to the larger set of "all data centers" vs only "GenAI" is that the data centers include all the predictive algorithms for social media and advertising. I, for one, would not want to misdirect ire at ChatGPT that really belongs directed at ads.
Current gen AI is going to result in the excess datacenter equivalent of dark fiber from the 2000's. Lots of early buildout and super investment, followed by lack of customer demand and later cheaper access to physical compute.
The current neural network software architecture is pretty limited. Hundreds of billions of dollars of investor money has gone into scaling backprop networks and we've quickly hit the limits. There will be some advancements, but it's clear we're already at the flat part of the current s-curve.
There's probably some interesting new architectures already in the works either from postdocs or in tiny startups that will become the base of the next curve in the next 18 months. If so, one or more may be able to take advantage of the current overbuild in data centers.
However, compute has an expiration date like old milk. It won't physically expire but the potential economic potential decreases as tech increases. But if the timing is right, there is going to be a huge opportunity for the next early adopters.
Nobody is actively building out nuclear power. Microsoft is turning on a recently decommissioned facility.
New nuclear is too expensive to make sense. At most there are small investments in flash-in-the-pan startups that are failing to deliver plans for small modular reactors.
The real build out that will happen is solar/wind with tons of batteries, which is so commonplace that it doesn't even make the news. Those can be ordered basically off the shelf, are cheap, and can be deployed within a year. New nuclear is a 10-15 year project, at best, with massive financial risk and construction risk. Nobody wants to take those bets, or can really afford to, honestly.
Plenty of bets being placed on nuclear, but they are moonshot style bets.
From where I'm standing, the immediate capital seems to be being deployed at smaller-scale (2-5MW) natural gas turbines co-located on site with the load. I haven't heard a whole lot of battery deployments at the same scale.
Of course turbines are now out at 2029 or something for delivery.
Only marginally at the edge of this space these days though, so what I hear is through the grapevine and not direct any longer.
Of course, remember that nameplate capacities from different technologies should be corrected for capacity factor, which is roughly 60% for gas, 40% for wind, and 25% for solar, but pre-correction EIA expects
And then there's an expected retirement of 1.6GW of old gas this year.
I'm pretty disconnected from the data center folks, but in general the current political environment is highly disfavorable to solar and batteries, and using them too much could have lots of political blowback that is very expensive.
Of course, small gas also has the benefit that the operating costs are spread over the lifetime, rather than being an up-front cost. So even if solar+batteries is cheaper than gas over the lifetime of the system, gas may seem more expedient if you don't want a lot of capital on the books.
If you're a utility, you may not like that solar and batteries are driving down electricity costs and reducing grid expenses. But even with the thumbs against the scale, we are seeing the most nameplate deployment (see caveats in my parallel reply) in decades, and will likely set a record, because of solar, batteries, and wind in that order:
> There's probably some interesting new architectures already in the works either from postdocs or in tiny startups
It is not clear to me why we will have a breakthrough after virtually no movement on this front for decades. Backpropagation is literally 1960s technology.
Your comment sort of implies that all this is some super standardized flow that is well studied and highly optimized but in my experience all this ML stuff is closer to the edge of broken than some kind of local maximum.
There is an ungodly number of engineering decisions that go into making ML work and any number of stupid things are all over the place that cause stuff to fail.
Like something stupid like your normalization was bad or your mix of data was bad or your learning rates were bad or you have some precision issues or your model has a bad init or some architectural problems cause poor training or straight up there are tons of bugs somewhere like your batching was doing something silly or there is some numerically unstable division or sqrt or somewhere etc etc.
At scale with stupid issues like hardware faults I imagine this only gets exponentially worse.
And then on product sides of integrating stuff more bugs sneak in like so many labs were releasing so many open source LLMs with broken and incorrectly configured chat templates that massively tanked performance.
Or they set up some parmeters in sampling wrong and stuff gets stuck in loops or hallucinates tons or something.
In his 2025 hotchips keynote Noam Shazeer (GDM VP) even says that you need hardware determinism because there are just so many bugs in ML experiments that you need to be able to tweak and test things.
Also there are just so many obvious issues with the way everything works conventionally in GPT2 style like with softmax causing attention sinks at punctuation and creating dispersion over longer sequences because of low sharpness and the whole previllaged basis thing making it so common information takes up a lot of model capacity.
This. I totally agree we will see better architectures for doing the calculations, lower energy usage inference hardware and also some models running on locally moving some of the "basic" inference stuff off the grid.
It's going to move fast I think and I would not surprised if the inference cost in energy is 1/10 of today in less than 5 years.
Also adding to that tendency, I suspect as the tech matures more and more consumer space models will just run on device (sure, the cutting edge will still run in server farms but most consumer use will not require cutting edge).
This is one possibility I'm assuming as well. It largely depends on how long this bubble lasts. At the current growth rate it will be unsustainable before many very large DCs can be built so it's possible the impact may not be as severe as the telecom crash.
Another possibility is that new breakthroughs significantly reduce computational needs, efficiency significantly improves, or some similar improvements that reduce DC demand.
If I'm interpreting this right it's estimating that ChatGPT's daily energy usage is enough to charge just 14,000 electric vehicles - and that's to serve in the order of ~100 million daily users.
Why would you assume that? It’s in line with estimates that were around before he posted that article and it’s higher than Gemini. It’s a pretty unsurprising number.
"Does smoking cause cancer?" and "Does burning fossil fuels cause global warming?" are a different category from "How much energy does it take to run a prompt?"
The prompt energy question is something that companies can both actively measure and need to actively measure in order to plan their budgets. It's their job to know the answer.
Those other questions, sadly, fall more into the category of it's their job not to know the answer.
You're not suggesting that Philip Morris and ExxonMobil didn't know the actual answers to both of those questions, surely? It's a known, easily verifiable fact that they did. They just lied about them.
I am saying that it is necessary for OpenAI to know the exact correct answer to the question about prompt energy costs in order to effectively run their business.
I would bet that it's far lower now. Inference is expensive we've made extraordinary efficiency gains through techniques like distillation. That said, GPT-5 is a reasoning model, and those are notorious for high token burn. So who knows, it could be a wash. But selective pressures to optimize for scale/growth/revenue/independence from MSFT/etc makes me think that OpenAI is chasing those watt-hours pretty doggedly. So 0.34 is probably high...
a) training is where the bulk of an AI system's energy usage goes (based on a report released by Mistral)
b) video generation is very likely a few orders of magnitude more expensive than text generation.
That said, I still believe that data centres in general - including AI ones - don't consume a significant amount of energy compared with everything else we do, especially heating and cooling and transport.
Pre-LLM data centres consume about 1% of the world's electricity. AI data centres may bump that up to 2%
You gotta start thinking about the energy used to mine and refine the raw materials used to make the chips and GPUs. Then take into account the infrastructure and data centers.
Seems ... low? And it will only get more efficient going forward.
I don't get why this is supposed to be a big deal for infrastructure since there's definitely way more than 14,000 EVs out there and we are doing well.
the infrastructure needs to go somewhere. And that somewhere needs to have access to abundant water and electricity. It just so happens those are things humans need too.
Before GenAI we were on our way to optimizing this, at least to the level where the general public can turn a blind eye. It was to the point where the companies would brag about how much efficient they are. Now all that progress is gone, and we're accelerating backwards. Maybe that was all a lie too. But then who's to say the current numbers are a lie too to make the pill easier to swallow.
That's why LLM providers are losing so much money. They're spending it on training and hardware growth. If you don't include those factors they make pretty good money on the services they provide.
>the manufacturing of GPUs and datacenters themselves consumes a large amount of energy, not just their operation. The operational energy use (for AI training, inference, cooling, etc.) gets most of the attention, but the embodied energy — the energy used to extract raw materials, manufacture chips and components, and construct facilities — is substantial.
and summarizes it with:
4. Bottom Line
• Manufacturing GPUs and datacenters is highly energy-intensive, but operational energy dominates over time.
• For a single GPU, embodied energy ≈ 0.5–1 MWh.
• For a datacenter, embodied energy ≈ 6–12 months of its operational energy.
• Given AI-scale deployments (millions of GPUs), the embodied manufacturing energy already reaches terawatt-hours globally — roughly comparable to the annual electricity use of a small country.
>The Schneider Electric report estimates that all generative AI queries consume 15 TWh in 2025 and will use 347 TWh by 2030; that leaves 332 TWh of energy—and compute power—that will need to come online to support AI growth. T
+332TW is like... +1% of US power consumption, or +8% of US electricity. If AI bubble burst ~2030... that's functionally what US will be left with (assuming new power infra actually built) mid/long term since compute depreciates 1-5 years. For reference dotcom burst left US was a fuckload of fiber layouts that lasts 30/40/50+ years. Still using capex from railroad bubble 100 years ago. I feel like people are failing to grasp how big of a F US will eat if AI bursts relative to past bubbles. I mean it's better than tulip mania, but obsolete AI chips also closer to tulips than fiber or rail in terms of stranded depreciated assets.
Honestly the excess fiber put in the ground back then has nothing to do with fiber capacity now. The same fiber back then that could carry 100mbs can use new transceivers that push a terabit
Current network capacity runs off excess fiber. Networking gets upgraded at nodes and dumb pipe (excess fiber) efficiency improves. Like we upgrade switching for 100+ year old rail to improve freight efficiency. Much of $$$$$ / capex "wasted" in dot bubble boom went to civil engineering - digging trenches to build out "agnostic" fiber conduits with multi decade life span that can be repuporsed for general use.
Bulk of AI capex build out is going to be in specialized hardware and data centers with bespoke power / cooling / networking profile. If current LLM approaches turn out to be deadend the entire data centre is potentially stranded asset unless future applications can specifically take advantage. But there's a good chance _if_ LLM crashes, then it might be due to something inherently wrong with current approach, i.e. compute/cost doesn't make commercial sense, and resuing stranded data centers might not make economic sense.
>We used the figure of 0.34 watt-hours that OpenAI’s Sam Altman stated in a blog post without supporting evidence. It’s worth noting that some researchers say the smartest models can consume over 20 Wh for a complex query. We derived the number of queries per day from OpenAI's usage statistics below.
I honestly have no clue how much trust to place in data from a blog post written by a guy trying to make people give him lots of money. My gut is to question every word that comes out of his mouth but I'm maybe pessimistic in that regard.
But besides that, the cost of this stuff isn't just the energy consumption of the computation itself; the equipment needs to be manufactured, raw materials need to be extracted and processed, supplies and manpower need to be shuffled around. Construction of associated infrastructure has it's own costs as well. what are we, as a society (as opposed to shareholders and executives) going to get in return and is it going to be enough to justify the costs, not just in terms of cash but also resources. To say nothing of the potential environmental impact of all this.
The question to ask is why supply of energy hasn't kept up with demand. Regulations (primarily in Democratic states) is most likely the answer. When you use government incentives to pick winners and losers with energy sources, it throws the entire energy market out of sync.
For reference, global energy consumption is about 180,000 TWh[1]. So while the numbers in this article are large, they're not a significant fraction of the total. Traveling and buying things are probably a much bigger part of your carbon footprint. For example:
- 25 LLM queries: ~8.5 Wh
- driving one mile: ~250-1000 Wh
- one glass bottle: ~1000 Wh [2]
- a new laptop: ~600,000 Wh [3]
- round-trip flight from LA to Tokyo: ~1,000,000 Wh
[1] https://ourworldindata.org/energy-production-consumption
[2] https://www.beveragedaily.com/Article/2008/03/17/study-finds...
[3] https://www.foxway.com/wp-content/uploads/2024/05/handprint-...
For reference, this is based on figures given by Sam Altman, which are worth as much as going to random.org and asking it for a number between 0 and 100 to use as Wh per LLM query.
Scratch that, the latter is probably more reliable.
What about LLM training? What about training all the discarded or unused LLMs?
Without knowing the cumulative amount of energy consumption it is not a fair comparison. If there are one billion llm sessions every day, it is still a lot of energy.
No, not really. A billion people (15% of the population) drive more than a mile a day. Well over 100 million laptops are sold every year. These are easy numbers to look up.
Just look at your own life and see how much of each you would use.
So do LLMs mean fewer people have to drive a mile every day?
These don't have to be dependent to be meaningful.
I think the point is that we all need to use less energy, we need to avoid flights from LA to Tokyo where possible, not using the energy use as an excuse to use even more energy.
If you want to meaningfully cut your energy usage, you need to identify its biggest sinks. 8 Wh per day is about as much as an idle charger you don't bother to remove from the outlet. I've yet to hear about anyone evaporating lakes with a charger, yet we almost all leave them plugged it.
It would be better to not use this energy, but it won't move the needle either way.
> we all need to use less energy
We need cheaper and cleaner forms of energy. More efficient uses of energy.
I do not agree that we "all" need to use less energy overall. Energy use tracks wealth pretty closely, and manufacturing/creating things tends to be energy intensive.
The more cheap clean energy we make available, the more novel uses will be found for it.
> We need cheaper and cleaner forms of energy. More efficient uses of energy.
Yep that's the dream, but it's not what I have coming out of my wall right now.
> Energy use tracks wealth pretty closely,
I'm guessing the majority of users on this site are in the 1% globally so it seems reasonable to consider what's produced/manufactured for us and what services like these that we're using
> The more cheap clean energy we make available, the more novel uses will be found for it.
That will be a brilliant future but it's not the reality today.
Do we need to use less energy, or do we need to use less fossil fuel based energy?
Both. We need to stop using fossil fuels altogether. And we need to use the other energy more efficiently.
Yes quoting energy use per query isn't the full picture, though it is still a useful benchmark for understanding the relative impact of one's use as an individual. As for cumulative impact, the ieee article gives an estimate of 347 TWh per year by 2030, which is still a very small fraction of global energy consumption today.
180,000 TWh total since the start of time or per year?
It was 170,000 TWh annually in 2021.
Isn't it easer to say average 20 TW
More succinct yes, but talking in units per year is often easier to reason about.
How did you get "one glass bottle: ~1000 Wh"?
[2] does not cite energy use, only CO2 emissions
Not OP but the EPA has a calculator for estimating emissions to energy consumption. https://www.epa.gov/energy/greenhouse-gas-equivalencies-calc...
Model usage seems quite small compared to training. I can run models on my phone which took millions of hours of GPU training time to create. Although this might change with the new AI slop tiktok apps every company is rushing to create now.
One thing it's doing is jacking up electricity rates for US States that are part of the [PJM Interconnection grid](https://en.wikipedia.org/wiki/PJM_Interconnection). It's a capacity auction price that is used to guarantee standby availability and it is [up significantly](https://www.toledochamber.com/blog/watts-up-why-ohios-electr...) at $270.43 per MW/day, which is far above prior years (~$29–58/MW/day) and this is translating to significantly higher consumer prices.
Why are consumers paying for electricity used by server farms? Why can't the electricity companies charge the server farms instead?
Where I live, the utility company bills you at a higher rate if you use more electricity.
Because electricity prices are an auction, so increased demand is bidding up the price anyway.
You need strong residential consumer protections to avoid this.
do you? maybe we just need more supply
The residential consumers also oppose that. Usually they try very hard to reduce it. E.g. Diablo Canyon NPP
Yeah. We have been turning off old plants and not bringing on-line new ones the entire time I've been alive now. At best we've perpetually been renewing licenses to grant operation of old plants well beyond their original design lifetimes. Anything new is fought tooth and nail by practically every local community. Even solar and wind brings out the NIMBYs in force.
Every recent "datacenters are evil" news segment/article these days has a section quoting a few local NIMBYs talking about how they are opposing more transmission lines in their area for various reasons. Then these same folks (usually literally the same person) is quoted as saying that they are "for" investing into the grid and understands America needs more capacity - just not here.
It's pretty frustrating to watch. There are actually large problems with the way many local communities are approving datacenter deals - but people cannot seem to put two and two together why we are where we are. If everyone vetos new electrical infrastructure in their community, it simply doesn't get built.
How can residential consumers successfully oppose power plants but not server farms?
Are they paying for electricity used by server farms. Or are they just paying more profits for owners of electricity producers? Do server farms get electricity below market price?
Ofc, possible long term contracts and options are involved in some of these markets. But there the option sellers would bear the cost.
This is a recurrent question and not just for servers.
In Europe it is constantly >"why does the households of half of Europe pay for German unwillingness to have a good power mix? Why should anyone want more cross country or long range interconnects if it drives up local prices?"
Say Norway with abundant hydropower, they should by all right have cheap power. But reality is not so in half of the country because they're sufficiently interconnected to end up on a common bidders euro market and end up paying blood money for the poor political choices of countries they don't even share a border with.
Addition: this also creates perverse incentives. A good solution for many of the interconnected flat euro countries would love enormous hydropower overcapacity to be built in Norway at the cost of the local nature. This is great for whoever sells the hydropower. This is great for whoever is a politician that can show off their green all-hydro power mix in a country as hilly as a neutron star. But this is not great for whoever gets their backyard hiking trails reduced to a hydro reservoir.
But hey we do it with everything else too, "open pit mines are too destructive to have in our country, so we'll buy it from china and pretend we're making green choice. Globalism in a nutshell: Export your responsibility.
> consumers paying for electricity used by server farms
wait what? consumers are literally paying for server farms? this isn't a supply-demand gap?
It's a supply-demand gap, but since the reasons for it are very apparent, it's completely reasonable to describe it as "consumers paying for [the existence of] datacenters".
I don't see how? It's much more reasonable to state "all electrical consumers are paying a proportionate amount to operate the grid based on their usage rates". This is typically spelled out by the rate commissions and designed to make sure one power consumer is not "subsidizing" the other.
In the case of your quoted article - taking it at face value - this means "everyone" is paying .02/khw more on their bill. A datacenter is going to be paying thousands of times more than your average household as they should.
I don't see a problem with this at all. Cheap electricity is required to have any sort of industrial base in any country. Paying a proportionate amount of what it costs the grid to serve you seems about as fair of a model as I can come up with.
If you need to subsidize some households, then having subsidized rates for usage under the average household consumption level for the area might make sense?
I don't really blame the last watt added to the grid for incremental uptick in costs. It was coming either way due to our severe lack of investment in dispatchable power generation and transmission capacity - datacenters simply brought the timeline forward a few years.
There are plenty of actual problematic things going into these datacenter deals. Them exposing how fragile our grid is due to severe lack of investment for 50 years is about the least interesting one to me. I'd start with local (and state) tax credits/abatements myself.
No, it's a lie. Consumers paying more because of data centers raising demand could be true, but that's not equivalent to them paying for the data centers' usage. The data centers also have to pay an increased rate when prices go up.
Data centers get commercial or maybe even industrial rates depending on their grid hookup and utilities love predictable loads. Those are lower than residential rates. If you're dishonest and don't understand the cost of operating a grid, you could say that's users paying for data centers. But then you'd need to apply it to every commercial/industrial user.
If the regular users were paying for data centers usage, why are so many of them going off-grid with turbines or at least partially on-prem generation?
The solution is more and cheaper energy.
Charge them more than individual consumers? Why? Let the market decide how much electricity should be. /s
Hehe. Well, if the market is no good for its participants, then at least there is a viable alternative for many of them.
I think the unit you and the article want are MW-Day of un enforced capacity UCAP, not MW/Day.
PJM claims this will be a 1.5-5% yoy increase for retail power. https://www.pjm.com/-/media/DotCom/about-pjm/newsroom/2025-r...
In India, we have different energy consumption bands like 0-200kWh, 200-400kWh and so on. People whose consumption is in 0-200kWh pay less as compared to 200-400kWh and so on.
The lack of investment in energy infrastructure - especially dispatchable power sources and grid transmission - is finally coming to bite us.
Datacenters are simply the final straw/tipping point, and make a convenient scapegoat.
At some point you run out of the prior generation's (no pun intended) energy investments. Efficiency gains only get you so far, eventually you need capital investment into actually building things.
Yep, this is no different from any other consumption market showing up for power. The US has talked about 'brining manufacturing back' and while I don't see it happening, what did we expect that was going to do to the power grid.
One thing we should be careful about regarding calculations related to the larger set of "all data centers" vs only "GenAI" is that the data centers include all the predictive algorithms for social media and advertising. I, for one, would not want to misdirect ire at ChatGPT that really belongs directed at ads.
You are paying for AI whether you want it or not. Just use it at least I guess. You have no say over anything else.
My thoughts.
Current gen AI is going to result in the excess datacenter equivalent of dark fiber from the 2000's. Lots of early buildout and super investment, followed by lack of customer demand and later cheaper access to physical compute.
The current neural network software architecture is pretty limited. Hundreds of billions of dollars of investor money has gone into scaling backprop networks and we've quickly hit the limits. There will be some advancements, but it's clear we're already at the flat part of the current s-curve.
There's probably some interesting new architectures already in the works either from postdocs or in tiny startups that will become the base of the next curve in the next 18 months. If so, one or more may be able to take advantage of the current overbuild in data centers.
However, compute has an expiration date like old milk. It won't physically expire but the potential economic potential decreases as tech increases. But if the timing is right, there is going to be a huge opportunity for the next early adopters.
So what's next?
If the end result here is way overbuilt energy infrastructure that would actually be great. There’s a lot you can do with cheap electrons.
I suspect it will mostly be fossil power capacity, which is much easier to scale up
I wouldn’t be so sure about that. Serves of the big names in this space have green energy pledges and are actively building out nuclear power.
Nobody is actively building out nuclear power. Microsoft is turning on a recently decommissioned facility.
New nuclear is too expensive to make sense. At most there are small investments in flash-in-the-pan startups that are failing to deliver plans for small modular reactors.
The real build out that will happen is solar/wind with tons of batteries, which is so commonplace that it doesn't even make the news. Those can be ordered basically off the shelf, are cheap, and can be deployed within a year. New nuclear is a 10-15 year project, at best, with massive financial risk and construction risk. Nobody wants to take those bets, or can really afford to, honestly.
Plenty of bets being placed on nuclear, but they are moonshot style bets.
From where I'm standing, the immediate capital seems to be being deployed at smaller-scale (2-5MW) natural gas turbines co-located on site with the load. I haven't heard a whole lot of battery deployments at the same scale.
Of course turbines are now out at 2029 or something for delivery.
Only marginally at the edge of this space these days though, so what I hear is through the grapevine and not direct any longer.
As far as the grid goes, there's a tiny bit of gas additions, but it's mostly solar, battery, and wind:
https://www.eia.gov/todayinenergy/detail.php?id=65964
Of course, remember that nameplate capacities from different technologies should be corrected for capacity factor, which is roughly 60% for gas, 40% for wind, and 25% for solar, but pre-correction EIA expects
And then there's an expected retirement of 1.6GW of old gas this year.I'm pretty disconnected from the data center folks, but in general the current political environment is highly disfavorable to solar and batteries, and using them too much could have lots of political blowback that is very expensive.
Of course, small gas also has the benefit that the operating costs are spread over the lifetime, rather than being an up-front cost. So even if solar+batteries is cheaper than gas over the lifetime of the system, gas may seem more expedient if you don't want a lot of capital on the books.
> The real build out that will happen is solar/wind with tons of batteries
That actually sounds awesome, is there a downside I’m not seeing?
If you're a utility, you may not like that solar and batteries are driving down electricity costs and reducing grid expenses. But even with the thumbs against the scale, we are seeing the most nameplate deployment (see caveats in my parallel reply) in decades, and will likely set a record, because of solar, batteries, and wind in that order:
https://www.eia.gov/todayinenergy/detail.php?id=65964
There are a couple companies doing HTGR and SMRs that seem to be on track.
they don't care much: https://www.selc.org/press-release/musks-xai-explores-anothe...
> There's probably some interesting new architectures already in the works either from postdocs or in tiny startups
It is not clear to me why we will have a breakthrough after virtually no movement on this front for decades. Backpropagation is literally 1960s technology.
Your comment sort of implies that all this is some super standardized flow that is well studied and highly optimized but in my experience all this ML stuff is closer to the edge of broken than some kind of local maximum.
There is an ungodly number of engineering decisions that go into making ML work and any number of stupid things are all over the place that cause stuff to fail.
Like something stupid like your normalization was bad or your mix of data was bad or your learning rates were bad or you have some precision issues or your model has a bad init or some architectural problems cause poor training or straight up there are tons of bugs somewhere like your batching was doing something silly or there is some numerically unstable division or sqrt or somewhere etc etc.
At scale with stupid issues like hardware faults I imagine this only gets exponentially worse.
And then on product sides of integrating stuff more bugs sneak in like so many labs were releasing so many open source LLMs with broken and incorrectly configured chat templates that massively tanked performance.
Or they set up some parmeters in sampling wrong and stuff gets stuck in loops or hallucinates tons or something.
In his 2025 hotchips keynote Noam Shazeer (GDM VP) even says that you need hardware determinism because there are just so many bugs in ML experiments that you need to be able to tweak and test things.
Also there are just so many obvious issues with the way everything works conventionally in GPT2 style like with softmax causing attention sinks at punctuation and creating dispersion over longer sequences because of low sharpness and the whole previllaged basis thing making it so common information takes up a lot of model capacity.
I'd like to add to this that in the recent Y combinator podcast with Anthropic head of pretraining the bugs issue is brought up as a major issue[1].
It is so easy to have good ideas broken by random bugs everywhere...
[1] https://youtu.be/YFeb3yAxtjE?t=2919
Because tremendous rewards will spur a huge increase in research?
This. I totally agree we will see better architectures for doing the calculations, lower energy usage inference hardware and also some models running on locally moving some of the "basic" inference stuff off the grid.
It's going to move fast I think and I would not surprised if the inference cost in energy is 1/10 of today in less than 5 years.
This said Jeavons paradox will likely mean we still use more power.
Hopefully there's a flood of good cheap used Supermicro and other enterprise gear and maybe a lot of cheap colo.
Yes, that makes sense.
Also adding to that tendency, I suspect as the tech matures more and more consumer space models will just run on device (sure, the cutting edge will still run in server farms but most consumer use will not require cutting edge).
This is one possibility I'm assuming as well. It largely depends on how long this bubble lasts. At the current growth rate it will be unsustainable before many very large DCs can be built so it's possible the impact may not be as severe as the telecom crash.
Another possibility is that new breakthroughs significantly reduce computational needs, efficiency significantly improves, or some similar improvements that reduce DC demand.
It's a line (remindme! 5 years)
If I'm interpreting this right it's estimating that ChatGPT's daily energy usage is enough to charge just 14,000 electric vehicles - and that's to serve in the order of ~100 million daily users.
> We used the figure of 0.34 watt-hours that OpenAI’s Sam Altman stated in a blog post without supporting evidence.
what do you think the odds of this being accurate are?
zero?
Why would you assume that? It’s in line with estimates that were around before he posted that article and it’s higher than Gemini. It’s a pretty unsurprising number.
Back in the 1980s, I'm sure Philip Morris' claimed similar numbers on cigarettes (not) causing cancer as R.J. Reynolds did.
I also wouldn't be surprised if Aramco and Rosneft gave similar estimates on global warming and oil's role in it.
"Does smoking cause cancer?" and "Does burning fossil fuels cause global warming?" are a different category from "How much energy does it take to run a prompt?"
The prompt energy question is something that companies can both actively measure and need to actively measure in order to plan their budgets. It's their job to know the answer.
Those other questions, sadly, fall more into the category of it's their job not to know the answer.
You're not suggesting that Philip Morris and ExxonMobil didn't know the actual answers to both of those questions, surely? It's a known, easily verifiable fact that they did. They just lied about them.
I am saying that it is necessary for OpenAI to know the exact correct answer to the question about prompt energy costs in order to effectively run their business.
Hard to say. Sam wrote that on June 10th this year: https://blog.samaltman.com/the-gentle-singularity
GPT-5 came out on 7th August.
Assuming the 0.34 value was accurate in the GPT-4o era, is the number today still in the same ballpark or is it wildly different?
the "AI" industry have identified that energy usage is going to be used as a stick to beat them with
if I was altman then I'd release a few small numbers to try and get influencers talking about "how little energy chatgpt uses"
and he can never be accused of lying, as without any methodology as to how it was calculated it's unverifiable and completely meaningless
win-win!
I would bet that it's far lower now. Inference is expensive we've made extraordinary efficiency gains through techniques like distillation. That said, GPT-5 is a reasoning model, and those are notorious for high token burn. So who knows, it could be a wash. But selective pressures to optimize for scale/growth/revenue/independence from MSFT/etc makes me think that OpenAI is chasing those watt-hours pretty doggedly. So 0.34 is probably high...
...but then Sora came out.
Yeah, something we are confident about is that
a) training is where the bulk of an AI system's energy usage goes (based on a report released by Mistral)
b) video generation is very likely a few orders of magnitude more expensive than text generation.
That said, I still believe that data centres in general - including AI ones - don't consume a significant amount of energy compared with everything else we do, especially heating and cooling and transport.
Pre-LLM data centres consume about 1% of the world's electricity. AI data centres may bump that up to 2%
You mean this Mistral report? https://mistral.ai/news/our-contribution-to-a-global-environ...
I don't think it shows that training uses more energy than inference over the lifetime of the model - they don't appear to share that ratio.
> don't consume a significant amount of energy compared with everything else we do, especially heating and cooling and transport
Ok, but heating and cooling are largely not negotiable. We need those technologies to make places liveable
LLMs are not remotely as crucial to our lives
You gotta start thinking about the energy used to mine and refine the raw materials used to make the chips and GPUs. Then take into account the infrastructure and data centers.
The amount of energy is insane.
And yet still tiny in relationship to transportation energy requirements and transportation itself is stuck on fossil fuels mostly.
At the end of the day green energy is perfect for AI and AI workloads.
I was about to post this exact thing.
Seems ... low? And it will only get more efficient going forward.
I don't get why this is supposed to be a big deal for infrastructure since there's definitely way more than 14,000 EVs out there and we are doing well.
the infrastructure needs to go somewhere. And that somewhere needs to have access to abundant water and electricity. It just so happens those are things humans need too.
Before GenAI we were on our way to optimizing this, at least to the level where the general public can turn a blind eye. It was to the point where the companies would brag about how much efficient they are. Now all that progress is gone, and we're accelerating backwards. Maybe that was all a lie too. But then who's to say the current numbers are a lie too to make the pill easier to swallow.
Hmm, I guess we'll have to do it the slow way ...
What % of EVs on the market is 14,000?
This doesn't include the energy for mining and chip production either. Can you imagine if it did?
Then when you take into account the amount of water used to cool the data centers as well as part o extraction and production process? Things get insane then https://www.forbes.com/sites/cindygordon/2024/02/25/ai-is-ac...
Very useful context:
"How much energy does Google’s AI use? We did the math": https://cloud.google.com/blog/products/infrastructure/measur...
This doesn't seem to factor in the energy cost of training which is currently a very significant overhead.
How do you know it is significant?
That's why LLM providers are losing so much money. They're spending it on training and hardware growth. If you don't include those factors they make pretty good money on the services they provide.
The energy used for manufacturing the datacenters including the GPU's must also be rather high. Manufacturing is an energy-intensive sector.
Edit: I asked ChatGPT-5:
https://chatgpt.com/share/68e36c19-a9a8-800b-884e-48fafbe0ec...
it says:
>the manufacturing of GPUs and datacenters themselves consumes a large amount of energy, not just their operation. The operational energy use (for AI training, inference, cooling, etc.) gets most of the attention, but the embodied energy — the energy used to extract raw materials, manufacture chips and components, and construct facilities — is substantial.
and summarizes it with:
Related:
OpenAI’s Hunger for Computing Power Has Sam Altman Dashing Around the Globe
https://news.ycombinator.com/item?id=45477192
(94 comments, 1 day ago)
Thanks a lot, Chris!
Math comparing new datacenter capacity to electric cars -
Projections estimate anywhere between 10GW to 30GW of US datacenter buildup over the next few years
1GW of continuous power can support uniform draw from ~2.6M Tesla Model 3s assuming 12,000 miles per year, 250Wh/mile.
So 26M on the lower end, 80M Model 3s on the upper end.
That's 10x-30x the cumulative number of Model 3s sold so far
And remember all datacenter draw is concentrated. It will disproportionately going to impact regions where they're being built.
We need new, clean power sources yesterday
And an increase in power demand and increased power prices will ensure those wind/solar farms get built.
>The Schneider Electric report estimates that all generative AI queries consume 15 TWh in 2025 and will use 347 TWh by 2030; that leaves 332 TWh of energy—and compute power—that will need to come online to support AI growth. T
+332TW is like... +1% of US power consumption, or +8% of US electricity. If AI bubble burst ~2030... that's functionally what US will be left with (assuming new power infra actually built) mid/long term since compute depreciates 1-5 years. For reference dotcom burst left US was a fuckload of fiber layouts that lasts 30/40/50+ years. Still using capex from railroad bubble 100 years ago. I feel like people are failing to grasp how big of a F US will eat if AI bursts relative to past bubbles. I mean it's better than tulip mania, but obsolete AI chips also closer to tulips than fiber or rail in terms of stranded depreciated assets.
Honestly the excess fiber put in the ground back then has nothing to do with fiber capacity now. The same fiber back then that could carry 100mbs can use new transceivers that push a terabit
Current network capacity runs off excess fiber. Networking gets upgraded at nodes and dumb pipe (excess fiber) efficiency improves. Like we upgrade switching for 100+ year old rail to improve freight efficiency. Much of $$$$$ / capex "wasted" in dot bubble boom went to civil engineering - digging trenches to build out "agnostic" fiber conduits with multi decade life span that can be repuporsed for general use.
Bulk of AI capex build out is going to be in specialized hardware and data centers with bespoke power / cooling / networking profile. If current LLM approaches turn out to be deadend the entire data centre is potentially stranded asset unless future applications can specifically take advantage. But there's a good chance _if_ LLM crashes, then it might be due to something inherently wrong with current approach, i.e. compute/cost doesn't make commercial sense, and resuing stranded data centers might not make economic sense.
>We used the figure of 0.34 watt-hours that OpenAI’s Sam Altman stated in a blog post without supporting evidence. It’s worth noting that some researchers say the smartest models can consume over 20 Wh for a complex query. We derived the number of queries per day from OpenAI's usage statistics below.
I honestly have no clue how much trust to place in data from a blog post written by a guy trying to make people give him lots of money. My gut is to question every word that comes out of his mouth but I'm maybe pessimistic in that regard.
But besides that, the cost of this stuff isn't just the energy consumption of the computation itself; the equipment needs to be manufactured, raw materials need to be extracted and processed, supplies and manpower need to be shuffled around. Construction of associated infrastructure has it's own costs as well. what are we, as a society (as opposed to shareholders and executives) going to get in return and is it going to be enough to justify the costs, not just in terms of cash but also resources. To say nothing of the potential environmental impact of all this.
MWh per day? TWh per year?
In what world are these sensible units of power? Why can't we just use Watts FFS?
Btw:
1 MWh per day ≈ 42 kW
1 TWh per year ≈ 114 MW
Or:
All Chat GPT users: 850 MWh / day = 310 GWh / year ≈ 35.4 MW
All AI users: 15 TWh / year ≈ 1.7 GW
[dead]
[dead]
[flagged]
The question to ask is why supply of energy hasn't kept up with demand. Regulations (primarily in Democratic states) is most likely the answer. When you use government incentives to pick winners and losers with energy sources, it throws the entire energy market out of sync.