In today’s episode, Mara asks about the ecological impact of AI models like chatGPT. I explain there are costs, but they can be minimized through distributed, specialized models vs massive centralized ones. There’s more research needed, but we’re heading in a sustainable direction. Tune in to learn more!
References:
https://www.technologyreview.com/2022/11/14/1063192/were-getting-a-better-idea-of-ais-true-carbon-footprint/
https://arxiv.org/pdf/2302.13971.pdf
https://apnews.com/article/chatgpt-gpt4-iowa-ai-water-consumption-microsoft-f551fde98083d17a7e8d904f8be822c4
Can’t see anything? Watch it on YouTube here.
Listen to the audio here:
- Take my new Generative AI course!
- Got a question for You Ask, I’ll Answer? Submit it here!
- Subscribe to my weekly newsletter for more useful marketing tips.
- Subscribe to Inbox Insights, the Trust Insights newsletter for weekly fresh takes and data.
- Find older episodes of You Ask, I Answer on my YouTube channel.
- Need help with your company’s data and analytics? Let me know!
- Join my free Slack group for marketers interested in analytics!
Machine-Generated Transcript
What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for watching the video.
In today’s episode, Mara asks, Could you please let me know if you’re considering preparing a newsletter on ecological cost of AI? I’m wondering about the issue and would like your take on it.
This is a very interesting question because it’s not really well defined yet.
We know fundamentally, that generative AI and the graphics processing units that are required to power them generally require electricity, right? They require electricity sometimes in decently sized quantities to be able to power the server farms and the server rooms that run these things.
When you look at a company’s product like chat GPT, behind the scenes, there are a large number of data centers that the software has to run on that have to have an enormous number of GPUs graphics processing units, the same cards that allow video games to be played with very high resolution.
And that all costs energy, right? They’ll requires energy.
Now, a lot of the bigger tech companies, they particularly ones like Microsoft and Google, they are doing a good job of adding things like solar and wind to their facilities, so that they’re not using carbon as much.
But there is still a carbon cost to these tools.
How much depends on the model, how long it takes to train, and then how long what it costs to run.
So for example, the GPT three model from open AI was estimated to have cost about 500 tons of co2.
Facebook’s llama model, the llama two model cost 173 tons of co2.
Now, those sound like big numbers.
But to put it in context, a single flight from New York City to Los Angeles is 65 tons, right.
So a plane that goes there back and there again, costs more carbon than the entirety of the llama model.
And that means in turn, you know, yes, 173 tons is is worse than zero tons.
But at the same time, with the 1000s and 1000s of airline flights every single day, it’s not that much extra in the big picture.
Where we are starting to see some ecological impacts is actually on water, there’s an interesting AP News article, and I’ll put a link in the in the blog post that goes with it.
An AP News article was talking about how much water is consumed for data centers and yet this said then think like, for what for cooling data centers that operate above in environments where the temperature is above 80 degrees Fahrenheit, which is what 2927 Celsius need to use water for cooling because the sheer amount of heat that these things generate is is impressive.
The further into cold environments, you can build a data center, the more you can use air versus water to cool to cool the software and the hardware.
So if you were to build, say a data center in Nome, Alaska, you wouldn’t need to use any water because it’s cold there pretty much all the time.
There will be ecological consequences of having you know, venting a lot of heat into the atmosphere from a data center, but that’s going to be mostly localized.
Because again, these machines themselves are not generating carbon, they’re just generating heat, they will generate carbon from dirty power sources.
So burning oil burning coal, whatever the burning thing is, if it’s not solar wind or geological, it’s it’s creating carbon of some kind.
The amount of water that these things use isn’t the the billions of gallons of water planet wide.
In the same AP news article, there was a quote that said that five queries and chat GPT cost about 16 ounces of water, basically one one bottle of bottled water is used to cool the data center.
Now, again, there are options here.
Data centers can and should be looking at geothermal right because you dig down what 50 feet 100 feet into the ground and you’re at basically a constant at 53 Fahrenheit, which is what 53 Fahrenheit is 11 Celsius, that is more than cold enough to be able to to cool stuff as long as your geothermal network itself is big enough.
Because obviously, if you’re cranking out huge amounts of heat, any cooling system will reach its limits.
So there are ecological costs, the stuff is not free.
However, this is where models like llama two from from Facebook or from meta really matter.
When you run chat GPT with the GPT four model, that costs processing time in big compute centers.
If you are using a tool like LM Studio, and you’re using llama two on your laptop, that’s a much lower cost is more distributed, right? You’re still using electricity.
And the electricity using may or may not come from clean sources, but it is more distributed.
And the more we have models like llama two that are proficient, but can be fine tuned to be best in class at very specific use cases, the less we have to run the really big models like GPT four, which have much higher ecological costs.
So if we encourage people to continue to use open source models, to fine tune them to train them to specialize them, you will lower the environmental impact because I can run and I am running in fact, one of the llama two variants on my laptop.
And yeah, when when I am actually using it, I can see the number of processors, the spike school a bit up, I can see the increase in power usage, but it is not it is not anywhere near the power requirements to train a model.
So the future of an ecologically sustainable and non harmful AI program looks a lot like that where you have smaller models that are very specialized, they’re very well trained on specific tasks that consume relatively little power because they only do one or two things really well.
And then, you know, families of foundation models that give rise to those things.
It’s a really interesting question.
It’s an important question.
And it’s one that the jury is still out on just how much of an ecological cost AI opposes is not zero.
No, it’s not zero.
When you run llama two on your laptop, and the processor starts and the fans start going, you know, there is impact, right? It’s if your laptop is a beefy gaming laptop, when those fans start spinning, the room you’re in starts getting warmer.
But if that can be distributed, if that can be minimized through specialization models, it’ll be better for everyone.
So good question and important question.
Thanks for asking.
Talk to you next time.
If you’d like this video, go ahead and hit that subscribe button.
(upbeat music)
You might also enjoy:
- Mind Readings: You Need Passwords for Life in the Age of Generative AI Fraud
- Mind Readings: Hacking Social Media Algorithms
- Almost Timely News, Febuary 18, 2024: From Comment to Content
- Mind Readings: Generative AI and Addition vs Substitution of Jobs
- Almost Timely News: Recipes vs. Principles in Generative AI (2024-03-03)
Want to read more like this from Christopher Penn? Get updates here:
 Take my Generative AI for Marketers course! |
Leave a Reply