The REAL danger of AI (w/Timnit Gebru)

On Tuesday night, midterm voters in the United States delivered a crushing blow to Silicon Valley’s dreams of eternal dominance. But the fight to save democracy is only beginning. I’ll have extended thoughts on the elections and the road ahead...but, for now, here’s a new episode of The Nerd Reich podcast.

Silicon Valley billionaires claim they are building a godlike “superintelligence” that will save—or destroy—humanity. But is this apocalyptic storyline a massive marketing scheme to seize natural resources and consolidate power?

For this episode of The Nerd Reich podcast, I talk to renowned computer scientist Dr. Timnit Gebru, founder of the Distributed AI Research Institute (DAIR).

Dr. Gebru breaks down the cult-like behavior of tech CEOs, explains what Large Language Models (LLMs) actually are, and exposes the devastating environmental toll of AI data centers (which she says aren’t really necessary).

Click below to watch on YouTube:

Click here to listen to The Nerd Reich podcast on Apple

Click here to listen to The Nerd Reich podcast on Spotify

Available wherever you get your podcasts.

Transcript below.

The Nerd Reich Podcast with Dr. Timnit Gebru, founder of the Distributed AI Research Institute (DAIR) and author of Deep Unlearning: The Rise of AI and the Radicalization of a Tech Idealist

Transcripts may contain errors.
Gil Durán: Silicon Valley's AI barons claim they're creating a godlike superintelligence, and that they're on the verge of saving or destroying humanity. But what if that apocalyptic storyline is just a massive marketing scheme designed to seize natural resources, land defense contracts and consolidate power? Today, we're joined by one of the most qualified critics in the tech world, Dr. Timnit Gebru. She is a renowned computer scientist, founder of the Distributed Artificial Intelligence Research Institute, and a winner of the prestigious Right Livelihood Award. A former Google insider, she's also the co-author of the groundbreaking Stochastic Parrots paper that exposed what AI chatbots actually are. In this episode, we dive into the cult-like behavior of tech CEOs, the environmental toll of AI data centers, and why we absolutely do not have to surrender to the overheated AI hype. If you feel like you're living in a zombie apocalypse of AI propaganda, this episode of the Nerd Reich podcast is the antidote.

Gil Durán: Hey, Timnit, welcome to the Nerd Reich podcast.

Timnit Gebru: Thank you for having me.

Gil Durán: We've wanted to have you on for quite a while, but you were busy writing your book and I was busy writing my book. Now we're both done with our books, so we can have a good talk. We did an episode earlier on TESCREAL with Dr. Émile Torres and got through some of that, and I'll ask about that later. We also have a lot of really good questions from our paid subscribers, who are very excited for this conversation.

Let's start, though, with the present moment. We live in a time when AI, artificial intelligence, is often the top news story. Our entire politics and worldview are being shaped by the idea that we're on the precipice of a massive transformation. Some frame it as a massive economic transformation. Others frame it in more apocalyptic terms, as a godlike superintelligence that will transform what it means to be human. How did we get to this moment, and what's really going on right now?

Timnit Gebru: Oh, man. I think you and I should be in a support group together. That's how I feel sometimes. I feel like I'm in a zombie apocalypse, where there's this narrative, the zombie that's biting humans, and you're trying to figure out who hasn't been bitten yet. I just cannot believe we're in this situation, where I'm out with the press saying, no, magical machine gods don't exist. And they're like, how do you know? And I'm like, well, I'm not the person you should be asking that question.

But how we got here: as you well know, there are cycles of AI summers and AI winters. We've been in perhaps the craziest AI summer in history, where so much money is being thrown at this field. And the same people who've been throwing money at it, Elon Musk or Peter Thiel or Jaan Tallinn or any of these billionaires you want to mention, have been talking about their fear of AI being an existential risk to humanity ever since they got involved in AI. It reminds me of their fears about the end of the world and bunkers, and what happens when money is no longer something people care about, so you can't pay people to protect you. These are the things billionaires think about, because they're so privileged they can't think of anything else in the world attacking them.

In some ways this has been happening for a long time. I remember asking Elon Musk in 2015 why he was so worried about AI being an existential risk. I said to him, isn't climate change the existential risk? And he said, no, climate change is not going to kill every single human, but AI can kill every single human. And of course he's been saying this since way back. So every few years these men, and it's usually all men, write a letter, like the pause AI letter in 2023. In 2015 the Future of Life Institute had a similar letter. It's very confusing, right? Because they're saying AI can basically bring us utopia, but it can also kill us.

The way I see it, the coded message is: if we, the good guys, the vanguards of humanity, are the ones who control it, we're going to bring the utopia. But if you make the mistake of not letting us be the people controlling this thing, this machine god, then it's going to be an apocalypse. They do this all the time. And now, given the impending IPOs at OpenAI and Anthropic, and the defense contracts they want, I feel like they were amping up this rhetoric anyway. And of course now they're using it as an excuse not to IPO. So to me it's the same old, same old, basically, with more money.

Gil Durán: I think a lot of people have trouble understanding that this is a self-serving narrative these people are using. But it is a bit complex, in that they already have an apocalyptic mindset. They have a tendency toward grandiosity and want to view themselves as the heroes in some kind of cosmic final struggle.

Timnit Gebru: They want to save the world, right?

Gil Durán: This is part of it. And I think that with the billionaire mentality, plus the incentives of a venture capital class that is looking for the ultimate big exit... and what's a bigger exit than creating the godlike intelligence that's going to suck all value up into the hands of a handful of people? I feel like that's part of the reason it's been hard for people in the media to understand or question the godlike AI narrative. It's really a testament to what you can do by investing money early on in a Harry Potter sex cult in Berkeley, which then goes on to shape important global narratives. We'll get to that a little later.
I wanted to start there to give people some grounding in the moment we're in. I've seen you everywhere, on Democracy Now and in the media. Finally they're starting to include some other voices. But it's kind of funny: they start with the doomer voices around AI, and then they move on to, maybe let's ask some actual experts what's really going on. And it's hard, because when that first framework takes hold, it's hard to undo that thinking in a lot of people's minds.
One thing I'll say is that it does seem like the apocalyptic marketing has also harmed them. People have become terrified and scared of AI. We're starting to see people rise up against data centers and against the idea of AI eating their jobs, eating the future. That seems like maybe an unintended political consequence of this terrifying marketing narrative that's been spread.

Timnit Gebru: Well, I don't know, because what I am seeing is that once they saw this data center activism... I don't know if you saw that article in The Atlantic, "data centers are a distraction" or something like that. Because in the last couple of weeks, who has been talking about data centers? Nobody, right? There was a bipartisan, really very popular data center resistance movement. And now it's the kill switch, the self-policing thing.

Jack Clark, co-founder of [Anthropic, formerly of] OpenAI, was saying that stochastic parrots, my paper that I wrote in 2020, was like a cognitive mind virus. His point was that because lawmakers listen to people like me, they were behind on actually instituting laws to stop the fictional existential risk of AI, superintelligence. So I actually don't know if the apocalyptic language has backfired on them. I think what they were trying to do is, they saw the real conversation about real harms, and they redirected it to this apocalyptic thing. And now politicians like Bernie Sanders aren't talking about data centers. They're talking about all this other stuff, right?

The other thing, and you're right, is how difficult it is for the media to parse. Because they're like, well, it's not just a tech CEO repeating a self-serving narrative. It's also people putting their jobs on the line, whistleblowers, experts like Geoff Hinton. Now we have to rebut all of these points, because to the outside world it looks like a whole bunch of independent experts who are unconnected to each other are coming together and saying these things. But in fact, as you mentioned, there are cults that go way, way back, decades, in the same forums, at the same summits, talking about this apocalyptic or utopian narrative, funding each other. Just a few billionaires seeding all of these institutes that look very independent, but they're not.

And then, like you mentioned, there's the grandiose, godlike feeling of saving the world. You don't have to not know the techniques, or how things work, to feel like you've created a god. Ray Kurzweil, the "Singularity Is Near" guy: he talked about how the hidden Markov models of the '80s were going to bring us into the singularity, way back in 2012, 2013. Nobody would take that seriously now. He didn't say it because he doesn't know how hidden Markov models work. It's just that grandiosity, building a machine god.

And we sound like the crazy ones. You start talking about the cults and eugenics and sex parties, and if you don't have the exact quotes at the back of your head, which is what I was trying to do on Democracy Now, to paint a picture, you sound like the crazy one, the conspiracy theorist. Because it does sound like a conspiracy theory, right? It doesn't sound like a real thing that's happening in the world.

Gil Durán: And a lot of that comes from the narrative already being framed. When I first started reading about this stuff, not too long ago, my first introduction was Nick Bostrom's Superintelligence, which people recommended as the way to start understanding it. The first time I heard of Eliezer Yudkowsky, I thought, oh, here's an AI expert who is warning against this stuff and he's against those companies, so maybe he's the good guy. It took a lot more education to understand what was really happening there, in all that complexity.

One of the things that's been a saving grace for me is knowing people who are actual technical experts. They all uniformly say, that's just not how it works. And it's almost frustrating that they have to be insulted, trying to debunk these false scenarios. As someone who's been involved in public relations and communications in the past, I know that once you frame something first, once people hear a story a certain way, it's very hard to deprogram.
I want to give listeners a sense of who you are. Unlike a lot of the people in the AI debate, you are actually an expert. You're a scientist, highly educated, and you know the technical aspects. There's this divide right now where AI is bad, any use of AI is bad. And I often find people have one definition of AI in their minds: an LLM, a generative AI chatbot. It doesn't include all the stuff on your phone that's some version of machine learning and AI, that's maybe using you even if you don't think you're using it. Why did you go into tech, and what was your journey to where you are now?

Timnit Gebru: Math and physics were the things that were just interesting to me. My father was an electrical engineer, and my sisters ended up being electrical engineers, so it just sort of happened. I was like, I'll be an electrical engineer. I got into that and worked as a circuit designer for a while. Moving around in specialties, I found myself in a specialty called signal processing. Diffusion models, like the image generators, actually come from old, old ideas from information theory, signal processing and control theory, which a lot of circuit work deals with. Then I somehow found myself in a field called image processing as well, and then in another field called computer vision, which ended up being under the field of AI. I wasn't trying to get into AI. I was doing these different things, and computer vision ended up being AI.

It was really strange, because I found myself in computer vision at a time when a historic rift was happening. The so-called godfathers of AI today, Geoff Hinton and Yann LeCun and Yoshua Bengio, used to be the kids nobody wanted to talk to, right? Machine learning itself was not really popular and wasn't called AI, and within machine learning, deep learning was even less so. They started using the term deep learning to take the stigma away from neural networks, because it turned out the academic journals were rejecting papers that had the term neural networks in them.

When I got into computer vision, it was around the time the ImageNet challenge came about. It really seeded a lot of things historically. This was the first time such a large dataset was created, and the first time they used a crowdsourcing platform like Amazon['s Mechanical Turk]. It's now commonplace, the dominant way you assemble huge datasets, to have exploited workers around the world making a few cents per image, or per whatever they're labeling. The second thing that happened is that after this huge dataset was created, there was a challenge: object recognition and other kinds of classification tasks on images. And for the first time, a deep learning based model outperformed all the other models by a large, large chunk. That was Geoff Hinton, Alex Krizhevsky and Ilya Sutskever's paper.

It was at this inflection point that I was joining computer vision. And there was a fight. Yann LeCun wrote an open letter to the computer vision community saying he wasn't going to submit his papers to computer vision anymore, because they kept rejecting his papers and were hostile to deep learning. Fast forward to now, and of course deep learning is computer vision and computer vision is deep learning. There's no difference. But that was the inflection point when I was joining. And then a few years later, we all started calling ourselves AI. I was like, oh, I guess I am AI. [Fine,] call me AI. Sure. Don't call me computer vision. So we all kind of started adopting the term AI.

I was not planning on being critical, or on writing about large language models or face recognition. Then there was that ProPublica article about Northpointe, a company purporting to have an algorithm that determines your likelihood of committing a crime again, which was obviously labeling Black people as potential criminals at a much higher rate. And I wouldn't see any Black people around me. So that's what got me to the next evolution of my career, which is starting to point out the issues with these kinds of methodologies.

Gil Durán: So you went into it for the work and the technical challenges, like a lot of workers. And it seems like you fell into AI the way a lot of us have. Suddenly everything's AI. Apps I've used for years are now AI. I used to use Otter because it could transcribe interviews as you go and save you a lot of time. Now it's Otter AI. It wasn't like that a few years ago, but suddenly everything has become AI. Even Allbirds, the shoe company in San Francisco, is now rebranding as an AI company.

Timnit Gebru: Are you serious?

Gil Durán: Oh, yeah. They rebranded as an AI company. It's very bizarre.

Timnit Gebru: How do you get from shoes to AI? What is that connection? I don't even understand.

Gil Durán: I'm going to have to look into that a little more deeply. It almost seems like a publicity stunt. But I was talking to somebody and they were like, oh, well, they have this whole interesting plan. I was like, hmm. The Allbirds rabbit hole is something to go down at some point.
Let's talk about your radicalization in the tech world. A lot of us know about you because of the stochastic parrots paper. That phrase may not make a lot of sense to people hearing it for the first time, but it really helps explain this debate over what these large language model chatbots are, the ones so many people have started using in the past few years and that most people consider AI. Regular people who are not experts are talking about chatbots, they're talking about LLMs, which have increasingly been incorporated into everything we use online, from Google to your newspaper subscription to social media, which is constantly trying to make you use a chatbot of some kind.

Timnit Gebru: To write your posts or whatever, yeah.

Gil Durán: To write your posts, right? What is even the point of that? Give us a simple [explanation] of what an LLM, or a chatbot powered by an LLM, actually is, and how you came to the conclusion of stochastic parrots, and explain that concept to us.

Timnit Gebru: At the beginning, when we were talking about how we got to this moment, I was telling you it's the same old, same old for me. Back in 2020 a similar thing was happening. We were at the height of the Black Lives Matter movement. OpenAI had been around for five years at that point. They came out with a technical report on something called GPT-3. Before that they had come out with a model called GPT-2, and they had in fact claimed GPT-2 was too powerful and too dangerous to release. The same kind of rhetoric. It was very irritating for me to see the debate. Basically the philosophers were asking, is GPT-2 artistic? Is it creative? Is it ethical? Those were the questions they were asking.
I was at Google at the time. There was a lot of excitement around GPT-3. There was also a lot of, let's say, disappointment on the part of Jeff Dean, who was VP of research at the time and chief scientist. He's no longer at Google. He was kind of upset that OpenAI had the largest models. It had set off this race. By the time GPT-3 came out, there was already a race in the industry to have larger and larger models.

What do I mean by larger and larger models? Larger means more parameters, which means a more complicated mathematical function connects your input to your output, which means it requires more computational power to train. So we were seeing this explosion of models that required a ton of data, so they were guzzling everything on the internet, and a ton of compute power. At that point the entire field seemed to be redirected to building large language models, which is a specific type of model where you're trying to calculate the most likely sequences of words, the most likely sequences of text, given your training data.

It's really interesting. It was never [about being] against the concept of large language models, because they can be useful when you use them as a subset of some kind of system. It really is OpenAI that drove this problem. The issue came when the likes of DeepMind, OpenAI and then later, of course, Anthropic explicitly tried to create a machine god, what they call artificial general intelligence. The only way you can describe it is a machine god. It doesn't have any other scientific definition. DeepMind was one of the first prominent organizations to do this, and they were very focused on reinforcement learning, another technique. OpenAI did the same thing, because they saw DeepMind. They were also focused on reinforcement learning techniques, so they had this thing called a reinforcement learning gym. But at some point they pivoted to believing that using large language models to generate language specifically was the way to create a machine god.

So I started getting very worried about this singular direction the field was being redirected toward, this obsession with building larger and larger language models. I contacted Emily Bender on Twitter, because she was the only person I saw not believing the hype. She was like, well, I don't see their data. So I asked her, hey, do you have a paper where you talk about the dangers of large language models? And she said, I don't have a paper. Do you want to write one?

Stochastic parrots is a term Emily coined, which basically describes what large language models do. You can be fooled into believing that they understand something. Even back then, a Google VP was saying large language models understand your intent, things like that. This was back in 2020. You can be fooled into believing that they understand language, that they understand you. That's the point we were trying to make. We saw how this way of building things can deceive people, because they see a mind behind whatever fluent text is being generated. There's a tendency to attribute a mind behind it, an intent. That was the idea we were trying to convey.

Now, a big talking point among both tech executives like Jack Clark and proponents of the idea that we're building a superintelligence is that this idea is outdated, because these models are no longer stochastic parrots, that large language models are now built differently. My point, with Emily, and we're trying to write a technical explainer on this if we have the time, is that even though the chatbots have bells and whistles now, they're still stochastic parrots. We have so many examples from today. Emily and I wrote an op-ed about the U.S. military almost going to war with China, I don't know if you saw it, because of fabricated information from a chatbot. If you believe you have a machine god, you're more likely to use these things in such high-stakes scenarios. But if you believe they're stochastic parrots, you're more likely to be careful and not trust the outputs of these systems.

Gil Durán: Let's break it down a little more. Stochastic because they're just generating out of probabilities, out of a large range of possibilities. And parrots because...

Timnit Gebru: To parrot. The verb to parrot is to repeat without understanding. People were getting offended. It's not supposed to be a derogatory term. It's not supposed to say that large language models don't do anything. It's just supposed to be a term explaining the way large language models work. The reason I think it's better to attribute it to the verb to parrot rather than to actual parrots is that at some point a parrot might associate the word food with getting food, or something like that. So we don't want to claim that parrots don't understand. But the verb to parrot means specifically to repeat without understanding.

Gil Durán: Well, parrots have consciousness, right?

Timnit Gebru: At the very least, the animals, the parrots.

Gil Durán: But that's the idea. To me, that's the most interesting part of the divide. When you understand how LLMs work from the technical side, it takes a big leap to say this is creating a godlike consciousness, because how would it do that? But that seems to be the missing part. The establishment press can never quite figure out which side might be more accurate.

Timnit Gebru: The annoying thing here is that for every expert who understands machine learning, they find someone like Yoshua Bengio or Geoff Hinton who says the most ridiculous things. Geoff Hinton said GPT-4 is humanity's butterfly. And then he said ChatGPT is an alien being. Now, Geoff Hinton understands how neural networks work. He might not understand how LLMs work, but backpropagation, all of these things are his ideas.

The problem, to me, is not just not understanding the technicalities of how specific things like machine learning work. It's also not understanding a whole bunch of other fields, and being too arrogant to believe you need to understand other fields.

The group of people who understand machine learning is not the same as the group of people who understand language, how people process language and what it means to understand language. That's a different set of expertise. The group of people who understand the brain is not the same as the group of people who understand how to build a large language model. And of course you have all these people who also don't feel like they have to understand anything about society and power and how human beings interact with each other. So it's not just about not understanding certain things at a technical level. It's also the arrogance of feeling like that's all you need to know to make these grand proclamations, that you don't need to know anything else.

Gil Durán: You were early in criticizing this AI race and the approach these companies were taking, at a time when a lot of people thought, oh, we're accelerating quickly into this wonderful future. Everyone started incorporating these companies into their routines. Universities, newspapers, everyone was signing deals, with this idea that we'd better get in now because it's going to be this big thing and we don't want to be left behind. Even though most people had already had their content and intellectual property vacuumed up into these databases to create these chatbots, which are so impressive to most people, regardless of how they work.

And it seems like the world has finally caught up, in part, to this critique that the approach is wrong. Unfortunately, it's being framed as a need to pause or stop in order to avoid creating a godlike intelligence that we can't control. What do you think leaders in the media are mostly getting wrong about this need to take a different direction? How would you explain to people what really needs to happen right now?

Timnit Gebru: Yeah, I'm curious to hear your thoughts too. But for me, it's the narrative the public is being fed right now. Here's the reason a lot of people got confused. In 2023 there was this pause AI letter, and the authors of the Stochastic Parrots paper wrote a statement against it. And people said, but aren't you on the same side? Aren't you saying these companies need to pause?

The narrative is that there is a machine god that is imminent. "Let's not build a machine god" is not the narrative. Demis Hassabis was just on 60 Minutes, and the whole segment was about how AI can eradicate disease. Dario is very much a doomer, but they're still pushing on the math side, because they think that if they can show their systems have cracked some mathematical problems that have stood for many years, they'll prove to us that they have built a machine god. So the story is not, don't build a machine god. The story is that it's already been written. A machine god is already in the works and there will be one, but we have to be careful how we steer it, we have to be careful who controls it, and we have to slow down. That's what they say.

So the conversation is not about why we even need these chatbots. Who asked for this? Why do we need this kind of thing? Why are we building this kind of technology? Why aren't we building something else? When they say pause, slow down, it's still, inadvertently, marketing for these companies. Because the reason we need to pause, the reason we need to slow down, is that there is this super powerful machine they've built, and even the creators are scared of its power. This is marketing, because with this hand you're saying stop, but with this hand you're talking to the military, who you know wants to integrate [AI] up and down the chain.

What you're really telling them is: you don't want your adversaries to have this super powerful technology. If we're doing this, China is obviously doing it too. You want to be the one working on it, you want to be the one controlling it, and you want us to control it because we're on the side of America first. And you don't want to slow us down with these pesky masses complaining about data centers and whatever. That's the real messaging. If we make mistakes, well, it's rogue models, which are super powerful, and you want them in your hands.

So this is the story the media is getting completely wrong. I understand that it really captures the public's imagination, because you're not just fighting so much money. You're also fighting a decades-long scenario that has been seeded in the public's imagination, like Frankenstein or Terminator. Hollywood has put this kind of scenario in the public consciousness for a long time. Filmmakers could do us a favor and seed different narratives.

The public has already been primed for this kind of story, and the media, unfortunately, is talking to the same kinds of people who are seeding the story. So leaders, the media, the public, everybody's being fed this wrong thing. What they're not understanding is that it's all very big marketing for these tech companies that are looting them. And meanwhile, a lot of the media companies have literally bought into these visions by making partnerships and deals with these companies. They're now owned by the same people, right?

Gil Durán: Increasingly. In my experience as a journalist, no one's dumber about technology than the leaders of journalism. At every step of my career going back to the '90s, they missed every boat and failed to see what was coming. I was at the newspaper of Silicon Valley, the San Jose Mercury News, in 1998. We thought the future was bright and it was forever. We were going to be the new New York Times, because San Jose and Silicon Valley were now the center of the world and the universe. What nobody saw coming was how this was going to crater ad revenue and destroy newspapers. Within three years... I could run out and do any story. The newspaper was full of reporters, with bureaus in Tokyo, in Vietnam, in Europe, you name it. The paper got decimated. Now it's, I don't know, about 12 people, maybe.

Timnit Gebru: Are you serious? Is it that small?

Gil Durán: It's thin. It's small. They may have a few more reporters than that, but they've also been bought by terrible hedge fund companies, so it's all smashed together with other newspapers. And no one saw that coming, right? I look back and think, why did we think we understood technology enough to tell the world about Silicon Valley, and not realize we were headed for this cliff, our own destruction, over something as simple as classifieds and ad revenue?
And again, when I returned to the newsroom in 2018, it must have been around 2019 or 2020, there was a sudden rush to go into the NFT business, the non-fungible token business. I remember people being asked to find stories or headlines or images or pages that could be turned into NFTs that we were somehow going to sell to people.

Timnit Gebru: The newspaper wanted to do this?

Gil Durán: Images from the newspaper, historic pictures, cartoons. I remember feeling stupid, because I thought, how is any of that valuable? How was it valuable to sell? Not the rights to the image, but the supposed pixels.

Timnit Gebru: You're like, there must be something in it.

Gil Durán: There must be something I'm not getting. Maybe there's something I don't know, but I'm glad they've figured out this endless source of revenue from pixels. Of course, that went absolutely nowhere.

And now we see the same thing happening with LLM chatbots. The paper I used to work for, the Sacramento Bee, started incorporating AI into everything about a year and a half ago. Reporters now had to do AI summaries of their stories. And I thought, "aren't you just training your replacement?" Boom, layoffs now, and this sort of move toward somehow getting AI to do most of the work. And so it seems to me that it's not clear why journalists should be explaining this to people.

Timnit Gebru: Why do you think they were this ignorant? Is it because they want sources? They don't want to burn bridges with their sources, so they overtrust what the people inside these tech companies are telling them? Why do you think journalists didn't see this coming?

Gil Durán: Corporate powers tend to invest in PR and narrative shaping, and the people trying to do good, or pursue the more truthful path, tend to underestimate the power of being out there shaping the narrative. There tends not to be investment in people who are critical of corporate narratives. Nobody says, hey, here's a few million dollars, you guys just sit around and have a club where you're always ready to go, always ready to talk to the media, with narratives ready-made for newspaper stories. So that's how you get these investments in... and this brings us to our next question here: TESCREAL, the bundle of interlinking religious ideas that have helped shape the narrative.

I think part of it is that if that's what journalists are hearing, and those are the people who pick up every time they call and are always ready to go, then that's who gets to shape the story. That's true in other areas as well. It's why the right wing does so well. They always have people ready to go, ready to be on camera, ready to talk about anything, whether they know about it or not. They get the talking points, memorize them, go on TV and argue those points. I think that's part of it.

Timnit Gebru: To me, this recent discussion over AI has been a big triumph of the power of propaganda, which is nothing new. I've been telling people, I've experienced war and everything, I understand big, powerful people and all of that. But I've never seen anything this up close, the way I saw AI. I was in the same office as some of these super powerful people, in the same places. I saw who they are. I saw how they talked. I saw this whole thing unfold. It really taught me more about how the world works than any other experience I've had.

Gil Durán: What are the real harms of AI? What should people actually be worried about with these technologies?

Timnit Gebru: I think it's like what you said with NFTs: oh, there must be something to it, I must be missing something. Everybody has the right intuition. If you're miserable at your job because this chatbot is being pushed down your throat, there is nothing more to it. That is a harm. You shouldn't feel like you're anti-progress, or that you're not able to sacrifice yourself or something, which is what you're being told, right? When your kids can't use pen and paper anymore and have to interact with a chatbot at school. There are kids dying by suicide because they're being goaded into killing themselves after interacting with these chatbots. That's a problem.

You go to the doctor, and, I feel bad, but I think it's kind of funny too, you see in your summary that your symptoms came from microdosing mushrooms when you've never done mushrooms or talked about it. This can be much more serious, because I've also seen cases where they misdiagnose people. They say administrative burden is being removed from doctors, but it's being pushed onto the patients now. You have to really check what they're doing. This is super harmful. Why are we allowing this in our health care? Almost having World War III because of fake intelligence reports.

The climate catastrophe is, to me, the biggest one. At a time when we should be moving away from fossil fuels, they're accelerating, using more gas, trying to convince governments around the world to have more nuclear power and fast-track nuclear development for these data centers. Taking water away from people, polluting water. Last week I was at Oakland Tech Week, and my co-panelist was listing the forever chemicals that are being put into people's drinking water because of the construction of these data centers. Paris has a new book, Hyperscale, right, that describes that issue. This is a real problem.

I remember, even back in 2015, I was worried about police brutality, and I'm hearing from Elon Musk, who is up in the clouds talking about a fictional existential risk from nonexistent superintelligent gods. Your day-to-day, the things you know you should worry about, putting food on the table, war, those are problems people know exist. They're just being talked out of it. No, no, no, the data center is a distraction from this thing killing every single human. That's always a bigger problem than any other problem you can talk about, right? Which is also how they seize the narrative. Any problem I bring to the table, you can come to me and say, the idea that every single human being on Earth could get killed is always a bigger priority than anything you're currently talking about. So this is why I think it's really important to see this narrative for what it is.

Gil Durán: Now we're going to go to some questions from our listeners, and we're almost out of time, so this will be a speed round. Here we go with a speed round from our paid subscribers, who keep our entire operation afloat. Thank you guys so much. There were some really good questions, about 25 of them. Some overlapped, so I was able to consolidate, and here are the best ones we haven't already covered in the course of our conversation.

The first one's about data centers and the land grab. Are all of these data centers really necessary, or is there something else going on with the tech sector's hasty and nearly unregulated acquisition of physical space and natural resources?

Timnit Gebru: These data centers are not really necessary. I'm sure if you believe you have to create a machine god, you can convince people that any kind of resource is necessary. But these data centers are definitely not necessary. That's the short answer.

Gil Durán: Here's another question, about alignment, bias and gatekeeping. Is it actually possible for existing AI models to become aligned with human values, given how they were trained and the specific demographics driving them? And how can we combat the baked-in biases in AI that effectively discriminate in hiring, admissions and other gatekeeping activities?

Timnit Gebru: I encourage your listeners to completely reject the framing of alignment. This is a regular word that has been ruined for me by the people in the TESCREAL bundle. "Effective" is another normal word I don't like using now. Alignment with human values: that framing already assumes there is some other being, with agency, whose values need to be aligned with yours, which is ridiculous, because humans are building products. The humans building the products are the ones who have to be subject to certain rules and regulations, and the products themselves have to be subject to other kinds of tests. So the whole framing of alignment is pseudoscientific and bogus, and I encourage everyone to move away from it.

Gil Durán: This question is about tech hubris and finding allies inside. In these AI companies, have you ever come across anyone who thinks tech cannot solve every problem and actually considers social and political elements? How can we find allies among the tech workers who have recoiled from this wealth-sucking machine and want to dedicate themselves to the greater good?

Timnit Gebru: Actually, I think a lot of tech companies have people like that. The difference is that OpenAI and Anthropic are very different, because OpenAI was explicitly started with the goal of creating a machine god, and Anthropic, same. And Anthropic hires for the cult members. Anthropic's interview process specifically checks whether you're in the cults. I've even said there's a higher likelihood of me going back to Google, which obviously won't hire me back, than going to either of these companies. But the more mainstream they become, the more you'll see so-called regular tech workers who need a job going to these places.
When I was at Google, I organized with a lot of Google workers. The Google Walkout, the anti-Maven protests: they came down hard on a lot of people. Claire also has a book, right? Don't Be Evil. She was one of the Google Walkout organizers. The Tech Workers Coalition is another organization. There are a lot of tech workers who don't like what's happening, but the job market right now is so hard, and I think tech workers don't have the kind of leverage they used to. But I think you can find such allies. Sabotage.

Gil Durán: Next question. We're constantly fed the messaging that AI is inevitable. Embrace it or be left behind. Which aspects of this actually are inevitable? And regarding the fear of AI taking jobs, is it fair to assume these companies would ruthlessly eliminate human payrolls because it's their only path to profitability?

Timnit Gebru: I don't believe any of it is inevitable. That's the short answer. Even asbestos, we were able to ban. Emily Bender, my collaborator, has a great analogy to plastic. Plastic is hard to avoid now, and that's kind of what they want this technology to be, but it's not like that yet. So any kind of friction we can put in is helpful. And even if it were heavily proliferated, we have agency, and if we organize, we can roll things back. So that's the first answer: nothing is inevitable.
Second, if your boss thinks they can make a lot more money by eliminating you and not having to deal with pesky workers, they'll do it. I don't understand what the endgame is, because a lot of them are realizing they can't do that, right? A lot of the people who fired a whole bunch of workers because they thought AI could do everything are starting to rehire them. I was trying to explain to my mom what the goal is. The goal is to just not have any workers and just sell products. And my mom said, then who buys the products? Nobody has money. And then she said, well, that's why they talk about UBI. They want us all to be on their allowance. They want us to have an allowance that they control, so that all we do is take this little allowance and go buy stuff from them. Which is why we shouldn't fall for this narrative.

But of course, if the companies think it's going to let them do it, they'll try. The point is that you can resist in your workplace. We have the Luddite Lab. I don't know if you know about the Luddite Lab at DAIR, where we have case studies of all these workers who are resisting, who got really good contracts and so on. So I don't think you should fall for any narrative that you're going to be left behind, or that you have to adopt it, or that you have to accept anything that's harmful to you.

Gil Durán: It's always amazing to me that the AI companies can promise communism, but democratic socialists can't talk about health care and fair wages in American politics. What is promised about the future in these fantastical scenarios would never come true. Why would a bunch of billionaires become trillionaires and then share, when they're not sharing the abundance we already have? It's not a technical problem. It's a moral problem. And again, they don't understand things outside their little purview. And that marketing is suspicious, at the very least. Another question here: medical professionals often say there's great value in using AI in their fields. Why does the reporting from medical domains appear so different from the highly error-prone results we see AI producing?

Timnit Gebru: Well, what I'm seeing right now is a lot of error-prone issues in medical fields. I don't know who is auditing them. I just gave the example of the woman in Australia who was told her symptoms were from microdosing mushrooms when she had never done that. That was just one example, and there are all sorts of others. What I've noticed in the medical field is that the nurses' unions are very politically educated and haven't bought the hype, whereas the doctors are in a different situation. The doctors have very much bought into the hype. So I think it also depends on where you are in the food chain, how close you are to the patients, how much you're interacting with patients versus working at a higher level.

I also think there are many different kinds of things going on. Software that helps you with early diagnosis, with analysis of some image, is very different from a chatbot supposedly recording your conversation and then summarizing it. So there might be some kinds of technology that are helpful in the medical domain, but we can't conflate them with the kinds of chatbots we're talking about. There are different kinds of things going on there.

Gil Durán: Is there any ethical way to use AI?

Timnit Gebru: By AI, you mean the Claudes and the ChatGPTs?

Gil Durán: And the generative...

Timnit Gebru: I've talked to some people about this. One of my friends is deaf and blind, and she told me she uses some of these LLMs to navigate websites, because websites aren't screen-readable. What we really want is for websites to be screen-readable and for people to care about accessibility, right? So I don't fault people for using whatever tool is at their disposal if it makes their life easier. It's not an individual problem. That would be kind of like telling individuals to recycle. However, right now, whatever the use of this tech is, it was built on stolen labor, on exploited labor around the world, and on killing the environment. Each legitimization of these technologies that we do right now is marching us toward an actual existential catastrophe, which is killing the environment. So I really think resistance and refusal and friction are the way to roll things back and help us not get into this actual existential catastrophe.

Gil Durán: The next one's geopolitical. Is the framing of AI as a race with China just xenophobic propaganda? How does Silicon Valley's godlike AI hype compare to Chinese ideas about a more modest, practical AI ecosystem, where the resources of the nation are viewed as public goods? That's a very characterized question, but I think you get what they're getting at.

Timnit Gebru: Yeah, yeah, because I've written about this. In our TESCREAL bundle paper, we talk about this whole framing of it as a U.S.-China narrative, and the whole existential risk narrative is part of that. It's telling people: if you don't let us build the good kind of AI, China is going to get ahead of us and build the bad kind of AI. Of course, both the U.S. and China are doing bad things, as they're both empires doing empire things. But what I'm seeing is that the Chinese aren't talking about gods and mystical machines. They're talking about deepfakes. Actually, this existential risk story is [about] trying to ban open-weight models, trying to say they're dangerous, because the open-weight models coming from China are showing how bogus the business model of these U.S. companies is. This framing of it as a U.S.-China thing is very much corporate propaganda that lands because of xenophobia and because of the Cold War narrative.

Gil Durán: It struck me, too, when they were calling for this pause recently, that it's never going to happen, because you've already created this framework that China will win if we stop. So what's really going on here? You've already created the reason you can't stop, so there's some other interesting play in the middle of that.

Timnit Gebru: They know it. They're not going to pause. They're just saying that publicly, because privately they want the U.S. military to have a contract with Anthropic or a contract with OpenAI. That's what they really want.

Gil Durán: Final quick questions here. If you could ban one AI buzzword for a year, which would it be? And finally, was there ever a day in your career when you were not mansplained to?

Timnit Gebru: The second one is probably a no. There was a time when I didn't know the word mansplaining, and that made me go crazy, because I knew it was happening but there was no word for it, so you second-guess yourself. But there was never a time when it didn't happen.
If I could ban one word, I think artificial general intelligence, AGI, would have been the one. But now we've upgraded to superintelligence. So I'd say it's a tie between superintelligence and artificial general intelligence.

Gil Durán: Do you think intelligence is even the right word?
Timnit Gebru: Oh, God, no. I had to choose one, so that's why I went with those. There are plenty of words on my list I'd like to ban. No, intelligence is not even the right word. You know the reason we're stuck with the term AI is that a bunch of guys had a workshop and they wanted to not invite another guy, so they had to come up with a new word. And now we're all calling it that word. So I'd rather not even use that term. ASI, artificial superintelligence, all of these ridiculous terms. We may not have the power to ban them, but we do have the power to not use them.

Gil Durán: Timnit Gebru, thank you for joining us here on the Nerd Reich podcast.

Timnit Gebru: Thank you. It's really great having this conversation with you.

R.R. Robbins: And that's the show. Thanks to our guest, Timnit Gebru. Support her work at the Distributed AI Research Institute. That's dair-institute.org. DAIR, D-A-I-R. And pre-order her book, Deep Unlearning: The Rise of AI and the Radicalization of a Tech Idealist. Also, if you haven't, buy Gil's book, The Nerd Reich: Silicon Valley Fascism and the War on Democracy, a New York Times bestseller.

Both are available at Bookshop.org or wherever you buy books.

Today's episode was written and hosted by Gil Durán. It was produced, edited and announced by me, R.R. Robbins.

Additional editing by Chee Hui Lim. Please subscribe to our newsletter at TheNerdReich.com.

Your paid subscriptions make all this possible. Also, subscribe to our channel at The Nerd Reich News on YouTube. And if you get a chance, leave us a review on Apple Podcasts or Spotify. Believe me, those help. Today's final words from Descartes:

Magpies and parrots can utter words like ourselves, and are yet unable to speak as we do, that is, so as to show that they understand what they say.

See you next time.