Below, Kevin Roose shares five key insights from his new book, The AGI Chronicles: The Inside Story of the Race to Create an Artificial Superintelligence.
Kevin is an award-winning technology journalist, bestselling author, and co-founder of Machine Gods Media. He previously spent a decade as a technology columnist for the New York Times, where he co-created and co-hosted the popular podcast Hard Fork. He now co-hosts Machine Gods, an AI-focused podcast produced in partnership with NPR.
What’s the Big Idea?
For years, the world’s leading AI researchers have raced to build a technology that many of them fear. But the finish line keeps moving as AI becomes more capable. The truth is, we’re already living in the early, imperfect years of artificial general intelligence, and our institutions are dangerously unprepared for what comes next.
Listen to the audio version of this Book Bite—read by Kevin himself—in the Next Big Idea App, or buy the book.
1. The people most afraid of AI are the ones building it.
This is, in some sense, the founding paradox of the whole AGI story. You’d assume the people building AI are true believers who expect it will do amazing things, like cure cancer and end poverty. And some of them are. But a surprising number of key figures in this race got into AI because they were terrified. They read the arguments about what a superintelligent machine might do to humanity, took them seriously, and concluded that if this thing was going to be built no matter what, it had better be built by people who were scared of it and would take the appropriate precautions.
Ultimately, I came to think of the AI world as having three tribes. There are the scientists, who see AGI as the ultimate intellectual challenge and want to use it to discover new drugs and cure disease. There are the seekers, who see it as an evolutionary turning point for our species. And then there are the survivalists, who think that if this technology isn’t built by the good guys, then the bad guys will do it instead.
The trouble is that getting there first is what everyone is trying to do. And they all end up concluding that the only way to move forward responsibly is to accelerate. Dario Amodei, the CEO of Anthropic, left OpenAI partly out of concern that it was moving too fast, and then went on to build a competitor that had to move just as fast to stay ahead. He’s living inside the paradox, and so are many of the people in this book.
2. The secret was a big blob of compute.
For most of AI’s history, smart people assumed building an intelligent machine would require understanding the human brain, writing down the rules of reasoning, and encoding knowledge the way you teach a student. But what worked instead was almost embarrassingly simple.
Amodei outlined it in a 2017 paper he called “Big Blob of Compute.” Essentially, you take a neural network, feed it enormous amounts of data, and then throw a huge computer at it. The model then learns to recognize the patterns in that data the same way a human toddler does, by looking at a lot of examples until it clicks.
“What worked instead was almost embarrassingly simple.”
Amodei called this his Big Blob of Compute hypothesis, and it reoriented OpenAI’s research away from narrow applications and toward a more general approach. It’s what led to GPT-1, GPT-2, and ultimately to the large language models that we use today. This simple recipe is the engine of everything that follows.
3. Feeling the AGI is a key ingredient in AI progress.
In San Francisco, people talk about “feeling the AGI” as a kind of spiritual awakening that comes from living on the technological frontier, internalizing the enormously fast pace of progress, and building up to something unimaginably vast. The people who work at these companies believe that they are building something closer to an alien species than a product, and they treat their work as a humanitarian mission: deliver AGI safely into the world before someone else screws it up.
“Feeling the AGI” is what Ilya Sutskever, the former chief scientist of OpenAI, called this feeling. He’d stand up at company gatherings and lead chants. He burned effigies. “Don’t lose sight,” he said, “of what we’re actually building here. This is not a tool. This is something more like a being.”
Feeling the AGI is now pretty common. It’s why we see researchers from AI companies speaking up about their fears and why lawmakers and regulators are desperate to slow down this industry. But for many years, the people feeling the AGI were few and far between. They worked at a handful of small AI companies that were little known to the general public. And they shared this secret which led them to double down on their research. The people who felt the AGI early won. They became the people who now run some of the most powerful companies on Earth.
4. There is no finish line.
For years, everyone in the race to AGI operated under the unspoken assumption that the race would eventually end. The idea was that someone would train an AI model that was clearly as smart as the smartest humans. That person would walk out on a stage and say, “We have built AGI!” And the race would be over.
Some leaders even started making plans for after AGI. Sam Altman, OpenAI’s CEO, mused about handing over control of the company to an AGI and finding some hobbies for himself. Demis Hassabis, former CEO of DeepMind Technologies, told friends he would take a year-long sabbatical and wander the gardens in Princeton, New Jersey, the way Albert Einstein did. Dario Amodei wanted to spend more time at his family’s farm in Italy with his favorite horse, Calypso.
“Nobody agrees on what AGI is or how we’ll know that we’ve achieved it.”
But around 2025, I started hearing a different phrase in AI circles. They called this the Red Queen’s race. It comes from evolutionary biology, via Lewis Carroll, where the Red Queen tells Alice that it takes all the running you can do just to stay in the same place. That’s what the race to AGI has become. Nobody agrees on what AGI is or how we’ll know that we’ve achieved it.
Every time one company ships a breakthrough, the others match it within months, and the goalposts get moved again. At Google, Larry Page, the co-founder who had been thinking about AGI longer than almost anyone, reportedly told people he would rather see Google go bankrupt than lose the race. Meanwhile, other Google executives were saying that AGI was still five years or more away. And the CEO, Sundar Pichai, was barely using the word at all.
Everyone in AI is running as fast as they can, and nobody is getting anywhere relative to each
other. So unless this changes, or we slow the whole thing down, the post-AGI future of garden strolls and hobbies and Calypso the horse will have to wait.
5. AGI is here, whether we feel it or not.
By the definitions of AGI that people in 2016 were using—a machine that can do most cognitive
tasks about as well as most humans—I think we’ve already gotten there. Today’s AI models can write, code, analyze documents, pass professional exams, solve unsolved math problems, and tutor children. If you went back in a time machine with a laptop and a copy of ChatGPT or Claude to 2017 or 2018 and showed them to a thousand of the leading researchers in the field, they would not have hesitated to say this is obviously AGI.
The reason that it doesn’t feel like AGI is that this intelligence is jagged. AI systems are superhuman at some things and bafflingly bad at others. They can solve the Riemann hypothesis or disprove the Jacobian conjecture and then miscount the letters in a word or fail at basic arithmetic. That unevenness makes it easy to dismiss them. You just find the one thing they can’t do and conclude that it’s all hype.
“AI systems are superhuman at some things and bafflingly bad at others.”
But a jagged intelligence is still an intelligence. In some ways, our human intelligence is just as jagged. I’ve come to think that the useful definition of AGI isn’t about capability. It’s about generality: a single system that can be pointed at almost any problem and make real progress on it. By that standard, we’re not waiting for AGI to arrive. We’re living in the early, awkward, jagged years of it.
One of the biggest risks I see isn’t a robot uprising. It’s a world where the technology has already changed, and the institutions in charge of our society—governments, companies, schools, and communities—keep acting like it hasn’t. The people in this book have been feeling the AGI for years. Now it’s time for the rest of us to catch up.
Enjoy our full library of Book Bites—read by the authors!—in the Next Big Idea App:
