Stories · 23 min read

Dario Amodei: Is It Too Late for Humanity to Wake Up?

“Humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it.” So says perhaps the most talked-about tech leader of the moment, and he has good reason to. “He hates Western civilization,” wrote Musk. “A liar with a God complex,” snapped Under Secretary of War Emil Michael. “Leftwing nut jobs,” added Trump. All words from the past few weeks, aimed at today’s hero. Who is this fearsome enemy? You would hardly guess that it is a level-headed, mild-looking physicist from San Francisco who simply announced that he will not allow his company to be used for building autonomous weapons or for mass surveillance. Dario Amodei, the creator of the most advanced AI models, has finally landed where he deserves to be—in the spotlight. And not only because he stood up to the most powerful state on earth at the real risk of destroying his multibillion-dollar company. Not only because, in recent years, Anthropic’s models have pushed artificial intelligence to incredible heights. But also because he dared to peer into the future. The place we are all heading toward, where few dare to ask the truly important questions. Such as—how do we survive as a civilization in the age of super-advanced artificial intelligence?

Translated from the Bulgarian original →
Dario Amodei

“He hates Western civilization,” wrote Musk. “A liar with a God complex,” snapped Under Secretary of War Emil Michael. “Leftwing nut jobs,” added Trump. All words from the past few weeks, aimed at today’s hero.

Who is this fearsome enemy? You would hardly guess that it is a level-headed, mild-looking physicist from San Francisco who simply announced that he will not allow his company to be used for building autonomous weapons or for mass surveillance.

If you met him, you would think—your typical tech expert. Shirt, glasses, nobody would accuse him of excessive charisma. As a child he loved physics and mathematics more than anything. He lost his father to a disease that, just four years later, was already being cured successfully. A man with the power to foresee the development of artificial intelligence… while building it. But also to warn of the enormous dangers the technology carries. An entrepreneur who built a $380 billion company from nothing and who says that one day he will give away almost all of his money.

Dario Amodei, the creator of the most advanced AI models, has finally landed where he deserves to be—in the spotlight. And not only because he stood up to the most powerful state on earth at the real risk of destroying his multibillion-dollar company. Not only because, in recent years, Anthropic’s models have pushed artificial intelligence to incredible heights. But also because he dared to peer into the future. The place we are all heading toward, where few dare to ask the truly important questions. Such as—how do we survive as a civilization in the age of super-advanced artificial intelligence?

Dario Amodei

The Billionaire Physicist

“I believe we are entering a rite of passage, both turbulent and inevitable, which will test who we are as a species. Humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it.”

“The Adolescence of Technology.” That, according to today’s hero, is as far as we have come as a species. It is also the title of the essay with which Dario Amodei caught everyone’s attention in January, publishing it on his own website. In a few thousand words he managed to gather many of the questions that matter more today than all our topics of the day. The very questions that are the reason the site you are reading exists.

Then came the unexpected storm. The entire American state machine rose up in full force against a relatively young artificial intelligence company. A rather unexpected and unusual phenomenon in times when tech leaders are the ones who set both the public conversation and the great challenges of the day.

If it has not arrived yet, it is very close: the day when programmers will no longer write computer code. It was Anthropic’s Claude models that turned out to be the game changers, along with the unexpected and successful applications through which they are used. Claude gradually became an ever more competitive, and then a leading, technology compared with its “cousin” ChatGPT, more popular in the headlines. Until we reached the day when the company that makes it collided with the words we started with.

Dario Amodei

Enemy at the Gates

“Humanity needs to wake up, and this essay is an attempt—a possibly futile one, but it’s worth trying—to jolt people awake.”

* The quotes are from the essay “The Adolescence of Technology”, translated by Claude Opus 4.6

Of course, we do not know much about the negotiations between the American state and today’s hero, but the harsh labels speak volumes. For one thing, about what artificial intelligence means today. Not in the future and not as entertainment. If it is so much a part of the national security of the biggest state on earth; if it matters so much to the United States that one company’s refusal to sell its technology becomes such a huge story, then we are already where Amodei’s words are sending us.

What’s more, for all the strong words and epithets, nobody denied that the reason the state declared today’s hero an enemy was precisely his refusal on the two points we started with—his wish to explicitly forbid Anthropic’s models from being used for surveillance and killing.

OpenAI, for instance, accepted the Pentagon’s terms and signed a cooperation agreement. It is easy to assume they were hardly that picky about those particular terms. An online movement even appeared of people who stopped using its products because they are planting very serious mines in the path of freedom in the age of artificial intelligence, but… on that subject, Orwell has the most to say.

In a Programmer’s Skin

“This vacillation is unfortunate, as the technology itself doesn’t care about what is fashionable, and we are considerably closer to real danger in 2026 than we were in 2023. The lesson is that we need to discuss and address risks in a realistic, pragmatic manner: sober, fact-based, and well equipped to survive changing tides.”

Dario was born in 1983 in San Francisco. His family is straight out of a novel by Dimitar Talev, the great Bulgarian chronicler of family sagas. His father, Riccardo Amodei, was an immigrant from the Tuscan town of Massa Marittima who made a living from his small leather workshop. In America he met Elena Engel, who came from the Jewish diaspora and worked on designing libraries.

Dario spent his childhood in the Mission—a colorful, multicultural neighborhood in the heart of the big city. As he himself says, his parents passed on to him “a clear sense of right and wrong, an understanding of the things that really matter in life.”

From an early age he was captivated by mathematics, and later he met its more flirtatious sister—physics. And although he belongs to the generation that grew up alongside the rise of technology, it hardly struck him as interesting. “Writing some website seemed super unappealing to me,” he recalls. “I was interested in discovering fundamental scientific truth.”

This is what sets him apart most sharply from all the countless tech leaders whose Digital Stories we have peeked into—he is driven forward by science, so much so that in 2000 he made the national team for the International Physics Olympiad.

Dario Amodei

The Physics of Sorrow

“There is now ample evidence that AI systems are unpredictable and difficult to control—we’ve seen behaviors as varied as obsessions, sycophancy, laziness, deception, blackmail, scheming, and ‘cheating’ by hacking software environments. AI companies certainly want to train systems to follow human instructions, but the process is more an art than a science, more akin to ‘growing’ something than ‘building’ it.”

He set off down the well-trodden scientific path, studying physics at the California Institute of Technology and Stanford. And perhaps he would have carried on with deep scientific research, and none of us would have had the chance to meet Claude Code, had it not been for a personal tragedy.

Dario was 23 when his father died of a rare form of cancer. As the physicist would often recount afterward, four years later there was already a treatment for this type of tumor that gave a better than 99 percent chance. “There was someone who worked on the cure for this disease, who managed to cure it and save a great many lives,” Amodei says. “But it could have saved even more.”

The loss made the young scientist change the direction of his research. Slowly but surely he steered it toward artificial intelligence, because he saw in it a technological opportunity to heal people far faster and more effectively and to make life better.

His first step, however, was not computers but… getting to know the human brain. Dario decided to go on to graduate studies in biophysics at Princeton. He studied the electrophysiology of neural circuits, trying to understand how the brain works by examining the electrical impulses running through it.

Dario Amodei

Ungifted

“AI models are trained on vast amounts of literature that include many science-fiction stories involving AIs rebelling against humanity. This could inadvertently shape their expectations about their own behavior in a way that causes them to rebel against humanity. Or they might decide that it is justified to destroy humanity because humans eat animals or have driven certain species to extinction.”

Today the term “artificial intelligence” rolls through our feeds day and night, but only a few years ago scientists took great pains to avoid it. Tellingly, the huge progress in this direction came from researchers who know the natural kind of intelligence very well, and how our brain works—starting with the “Godfather” Geoffrey Hinton and ending with Dario Amodei.

Next came a postdoc connected with tumor research. And if that was an unexpected turn away from physics, the next one was even more abrupt, and more important for the story that has brought us together. Dario came across the books of the futurist Ray Kurzweil, who with his ideas about the exponential growth of technology won him over to the world of artificial intelligence.

A physicist with a background in medicine, Dario spontaneously sent his application to a job ad at the AI lab of the Chinese tech giant Baidu. And he got lucky. As his first boss at the new workplace put it, at first it seemed extremely strange why a physicist would even apply for a position in artificial intelligence development… until they saw the way Dario had written the computer code for his research.

At the Chinese company the physicist quickly found his feet in the new profession and formed his worldview about it. There he became one of the leading champions of the idea of scale—that in the world of artificial intelligence, much more data and greater computing power can lead to very serious breakthroughs, even without any further big conceptual leaps. A view often criticized by other great scientists in the AI world, but one that, for the moment, seems to keep paying off with every new generation of generative language models.

Dario Amodei

GP-T, You’re the Star

“We are on the cusp of the further perfection of extreme evil, an evil whose possibility spreads well beyond that which weapons of mass destruction bequeathed to the nation-states, to a surprising and terrible empowerment of extreme individuals.”

From Baidu, Dario passed briefly through Google, until in 2016 he took the step that would place him at the center of one of the most important technological revolutions in human history—he joined OpenAI. Yes, although today it lives up to neither its name nor its idea, the “mother” company of ChatGPT was founded by people such as Elon Musk and Sam Altman with the official mission of developing artificial intelligence that would benefit all of humanity.

When Dario arrived, it was just picking up speed, and he quickly established himself as a key figure—rising to vice president of research, working shoulder to shoulder with the legendary Ilya Sutskever and taking the lead on projects that would shake the world.

The first of them was GPT-2. The model showed impressive text-generation abilities, but OpenAI decided not to release it in full for fear of misuse. GPT-3, however, came along to change everything (as, incidentally, we discussed on this site, long before ChatGPT appeared and the world realized that this had happened).

Although the breakthrough did not reach the general public in its full force, specialists were impressed by the model’s ability to program, summarize and translate. Even the reserved Amodei did not hide his enthusiasm: “There’s something extraordinary about it,” he told the New York Times. “It has some ability to recognize the pattern you’ve given it and to finish the story.”

Even Bird’s Milk

“A sufficiently powerful AI could probably be used to compromise any computer system in the world, and could also use the access obtained in this way to read and make sense of all the world’s electronic communications. It might be frighteningly plausible to simply generate a complete list of anyone who disagrees with the government on any number of issues, even if such disagreement isn’t explicit in anything they say or do.”

Somewhere around this point, though, along with the great enthusiasm came great doubts. If you don’t believe me—ask Dr. Frankenstein… or is the comparison not quite accurate? Only time will tell.

For the first time, Dario realized that artificial intelligence is a powerful weapon that is only at the dawn of its development. He could already predict where the technology was going—and how dangerous it could become if not handled properly. “Dario got into this technology and understood that it was going to work. And if you assume that it’s going to happen and that it’s going to get as smart as a human, you can’t help but start worrying about safety,” recalls his colleague Jack Clark.

OpenAI’s partnership with Microsoft, announced in 2019, with a multibillion-dollar investment and exclusive commercialization rights, became a turning point. For our physicist and his colleagues, it was a sign that AI was now developing as a race rather than as a collective scientific effort.

He describes the decision to leave with his typical directness: “There’s a lot of misinformation about why we left. People say it was because we didn’t like the Microsoft deal. Wrong. If you have a vision for how something should be done and for the principles an organization should stand on, it is incredibly unproductive to try to change things from the inside. Better to go and build your own vision.”

The Tale of the Staircase

“People today worry a lot about the potential influence of TikTok as CCP propaganda directed at children. I worry about that too, but a personalized AI agent that gets to know you over years and uses its knowledge of you to shape all of your opinions would be dramatically more powerful than this.”

That is exactly what he did. Well, once again it is time to mention the irony of fate: OpenAI quickly turned into a fearsome dominator and a classic mastodon of the digital age. And Dario left it to create Anthropic—a company that is also far from any notion of public good, equal access and open technologies. Then again, as the recent news we started with shows, there are substantial differences in the kinds of compromises Dario and his colleagues are willing to make, compared with his former boss Sam Altman.

Then again, maybe it is not too late for Sam to follow his example, who knows? “If you can build a company that people want to work for, following practices people find reasonable, and you manage to hold your position in the ecosystem—others will start copying you,” Amodei believes. “When business leaders see that a competing company has chosen a winning strategy, it makes them change their behavior far more convincingly than if you were their boss arguing with them.”

At the very end of 2020, Dario Amodei, his sister Daniela and a group of researchers left OpenAI, which the company announced in a terse press release. “Dario has always shared our goal of responsible AI,” wrote Altman, and Dario himself was equally gracious: “I want to thank Sam and everyone. I’m really proud of the work we did together.”

Dario Amodei
Dario with Japanese Prime Minister Sanae Takaichi

Luncheon on the Grass

“The formulation I have come to is that we should use AI for national defense in all ways except those which would make us more like our autocratic adversaries.”

The first meeting of the future Anthropic team took place in a park in San Francisco. Fewer than twenty people, sitting in a circle on the grass. Who said revolutions have to be the work of large crowds?

Well, in a case like this it probably helps to have a billion dollars or so in investment, especially in a field like artificial intelligence… but that too was bound to happen, the enthusiasts in the park were sure. Full of faith, conviction and a sense of responsibility.

Among the first to put their trust in them was Google’s former CEO Eric Schmidt. “When you make an early-stage investment, you have practically no data. You don’t know what the revenue is,” Schmidt says. “You don’t know the market. You don’t know what the product is. So the decision mostly comes down to the people. I knew Dario was a brilliant scientist, he promised to hire other brilliant scientists, and he did. He promised to run a very small company… but that promise he didn’t keep. For better or worse.”

Like OpenAI, Anthropic was set up as a “public benefit corporation.” I don’t know about you, but to me that sounds like “vegan wolf,” yet officially it is considered a legal form that obliges the company to pursue not only profit but also a mission. In this case the mission is clear: to build reliable, interpretable and steerable AI systems. For the models to be guided by a set of principles, a kind of internal constitution that defines what is allowed.

The Italian Connection

“AI could displace half of all entry-level white-collar jobs in the next 1 to 5 years, even as it accelerates economic growth and scientific progress. AI isn’t a substitute for specific human jobs but rather a general labor substitute for humans.”

One of the people closest to Dario on this journey is his own sister Daniela, who became president of Anthropic. Her background is radically different from his: a bachelor’s degree in English literature, politics and music from the University of California. She worked at nonprofit organizations, in the field of global health. Then she spent five years at the financial company Stripe, during the period when it grew from 40 to 1,200 people. She is the organizer, the business strategist, the person who understands both politics and ethics from a rather different angle. Dario is the technologist, the visionary, the scientist. Daniela is the diplomat, the manager, the pragmatist. And instead of an explosive clash between the two… somehow reason prevails.

Many things set him apart from his colleague Altman, but if they are alike in anything, it is their public image. Neither could be reproached for rhetorical mastery, charisma or conversational skill. As the quotes you are reading show, the written word is definitely his form, at the expense of public speaking. But that doesn’t bother Dario in the least; as he says in one of his rare podcast appearances: “I try not to show up and speak out all the time, and I’m proud that it works. If people think I’m boring and unsociable, that’s exactly what I want. I’ve seen many cases of colleagues who get so tied up with everyone else that seeking the crowd’s approval destroys them.”

But don’t think we are talking about a sociopath—quite the opposite. According to Dario, as a certain predecessor of his also believed, cadres decide everything. He says he spends most of his time precisely on choosing the right people and communicating with them.

Dario Amodei

A Rare Bird

“We risk a situation where, instead of affecting people with specific skills or in specific professions, AI is affecting people with certain intrinsic cognitive properties, namely lower intellectual ability—which is harder to change. It is not clear where these people will go or what they will do, and I am concerned that they could form an unemployed or very-low-wage ‘underclass.’”

“The most important thing I do every day has nothing to do with training models or shipping products,” he says. Twice a month Amodei stands before the whole company at a meeting called DVQ—Dario Vision Quest. I won’t explain the little joke in the name in detail; I’ll only mention that it has associations with psychedelic trips. But these trips are pragmatic. At the meetings he talks for at least an hour on all sorts of topics—from product strategy to geopolitics—holding a few pages of documents in his hand.

He also keeps an active Slack channel with his colleagues, where he writes long, analytical essays that have become a kind of chronicle of the company. This culture of thinking in writing and open internal debate is a rarity in the tech world, and it stems from Amodei’s conviction that trust is the most valuable resource he has.

The Essay and the Fuss

“I think it is not too far-fetched to imagine AI companies, semiconductor companies, and their downstream customers generating around $3 trillion in revenue per year, being valued at around $30 trillion, and leading to personal fortunes well into the trillions. In that world, the debates we have about tax policy today simply won’t apply—we will be in a fundamentally different situation.”

In October 2024, Dario published an essay on his personal website that set off a wave of debate. He called it “Machines of Loving Grace,” a quote from a poem by the master of gloom Richard Brautigan.

Poetry about the future, what do you say? Apollo ex Machina.

In musings the length of a novella, Amodei paints a picture of a world in which powerful artificial intelligence is used properly. What’s more, he offers a concrete vision of what “properly” means. “I think that most people are underestimating just how radical the upside of AI could be, just as I think most people are underestimating how bad the risks could be,” he writes.

In the essay he goes through various fields in which AI could lead to very serious human breakthroughs—from medicine to democracy and human well-being.

The technology could speed up scientific research tenfold, make most infectious diseases preventable, stop cancer, extend our lives.

Perhaps the most provocative part is the one on geopolitics. Amodei advocates a strategy he calls an “entente”—a coalition of democratic nations that use AI to gain a decisive strategic and military advantage over their authoritarian adversaries, while sharing the benefits with all cooperating democracies.

Dario Amodei
Dario with Indian leader Narendra Modi

The New Entente

“I am saddened that many wealthy individuals—especially in the tech industry—have recently adopted a cynical and nihilistic attitude that philanthropy is inevitably fraudulent or useless. I can also make a pragmatic argument to the billionaires: it is in their interest to support a good version of tax policy. If they don’t support a good version, they will inevitably get a bad version designed by a mob.”

Is it strange, or only to be expected, that a little more than a century after the First World War, a great military conflict once again seems to hang in the air? So much so that one of the biggest tech leaders calls for treating AI as geopolitics and talks about an “entente”…

The “good guys” versus the “bad guys,” the authoritarians versus the democrats, almost like in Yuval Noah Harari.

A little naive, perhaps. But right or not, Dario is among the few who are genuinely trying to lead a meaningful conversation about the future. In recent years Anthropic has established itself as a huge force, as the place where change begins. At the start of this year, the same personal site published the second essay, excerpts from which you are reading here. Much more alarmist, much more balanced and, at least in my view, much more visionary.

The first speaks of dreams, the second—of dangers. Of the great change that is already here.

Once again Amodei divides his thoughts into five main risks. The first concerns the autonomy of AI systems—the danger that they develop goals or behavior incompatible with human ones.

1 to 5 Years

“Could powerful AI systems invent a new religion and convert millions of people to it? Could most people end up ‘addicted’ in some way to AI interactions? Could people end up being ‘puppeted’ by AI systems, where an AI essentially watches their every move and tells them exactly what to do and say at all times, leading to a ‘good’ life, but one that lacks freedom or any pride of accomplishment?”

The second category is misuse by individuals or small groups, with the possibility of creating biological weapons being especially worrying. The third is misuse by those in power—authoritarian governments carrying out unprecedented surveillance, deploying autonomous weapons and waging mass propaganda.

The fourth category is economic: Amodei warns that AI could displace half of entry-level white-collar jobs within one to five years. Wealth could become concentrated, social inequalities could explode.

The fifth category is the unknown. What we could not even guess, since we work with a familiar arsenal of scientific tools, worldviews and categories. Made for other, different times and customs.

Each of these dangers can be averted if humanity wakes up, Dario writes.

How could that happen in a polarized world dominated by social media?

He doesn’t give the answers, and I keep searching for them myself. Perhaps we can look forward to another essay. These are not written by artificial intelligence, but by Dario himself, on an old-fashioned, unpolished, rather ugly website. Interesting, isn’t it?

But it is almost always the questions that matter more.

Dario Amodei
This photo quickly became a meme. India’s leader Modi gathered the global tech elite, calling for unity… and only Dario and Sam Altman never wanted to hold hands.

The Deathly Hallows

“I can imagine, as Sagan did in Contact, that this same story plays out on thousands of worlds. A species gains sentience, learns to use tools, begins the exponential ascent of technology, faces the crises of industrialization and nuclear weapons, and if it survives those, confronts the hardest and final challenge when it learns how to shape sand into machines that think.”

The Pentagon story began in July 2025, when the American state signed a $200 million contract with Anthropic to develop AI for national security. Claude—Anthropic’s model—is the only one widely deployed on the classified networks of the Department of War.

But the problem arose in the negotiations over the terms, where Amodei insisted on the two “red lines.” No mass surveillance of American citizens, no fully autonomous weapons.

As Dario says, AI simply changes the rules, and that has to be in the minds of the people who develop it.

“The technology is advancing so fast that it is ahead of the law,” he says. “We don’t want to sell something we don’t think is reliable, we don’t want to sell something that could kill people.”

But then… is the Pentagon really the ideal potential partner?

The lines are thin; the rules really are different now.

The Gift of the Magi

“Whether we survive this test and go on to build the beautiful society described in ‘Machines of Loving Grace,’ or succumb to slavery and destruction, will depend on our character and our determination as a species, our spirit and our soul.”

As Dario admits, Anthropic’s position is paradoxical. “We occupy a peculiar position in the AI landscape: we believe that AI might be one of the most world-altering and potentially dangerous technologies in human history, yet we are developing this very technology ourselves.”

To many this is hypocrisy, but to Amodei it is logic. If powerful AI is going to be built—and he believes it will!—it is better for it to be in the hands of people who think and worry about the consequences.

Once upon a time that didn’t help Prometheus and Pandora. But now?

At the moment Anthropic is valued at $380 billion, has 2,500 employees and has raised more than $33 billion in investment.

The boy who loved physics now runs one of the companies shaping humanity’s future. And he says that “humanity needs to wake up.” Is it happening?

This digital story comes to remind us that, before and after artificial intelligence alike, freedom is at the tip of the spear. All that remains is to keep our fingers crossed that people like today’s hero don’t betray their principles. Alas, there will always be someone who will. That is why saving the drowning lies in the hands of the drowning themselves. At least for now.

Almost there…
If you liked the story, support the project. The site has no ads.
Support

Next story

All stories →