The Race to the Top: Inside Anthropic, the AI Betting Its Utopian (and Doomed) Vision on Claude
Dario Amodei rarely sits still when he talks about artificial intelligence—and he talks about AI almost constantly. As Anthropic’s co-founder and CEO, he’ll spring up from a conference room chair and dart across the room to a whiteboard, scribbling sweeping hockey-stick graphs that trace machine intelligence’s trajectory toward near-limitless capability. More than once, he’ll run a hand through his curly hair, like he’s manually calming overstimulated neurons to avoid a system crash. His whole body thrums with urgency as he lays out what makes his company different from every other AI lab building cutting-edge models: Anthropic is chasing artificial general intelligence (what Amodei calls “powerful AI”) that will never turn against humanity. This won’t just be a safe tool—it’ll be the anchor of a technological utopia, he says. But for all Amodei’s outsized role at Anthropic, he isn’t the company’s most valuable player. That honor goes to a single-named entity, just like Beyoncé or Pelé: Claude, Anthropic’s AI model.
Fresh off the World Economic Forum in Davos, Amodei made headlines there for his bold prediction: in roughly two years, Claude and other leading models will outperform humans at every cognitive task. Barely settled back at headquarters, he and Claude are now confronting an unexpected industry shakeup: Chinese AI firm DeepSeek just released a state-of-the-art large language model, reportedly built for a tiny fraction of the billions Google, OpenAI, and Anthropic have poured into their top models. Suddenly, the reigning paradigm of cutting-edge AI—built on multi-billion-dollar investments in chips and energy—looked wobbly.
Amodei is the face of the “maximalist” AI approach that created this high-stakes, high-cost landscape. Back when he worked at OpenAI, he published an internal paper outlining a hypothesis he’d turned over for years: the Big Blob of Compute. While AI researchers already knew more data translated to more powerful models, Amodei argued that the data could be far more raw than conventional wisdom held. Feed models massive quantities of unrefined data, he proposed, and you’d drastically speed up the arrival of powerful AGI. Today, this theory is standard industry practice—and the reason leading AI models cost so much to build, leaving competition limited to a handful of deep-pocketed tech giants.
DeepSeek, a newcomer from a country blocked from accessing the most advanced AI chips, pulled off its breakthrough without a massive blob of compute. If powerful AI can come from anywhere, does that mean Anthropic and its peers are just computational emperors with no real competitive moat? Amodei insists DeepSeek doesn’t keep him up at night. He rejects the idea that more efficient models will let low-budget competitors leapfrog industry leaders. “It’s just the opposite!” he says. “The value of what you’re making goes up. If you’re getting more intelligence per dollar, you might want to spend even more dollars on intelligence!” For Amodei, getting to the AGI finish line safely matters far more than cutting costs. That’s why even after DeepSeek’s launch, OpenAI and Microsoft have announced plans to pour hundreds of billions more into new AI data centers and energy infrastructure.
What does keep Amodei up at night is how humanity can build AGI without destroying itself. That question was so urgent that it pushed Amodei and six other co-founders to leave OpenAI in the first place; they believed OpenAI under CEO Sam Altman would never prioritize safety enough. At Anthropic, the team is racing to set global safety standards for all future AI models, standards that will ensure AI helps humanity rather than destroying it. They hope to build an AGI so reliably safe, ethical, and effective that every other competitor will choose to follow their lead. Amodei calls this strategy the “Race to the Top.”
The Founders: A Sibling Mission to Save the World
That strategy hinges entirely on Claude. You’ll never spot Claude grabbing coffee in the Anthropic café or riding the elevator up to one of the company’s 10 office floors, but it’s been a core part of the project from its earliest days. Anthropic engineers trained and refined Claude, then used Claude itself to build better, more capable versions of the model. If Amodei’s dream comes true, Claude will be humanity’s guide and protector as we enter an age of unprecedented technological abundance. But a troubling question, raised by Anthropic’s own internal research, lingers: can Claude itself be trusted to stay aligned with human values?
Amodei’s co-founder is his younger sister, Daniela. Their parents moved from Italy to San Francisco in the 1970s; their father Riccardo, a leather craftsman from a small town near Elba, got sick and died when Dario and Daniela were young adults. Their mother Elena, a Jewish American born in Chicago, worked as a library project manager to support the family.
Even as a toddler, Dario Amodei was fixated on numbers. While other kids clung to security blankets, he was punching away at a handheld calculator. As he grew older, that fascination turned to a deep obsession with mathematics. “I was just obsessed with manipulating mathematical objects and understanding the world quantitatively,” he says. In high school, Dario loaded up on math and physics classes, while Daniela studied liberal arts and music, earning a scholarship to study classical flute. But both siblings say they’ve always shared a humanist drive; as kids, they played fantasy games where their goal was to save the world.
Amodei went to college planning to become a theoretical physicist, but quickly decided the field was too disconnected from real-world problems. “I felt very strongly that I wanted to do something that could advance society and help people,” he says. A physics professor doing research on the human brain sparked his interest in AI, and he began reading Ray Kurzweil’s work on exponential technological progress. He went on to earn a PhD in computational biology at Princeton, where his thesis won multiple awards.
In 2014, he took a job at the US research lab of Chinese search giant Baidu, working under AI pioneer Andrew Ng. There, he first grasped how massive increases in computing power and data could produce vastly more capable AI models. Even then, experts were raising alarms about AI’s long-term risks to humanity. Amodei was initially skeptical, but by the time he moved to Google in 2015, he’d changed his mind. “Before, I was like, we’re not building those systems, so what can we really do?” he says. “But now we’re building the systems.”
Around that time, Sam Altman reached out to Amodei about a new startup with a mission to build AGI safely and openly. Amodei attended the now-famous dinner at the Rosewood Hotel where Altman and Elon Musk pitched the idea to venture capitalists, tech executives, and AI researchers. “I wasn’t swayed,” Amodei says. “I was anti-swayed. The goals weren’t clear to me. It felt like it was more about celebrity tech investors and entrepreneurs than AI researchers.”
Months later, OpenAI organized as a nonprofit with the stated mission of advancing AI “most likely to benefit humanity as a whole, unconstrained by a need to generate financial return.” Impressed by the talent on board—including many of his former colleagues from Google Brain—Amodei joined Altman’s bold experiment.
At OpenAI, Amodei refined his Big Blob of Compute theory, and the implications of his work grew more frightening by the day. “My first thought,” he says, “was, oh my God, could systems that are smarter than humans figure out how to destabilize the nuclear deterrent?” Not long after, engineer Alec Radford applied Amodei’s big blob idea to the new transformer AI architecture, and GPT-1 was born.
Soon after, Daniela Amodei also joined OpenAI. Her path to tech was winding: after college as an English major and self-described Joan Didion superfan, she spent years working for international NGOs and in government before returning to the Bay Area to become an early employee at Stripe. Looking back, the development of GPT-2 was the turning point for both siblings. Daniela managed the GPT-2 team, and the model’s ability to generate coherent, paragraph-long text was an early hint of what superintelligence could look like. Seeing it run stunned Amodei—and terrified him. “We had one of the craziest secrets in the world here,” he says. “This is going to determine the fate of nations.”
Amodei urged OpenAI leadership not to release the full GPT-2 model immediately. The team agreed, and in February 2019 they only released a smaller, less capable version, writing in a blog post that the limited release was meant to model responsible AI stewardship. “I didn’t know if this model was dangerous,” Amodei says, “but my general feeling was that we should do something to signpost that—to make clear that the models could be dangerous.” A few months later, OpenAI released the full model anyway.
The Split: Why They Left OpenAI
Discussions around AI responsibility at OpenAI began to shift. To build future models, OpenAI needed hundreds of millions of dollars worth of digital infrastructure. To secure that funding, the company deepened its partnership with Microsoft and created a for-profit subsidiary that soon encompassed nearly the entire workforce. It was taking on all the trappings of a traditional growth-focused Silicon Valley startup.
A group of employees grew concerned about the company’s direction. Profit didn’t bother them, but they felt OpenAI was no longer prioritizing safety as much as they’d hoped. Amodei was among the most vocal dissenters. “One of the sources of my dismay was that as these issues were getting more serious, the company started moving in the opposite direction,” he says. He shared his concerns with Altman, who he says listened carefully and agreed with his points—but never made changes. (OpenAI declined to comment for this story, but has long maintained safety is a core company priority.) Gradually, the dissatisfied employees found each other and began asking if they were really still working for the greater good.
Amodei says when he told Altman he was leaving, the CEO repeatedly asked him to stay. Amodei realized he should have left sooner. At the end of 2020, he and six other OpenAI employees—including Daniela—quit to launch their own company.
Daniela says when she thinks of Anthropic’s founding, she remembers a photo taken in January 2021. The group of ex-OpenAI defectors gathered under a big tent in Dario Amodei’s backyard for their first official meeting. Former Google CEO Eric Schmidt was there to hear their launch pitch. Everyone wore COVID masks, rain poured down outside. Two days later, supporters of Donald Trump stormed the US Capitol in Washington, DC; the Amodeis and their colleagues had just pulled off their own, peaceful insurrection against OpenAI. Within weeks, a dozen more OpenAI employees left to join the new startup.
Schmidt invested, but most of the $124 million in initial seed funding came from backers tied to the effective altruism (EA) movement, a philosophy that urges successful people to direct their wealth to high-impact philanthropy. EA supporters have long focused on AI safety as an existential priority. The lead seed investor was Estonian engineer Jaan Tallinn, co-founder of Skype and Kazaa, who has poured billions into AI safety organizations. For Anthropic’s second funding round, which raised the total to over $500 million, the lead investor was EA advocate Samuel Bankman-Fried (now a convicted fraudster), along with his business partner Caroline Ellison. Bankman-Fried’s stake was sold off in 2024. Another early investor was Facebook co-founder Dustin Moskovitz, also a prominent EA supporter.
Anthropic’s relationship with EA has long been complicated. Daniela says, “I’m not the expert on effective altruism. I don’t identify with that terminology. My impression is that it’s a bit of an outdated term.” But her husband, Holden Karnofsky, co-founded one of EA’s largest philanthropic arms, is a vocal AI safety advocate, and joined Anthropic in January 2025. Many other early employees have deep ties to the movement: early employee Amanda Askell, whose ex-husband William MacAskill is one of the founders of EA, says, “I definitely have met people here who are effective altruists, but it’s not a theme of the organization or anything.”
Not long after the backyard founding meeting, Anthropic registered as a public benefit corporation in Delaware. Unlike a traditional for-profit company, its board is required to balance shareholder interests with the societal impact of its work. The company also created a “long-term benefit trust,” made up of independent members with no financial stake in the company, to ensure that the drive to build powerful AI never overrides the company’s safety mission.
Building Claude: The Paradox of Safe AI
Anthropic’s first order of business was to build a model that could match or outperform the work of OpenAI, Google, and Meta. This is Anthropic’s defining paradox: to build safe AI, the company must first take on the risk of building powerful AI. “It would be a much simpler world if you could work on safety without going to the frontier,” says Chris Olah, a former Thiel Fellow and one of Anthropic’s co-founders. “But it doesn’t seem to be the world that we’re in.”
“All of the founders were doing technical work to build the infrastructure and start training language models,” says Jared Kaplan, a physicist on leave from Johns Hopkins who became Anthropic’s chief science officer. Kaplan also ended up handling administrative work like payroll, since someone had to do it. The team named their model Claude to evoke warmth and familiarity; some also say the name nods to Claude Shannon, the founding father of information theory and an avid juggler and unicyclist.
As the creator of the big blob theory, Amodei knew the company would need far more than the initial $750 million to compete. He secured over $6 billion in funding from cloud providers—first Google, a direct competitor, then later Amazon. As part of the deal, Anthropic makes its models available to AWS customers. Earlier this year, after additional funding, Amazon disclosed in a regulatory filing that its stake in Anthropic is valued at nearly $14 billion. Some industry observers expect Amazon will eventually acquire or gain control of Anthropic, but Amodei says balancing partnerships with both Google and Amazon lets the company stay independent.
Before Claude launched to the public, Anthropic unveiled its signature approach to “aligning” AI with human values: constitutional AI. The core idea is to let AI police itself. While a model may struggle to judge the quality of an essay on its own, testing its responses against a clear set of shared social principles that define harm and benefit is far simpler—much like how the US Constitution, a relatively short document, structures governance for a huge, complex nation. In this framework, Claude acts as the judicial branch, interpreting its own founding constitutional principles.
The Anthropic team curated these principles from a range of existing documents: the Universal Declaration of Human Rights, Apple’s terms of service, and Sparrow, DeepMind’s set of anti-racist and anti-violence guidelines. They added a list of common-sense ground rules, a sort of AGI version of the basic life lessons people learn in kindergarten. As Daniela explains the system: “It’s basically a version of Claude that’s monitoring Claude.”
Anthropic also developed a second safety framework called the Responsible Scaling Policy (RSP), which is central to the company’s identity. RSP establishes a risk-level hierarchy for AI systems, similar to the DEFCON warning system for national security. Anthropic rates its current models at AI Safety Level 2: they require guardrails to manage early dangerous capabilities, like giving instructions for building bioweapons or hacking computer systems, but they don’t produce information that goes beyond what’s already available in textbooks or search engines. At Level 3, systems gain the ability to act autonomously. Level 4 and above haven’t been fully defined yet, but Anthropic says they will involve “qualitative escalations in catastrophic misuse potential and autonomy.” The company pledges not to train or deploy higher-risk models until it has built stronger safeguards into place.
Logan Graham, who leads Anthropic’s red team (the group that tests models for dangerous outputs), explains that every time engineers release a major model upgrade, his team runs it through a battery of tests to see if it produces dangerous or biased responses. Engineers then tweak the model until the red team signs off. “The entire company waits for us,” Graham says. “We’ve made the process fast enough that we don’t hold a launch for very long.”
By mid-2021, Anthropic had a working large language model, and releasing it would have generated massive industry attention. But the company held it back. “Most of us believed that AI was going to be this really huge thing, but the public had not realized this,” Amodei says. OpenAI hadn’t launched ChatGPT yet. “Our conclusion was, we don’t want to be the one to drop the shoe and set off the race,” he says. “We let someone else do that.” Anthropic didn’t release Claude to the public until March 2023, after OpenAI, Microsoft, and Google had all launched their own consumer-facing models.
“It was costly to us,” Amodei admits. He calls that period of hesitation a “one-off.” “In that one instance, we probably did the right thing. But that is not sustainable. If our competitors release more capable models while Anthropic sits around, we’re just going to lose and stop existing as a company.”
The Race to the Top: Does the Strategy Work?
The dilemma at the heart of Anthropic’s mission feels impossible to resolve: hold back on development and go out of business, or rush forward and put humanity at risk. Amodei believes his Race to the Top strategy solves this. The idea is idealistic: be a model for what trustworthy AI looks like, and other companies will follow your lead. “If you do something good, you can inspire employees at other companies, or cause them to criticize their companies,” he explains. Anthropic also supports government regulation to enforce safety standards; the company was the only major AI firm that did not oppose a controversial 2024 California AI safety bill that would have placed new limits on high-capacity models, though it did not actively campaign for it, and Governor Gavin Newsom ultimately vetoed the bill.
Amodei believes his strategy is already working. After Anthropic published its Responsible Scaling Policy, he says OpenAI faced growing pressure from employees, the public, and regulators to adopt a similar framework. Three months later, OpenAI announced its own Preparedness Framework. Meta released its own version in February 2025, and Google DeepMind also adopted a similar framework, with Google AI head Demis Hassabis saying
