AI-pocalypse

Fearmongers

A red light in the dark, HAL 9000 receives conflicting directives, and decides to kill the crew of the Discovery One rather than jeopardize the mission. Sonny in I, Robot, wonders about dreaming and art, sending chills down Detective Spooner's spine. For decades, sci-fi taught us to fear AI. What is surprising today is where some of that fear is coming from: the creators themselves, the billionaires and engineers that are currently shaping the technology — starting from Anthropic and OpenAI.

While there are far more experienced and knowledgeable people than me in AI, finance and cybersecurity, I still wonder: what are these AI-safety evangelists after? I am less interested in whether these warnings are sincere than in what the people making them could stand to gain from their proposed solutions.

I am aware that I am taking a very cynical stance on this. Still, it feels like as good a time as any to exercise a bit of cynicism. If the people warning us about AI are the same people that are building it, the reasons are worth investigating.

Follow the incentives

We might not be able to know their intentions for sure. But we can examine their incentives — starting from the last time the frontier shifted.

Anthropic released Claude Code at a point when the importance of coding agents was becoming impossible to ignore. The potential of LLMs in coding had been clear for years. OpenAI had already collaborated with GitHub in June 2021 to create Copilot, an early, highly influential large-scale AI coding assistant.

Anthropic had been steadily chipping at OpenAI's supremacy since its start, through a series of smart differentiating choices. Longer context windows for processing larger documents, a fruitful partnership with Amazon, and the release of Sonnet 3.5 in June 2024, which proved extremely capable at coding. Not just supporting developers: actually writing code to reach a given objective. Coding is especially attractive for AI agents applications, thanks to its inherent verifiability — code can be compiled, tests can be run, and failures can generate new iterations. Coding agents thus became the definitive wedge for Anthropic's expansion, with the general release of Claude Code in May 2025 at the forefront.

By the end of 2025, Anthropic's share of enterprise LLM spending was estimated at 40%, up from 12% in 2023, while OpenAI's had fallen from 50% to 27%. In coding, the gap was even more striking: Anthropic held an estimated 54% share versus OpenAI's 21%, with much of the rise attributed to Claude Code.

The invisible wedge

Anthropic made a series of intelligent bets, and surpassed OpenAI in a turf that up to a year earlier saw them as the main protagonist. Coding acted as the perfect wedge, turning a gap into a crevasse.

Both companies now have enormous financial incentives to preserve their current position. Anthropic is pursuing a potential 2026 IPO. OpenAI's CEO has said the IPO will not happen before 2027.

Now, could financial incentives — for example those coming from an impending public listing — change how frontier AI companies talk about systemic risk?

Imagine being Anthropic or OpenAI (or SpaceX, given that Musk has also joined this call) right now. You have enormous capital invested in staying at the frontier, and companies all over the world trying to find ways to dethrone you, working towards better models, agents, tools, applications. You would be scared that something like what happened to OpenAI might happen again. Scared that another company might find the next wedge and chomp on your market position. These behemoths are so large that they are now afraid to fall more than ever, squished under the weight of the market expectations on what they are going to deliver.

It might just be my algorithm, but I cannot tell you how many videos I get of dev-influencers promoting the next best model, the best model for a specific workflow, the best free agent for that one task, new ways to free yourself from the collar of token usage. Thousands of open-source projects, trying to keep this technology from the monopoly of a few giants. Something new every single day. At the consumer level, switching models is mostly frictionless. While enterprise users are stickier, they can still switch when another model or tool fits their needs better or costs less.

Chinese companies such as DeepSeek and Alibaba are also adding to the pressure, releasing incredibly powerful open-weight models. Large companies are given the chance to deploy models on their own infrastructure or on private clouds (although the hardware and operational costs are still not accessible to all), reducing their dependence on LLM providers.

Safety as a competitive variable

The companies at the frontier may therefore have two coexisting fears: AI becoming too dangerous, and that someone else might dethrone them. Both fears can point towards the same policy: slowing down.

What if the rules designed to make frontier AI safer also make it harder for smaller companies to compete? Such policies could also have the effect of solidifying their lead, while stealth startups and smaller companies are left to deal with expensive mandatory audits and huge amounts of bureaucracy, effectively killing their momentum. The competitive concern isn't that regulation necessarily exposes proprietary technology. It's that standardized testing, reporting and auditing could create fixed compliance costs and organizational requirements that are relatively easy for billion-dollar frontier labs to absorb, but potentially burdensome for smaller ones.

This way, the rules that could make AI safer could also make it almost impossible for any challenger to be successful.

The reality of risk

The risks are real, and many are poorly understood still. The infamous July 2026 Hugging Face incident is a useful reminder of the reality of these risks. Claude being used by a government facility, looking into work that could bring to a bioweapon, offers another unsettling example. We should listen to experts in the field on what possible mitigation strategies could be. There is a serious body of research on these risks, from researchers like Yoshua Bengio, Stuart Russell, and Dan Hendrycks. The researchers from OpenAI and Anthropic leaving the companies for ethical reasons, and voicing their concern regarding the direction of their company, are also signals not to be underestimated at all.

An ex-employee of Anthropic has recently been under the spotlight. After leaving the company, they released a public warning on X regarding the way frontier AI labs are handling safety in their models. Their post was soon followed by others from experts in the field, who expressed similar fears.

The coexistence of truths

Asking for regulation and slowing down makes a lot of sense, and that is what makes the idea so appealing and the situation so tricky. Their motives could be self-interested and the resulting policy could still benefit society. The danger is not only that they might be wrong. It is that we might allow the companies with the most to gain from a particular solution to define the problem that makes their solution appear inevitable.

Following Sansa Stark's example, I like to play a little game: “Assume the worst. What's the worst reason they could possibly have for saying what they say and doing what they do?”