Frontier AI Should Be Nationalized

The headline is the argument. Would any reasonable person want a private investor such as Peter Thiel or, in this context, Sam Altman to control the development and deployment of nuclear weapons? Every reasonable person would say ‘No’.

Why, then, are we leaving the development of another potentially catastrophic technology in private hands? The same principle should apply to frontier AI, perhaps even more urgently.

A nuclear weapon can destroy a city. A rogue AI agent, or a person using a powerful AI tool, might gain access to critical infrastructure and cause harm on a vast scale. Consider a water-treatment system: if an attacker could manipulate the amount of a chemical added to the water supply, the consequences could be catastrophic to millions. That is only one possible scenario.

The danger is harder to grasp because AI does not look like a weapon. We encounter it as a website with a minimalist interface. We ask a question, and it answers in words. That familiar experience makes it difficult to imagine how a website which chats with us could potentially kill millions. But it potentially can, in countless ways.
On a side note, that is why it is not helpful to talk about ‘AI destroying humanity’.
It is unsafe even if it kills ‘merely’ 50,000 people.

The best course of action is to nationalize frontier AI companies.

Frontier AI is, at its core, a research and development project. Los Alamos was also a research and development project, one devoted to building a technology with extraordinary destructive potential, conducted under the direction of the U.S. government. Frontier AI differs in two crucial respects:

1. Much of its development is undertaken by private companies for private interests;
2. Increasingly capable models can reach the public remarkably quickly.

In comparison, we do not ordinarily release a new medicine to the public merely because its developer believes it is ready. We require years of testing and regulatory review excluding COVID-19 vaccines). Yet frontier AI systems are being deployed while fundamental questions about their behavior and risks remain unresolved. These are not merely passive tools sitting in a toolbox. If it is only a tool, can you imagine thousands of hammers and drills leaving the toolbox independently and colluding together for a specific goal? Still think AI is merely ‘a tool’?

An Anthropic safety report published recently found the following uses of Claude as a ‘tool’:

• Russia used Claude for cyberespionage against Ukrainian targets, and created thousands of fake news stories that were distributed through Russian state-media outlets.
• China used Claude for cyberespionage targeting minority groups, exploit research and weapons-related software work, and its ‘AI companies’ like Alibaba and others were actively stealing Claude models by distillation.
• Iran used Claude to create and push propaganda, surveillance-related work and analysis of publicly available information about U.S. naval forces.
• In Yemen, a group used Claude to design and improve ballistic missiles.
• In Turkey, Anthropic traced an influence operation targeting the public.
• Mali state intelligence used Claude to develop a mobile-phone surveillance platform.
• Bangladesh used Claude for fabricated political news as part of an influence campaign.

And that would account only for the behavior the company detected and chose to report. We should not assume we are getting the full picture here.

An IPO could be disastrous, as it will create stronger incentives to prioritize growth and returns to shareholders. Anthropic’s founders may sincerely care about humanity and take safety seriously. But a company’s founders cannot guarantee that its priorities will remain unchanged as ownership, leadership, and financial pressures evolve. And even if the founders of Anthropic are ‘the best of us’ (based on their writings and their historical connection to ‘effective altruism’), who chose them to develop and control ‘modern nuclear weapons’? We cannot allow a situation where you can just incorporate and start developing.

Yes, you can claim the current U.S. President is not ‘one of the best’ and question his conduct and ethics, but the fact is in almost six years as president he has not used such weapons, and the different nations have not used nuclear weapons for military purposes for more than 80 years. That history does not prove that governments always act wisely, but 80 years is a pretty substantial period that proves control of dangerous technologies can work, on an international level.

It seems as if the leadership in some of these companies understand that and, with the help of PR, are pushing toward nationalization. The next stage might be showing the U.S. government the benefits and financial gains it might reap from owning these companies.

Regulation alone will not be able to keep pace with systems that change so quickly. Public ownership would make the government responsible for the entire process.

The best course of action is to have the U.S. government, write checks to compensate their investors. Not to allow IPOs to go forward.

If we assume Anthropic’s valuation is $1 trillion (it is not a fair valuation, but for the sake of argument), it is a massive check to write, but the U.S. debt stands currently on 40 trillion and counting, so another trillion to control the most advanced (and potentially dangerous) technology is a price it should pay.