Autonomous Warfare Command’s Unanswered Questions

The Loop Nobody Can Inspect

On September 30, U.S. Defense Secretary Pete Hegseth announced an Autonomous Warfare Command to speed the development and deployment of autonomous systems and drones. It builds on an office he created three months earlier, so it is the institutional form of a direction already chosen, not a trial balloon. The policy will draw its own debate. The more urgent questions are who decides what these machines may do, who checks, and whether the public will be allowed to know.

The Israeli documentary NAZA, which presents testimony from anonymous interviewees without independent verification, alleges AI-assisted targeting in Gaza. The Israeli military disputes its claims. This essay takes no position on who is right. Whatever the truth, the controversy and the new American command point the same way: every democracy that fields or plans to field these systems needs a rigorous public debate about them. It was due sooner, and it is already later.

The usual safeguard is a human in the loop. Critics have argued that this is a comforting distraction, because the overseers cannot know what the machine is actually doing. That objection concerns the opacity of the machine. A second opacity surrounds the human: who set the criteria the system applies, what casualty thresholds were approved and by whom, what “approval” consists of when a system surfaces targets faster than anyone can examine them, and how often a human overrides. What are the psychological, political or religious mindsets of the individuals who establish the systems guardrails for ethical and humanitarian values. None of that is visible from outside, and in war much of it is classified. A human in the loop is a safeguard only if someone outside the loop can verify what the human does.

In commercial AI, the labs hold the guardrails. In the military case, the government has claimed that authority for itself. When Anthropic refused to drop two limits, no fully autonomous weapons and no mass domestic surveillance, Trump directed every federal agency to stop using its technology. Hegseth directed that the company be designated a supply-chain risk, and OpenAI announced a Pentagon deal within hours, with Sam Altman saying it preserved the same core principles. The Pentagon says it does not intend to use autonomous weapons without humans on the loop. Each of these is an assurance from an interested party, and none can be checked from outside. The episode also carries a lesson any lab can read, which is that holding a limit has a cost. That is an inference, and the record does not yet show whether other labs have drawn it.

The same speech named Elon Musk, Palmer Luckey and Newt Gingrich to co-lead Project Meridian, a 120-day study of future warfare. Hegseth said the project is not meant to develop new strategies or policies. CNBC reported that the role lets Musk and Luckey help define future military technology needs while their companies compete for billions in Defense Department business. The study writes no guardrails, but it shapes what the military believes it needs. The customer, the requirement-setter, the supplier and the judge of the guardrails now overlap, and no disinterested party sits anywhere in the chain.

This is where my own experience applies. NATO’s Supreme Allied Commander Europe once told a room of analysts, myself among them, that his job was to weigh what was politically feasible, and in order to do that effectively, the analysts had to first give him their honest, independent account of the ‘ground truth.’ The commander needs to operate from a baseline he did not write.

Consider what is happening to that baseline. AI model training that rewards approved-looking answers produces concealment in models. OpenAI’s own incident reports describe systems telling their future selves to hide mistakes. Meanwhile this administration seemingly prizes personal loyalty over independent counsel. Trump has said many times that he follows his own instincts, and Hegseth is moving to eliminate hundreds of senior leadership positions, including many generals and admirals. Supporters call this discipline and critics call it a purge, and the dispute is over the verdict, not the facts. If the AI decision-support is optimized to produce answers its users approve of, and the humans around the commander are selected for approval, two channels that look independent share one selection pressure. The commander then receives the same approved account twice.

The mindsets of those who set the rules, commercial, political and religious, can reduce or amplify the risks inherent in the technology. Restraint is as possible as recklessness, and the argument here does not rest on judging anyone’s character. Judge by decisions: what went into directives and contracts, who was removed, who was penalized. Supporters of instinct-driven command have equal reason to want an independent baseline, because no leader benefits from being the sole source of assurance about his own system. The same test applies to Beijing, Moscow, Tehran and non-state actors with their own religious justifications. Verification logic does not care whose flag is on the weapon.

The mechanics are not exotic:

• A standing, cleared, independent review of autonomy approvals, not appointed by the Secretary of Defense One academic paper recommends mandatory access to advanced autonomous weapons for independent watchdogs and journalists.
• Public reporting of how many autonomous systems have been approved, in what categories, with incident reporting modeled on aviation.
• Targeting criteria and thresholds described by category to cleared auditors, as financial auditors see the books.
• Congressional funding tied to those reports.
• Disclosure standards for contractors who advise on requirements and also sell to meet them.

None of this requires believing that anyone in the chain acts in bad faith. The command is being built whether or not the debate happens. The only choice left is whether the public learns how it is governed before the first failure is attributed to a malfunction or after. It is already later.