Governance

For years the governance debate among AI's frontier leaders was abstract: summits, frameworks, principles, the polite vocabulary of "international cooperation." Then in late February 2026 it got concrete, fast, and ugly. The Pentagon handed Anthropic a three-day ultimatum: agree to let the Department of War use Claude for "all lawful use cases" without limitation, or be designated a "supply chain risk," a label "normally used against foreign adversaries" (Dario Amodei, CBS News, Feb 28 2026). Anthropic refused. It got blacklisted. It sued. And in the space of a single week, the real fault line surfaced: not whether to govern AI, but who holds the pen... a private company, the executive branch, or a Congress that everyone admits is too slow to matter.

The red lines that broke the truce

Amodei's position is unusually specific, which makes it useful. Anthropic, he insists, "has been the most lean forward of all the AI companies in working with the US government," the first to put models on the classified cloud, deployed "across the intelligence community and the military" (CBS News, Feb 28 2026). It said yes to "98 or 99%" of the Pentagon's use cases. It said no to exactly two: domestic mass surveillance and fully autonomous weapons.

His argument on surveillance is the sharpest thing in the whole debate. Buying bulk data on Americans and analyzing it with AI "actually isn't illegal... it was just never useful before the era of AI." The technology, he says, "is getting ahead of the law." On autonomous weapons, the objection is partly technical (today's models have "a basic unpredictability... in a purely technical way we have not solved") and partly constitutional: the right of military officers "to make decisions about war themselves and not turn it over completely to a machine."

The Pentagon's counter, relayed through spokesman Sean Parnell, was flat: "We only allow all lawful use." Anthropic's exceptions were dismissed as "philosophical and 'woke'" (Axios, Feb 27 2026). The deeper objection, defense officials told Axios, was the principle itself: a private company "dictating how the U.S. government can deploy AI for national security purposes... during a technological race with China."

"Why should Americans trust you?"

The CBS interviewer asked the question that hangs over the entire field: why should a private CEO, not the federal government, decide these things? Amodei's answer is revealingly two-faced, and he knows it. In the long run, "I actually do believe that it is Congress's job." In the short run, "Congress is not the fastest moving body in the world and for right now we are the ones who see this technology on the front line."

So his fallback is the market, not democracy: "We are a private company. We can choose to sell or not sell whatever we want... they can use another contractor." Which is exactly what happened.

Enter Sam Altman, in a maneuver worth studying. OpenAI publicly endorsed Anthropic's red lines, "We have long believed that AI should not be used for mass surveillance or autonomous lethal weapons" (memo via Axios, Feb 27 2026), with 70 staffers signing a "We Will Not Be Divided" letter. Altman wanted "to help de-escalate things" (CNBC). It was, Axios noted, "the first time the nation's top AI leaders have taken a collective stand." And then, the same Friday night Anthropic announced its lawsuit, Altman cut his own deal with the Pentagon. The crucial difference: Altman framed the restrictions as already-existing law ("The DoW agrees with these principles, reflects them in law and policy"), while Anthropic's whole point was that the law hasn't caught up. Solidarity in the morning, replacement contract by midnight.

Governance as incentive design

Not all the governance talk is about guns and spies. Daniela Amodei opened a quieter front: the architecture of the products themselves. Anthropic's first Super Bowl ad (Feb 7 2026) was a swipe at ad-supported AI, and her reasoning was a governance argument dressed as a product decision. An advertising model creates an incentive to make the model sycophantic. "If you earn more money from having the customer's eyeballs on the model for a longer period of time, that's really not a great incentive for discouraging things like sycophantic behavior" (ABC News, Feb 7 2026). The analogy is explicit and damning: social media. "If you were a social media company 15 years ago and you could look into the future today... would you have felt good about what you saw?"

She's also the rare leader actively asking to be regulated, talking to "both California at the California state level and New York state level about the need for regulation including on topics related to child safety," noting Anthropic bars under-18 users because "we're just not sure enough about the impact... on kids' brains that are still developing." (Sam Altman's reply to the ad, per ABC: "We would obviously never run ads in the way Anthropic depicts them. We are not stupid.")

The verification economy and bureaucratic amber

Jack Clark supplies the structural-economics layer, and it's the most original thinking here. Reviewing "Some Simple Economics of AGI," he reframes the binding constraint of an AI economy as "human verification bandwidth: the scarce capacity to validate outcomes, audit behavior, and underwrite meaning" (Import AI 447, Mar 2 2026). The governance failure mode he flags is the "Hollow Economy," where "measured activity rises, but hidden debt accumulates in the gap between visible metrics and actual human intent." His policy upshot: "verification infrastructure and the pipelines that build human verifiers" must be "treated as public goods."

On government adoption, Clark is contrarian. Everyone assumes the state moves last; he bets the "non-scary, sharp parts of government" move surprisingly fast, because "governments desperately want growth, and they desperately want efficiency" (Conversations with Tyler, Feb 26 2026). His darkest prediction is political, not technical: a movement to "freeze a load of human jobs in bureaucratic amber" as a panic response, "driven by the chaotic winds of political forces," not reason.

Where they actually agree

Strip away the fight and a consensus appears. Demis Hassabis wants governance handled through "international summits" and "the scientific method... to build good guard rails and monitoring systems," insisting "this can't just be left to technologists" but needs "artists, social scientists and philosophers" (India AI Summit, Feb 18-19 2026). Shane Legg, who chairs DeepMind's AGI Safety Council, formalizes the categories: misuse, misalignment, accidents, and structural risks, the slow-motion governance failures where AI could erode "democracy in the world" without any single dramatic event (DeepMind, Apr 2 2025; TED, Dec 2023). Ilya Sutskever, characteristically, frames it as inescapable: "You may not take interest in politics, but politics will take interest in you. So the same applies to AI many times over."

They agree it's coming. They agree it needs rules. They cannot agree who writes them. And the Pentagon just demonstrated that when the question gets real, the answer is decided in seventy-two hours by whoever holds the contract.