Anthropic has proposed federal AI safety legislation that would establish catastrophic harm thresholds for model deployment whilst simultaneously restricting government agencies’ ability to intervene without judicial approval, according to documents reviewed by TechCrunch AI.
The proposal, which sets a threshold requiring action when AI models pose risks that could kill 50 or more people, includes a provision mandating that federal agencies obtain court orders before blocking model deployments—a requirement that legal experts warn could introduce dangerous delays in emergency situations.
The legislative framework represents an unusual approach to AI governance: whilst appearing to strengthen safety standards through specific harm thresholds, it simultaneously constrains regulatory authority in ways that could benefit the companies it purports to regulate. The requirement for judicial intervention before enforcement action marks a significant departure from existing regulatory frameworks in sectors such as pharmaceuticals and aviation, where agencies possess emergency powers to halt dangerous products.
The 50-person fatality threshold itself raises questions about risk assessment methodology. The proposal does not specify whether this refers to immediate casualties from a single incident or cumulative deaths over time, nor does it address non-fatal catastrophic harms such as critical infrastructure failures or economic collapse. This ambiguity could create substantial legal grey areas that delay enforcement whilst interpretations are litigated.
The business implications are considerable. Requiring judicial approval before model blocking effectively creates a higher bar for regulatory intervention than exists in most safety-critical industries. This could accelerate deployment timelines for frontier AI companies, reducing compliance costs and legal uncertainty. However, it also shifts liability risk in unpredictable ways—if courts consistently deny agency requests to block models that later cause harm, tort litigation against developers could intensify.
Anthropic, which has raised over $7.3 billion in funding and competes directly with OpenAI and Google DeepMind, stands to benefit from a regulatory framework that constrains agency discretion. Smaller AI developers with fewer legal resources, however, may find themselves disadvantaged if compliance requires navigating both agency oversight and judicial proceedings.
The proposal arrives as Congress considers multiple AI safety bills, including California’s recently vetoed SB 1047, which Anthropic publicly opposed despite initially supporting its aims. That bill would have established developer liability for catastrophic harms without requiring judicial intermediation for state enforcement actions.
Critics argue the judicial requirement could enable regulatory capture by design. “Requiring a court order before an agency can act on imminent safety threats inverts the normal regulatory process,” said one policy analyst familiar with the proposal who requested anonymity. “It’s akin to requiring the FDA to obtain judicial permission before recalling contaminated food.”
The proposal also lacks clarity on international coordination. AI models deployed overseas can affect domestic safety, yet the legislation appears focused solely on US-based enforcement mechanisms. This jurisdictional limitation could create regulatory arbitrage opportunities, with companies deploying high-risk models through foreign subsidiaries to avoid US oversight.
The timing coincides with increasing pressure on AI companies to demonstrate commitment to safety governance whilst maintaining rapid development pace. Anthropic has positioned itself as a safety-focused alternative to competitors, making its legislative proposals particularly influential with policymakers seeking industry input.
Whether this framework gains traction will depend largely on how Congressional committees balance industry preferences against regulatory effectiveness. The judicial approval requirement will likely face scrutiny from lawmakers concerned about agency independence, whilst the specific harm thresholds may attract debate over whether quantitative triggers can adequately capture AI risk.
Observers should monitor whether other frontier AI developers endorse similar frameworks, which would signal coordinated industry strategy to shape regulation. The response from federal agencies—particularly those with existing AI oversight responsibilities—will indicate whether this approach can survive inter-branch negotiations or whether it represents an opening position in longer legislative bargaining.







