Anthropic pitches AI safety bill that may block a model if 50+ people could be killed

AI lab Anthropic has circulated a draft federal bill proposing that the U.S. government block the distribution of artificial intelligence models under specific dangerous conditions. According to Politico reports, the legislation would apply if a model poses a foreseeable risk of causing over 50 deaths or one billion dollars in property damage.

Those catastrophic threats must stem from biological weapons, cyberattacks, or models evading developer controls. The thresholds match those in California Senate Bill 53. The draft bill outlines a process where registered independent evaluators would analyze models for disaster potential. If evaluators find an imminent risk, they would have 72 hours to notify the U.S.

Commerce Department. The department would then need to ask a D.C. judge to halt the model's release, as federal agencies would be barred from stopping distribution independently. An Anthropic spokesperson confirmed the draft to Politico, stating it reflects proposals from the company's Advanced AI Framework. Questions remain regarding the evaluation system, including how to ensure that payments from AI labs to independent evaluators do not influence test results.