California’s sweeping AI safety bill passes first major hurdle

SACRAMENTO, California — A California proposal to impose first-in-the-nation safety standards for powerful AI models passed a major hurdle by clearing the state Assembly on Wednesday, overcoming fierce resistance from tech companies and leading California House Democrats.

Senate Bill 1047 by San Francisco's state Sen. Scott Wiener would require the largest AI models to certify safety testing before deployment with the aim of protecting people from potential dangers like the creation of bioweapons. The bill has divided Silicon Valley with top figures such as Elon Musk supporting the measure as a way to help mitigate potential risks to the public, while opponents such as OpenAI argue its requirements would unduly burden developers, especially small startups.

Wiener has also faced strong opposition from Democrats in Congress who represent areas around Silicon Valley. San Francisco Bay Area Reps. Zoe Lofgren and Ro Khanna have voiced their dissent, along with former Speaker Nancy Pelosi , marking a major blow to Wiener, who intends to run for her seat after she retires. One of his closest allies, San Francisco Mayor London Breed, warned the measure would undermine the city's economy .

Lofgren, Khanna and Pelosi all suggested California should leave AI regulation to the federal government, but a bipartisan group of state lawmakers on Monday said they can't wait for the gridlock in Washington to give.

"If we don't do it, Congress is not going to act," said Assemblymember Al Muratsuchi, a Los Angeles-area Democrat. "They have failed to act to date and this is why California must take leadership in the first step to regulate AI."

Lofgren countered that argument in an interview before the vote on Wednesday, saying Wiener and other lawmakers who have assailed congressional inaction "don't know what they're talking about."

"There is a lot that has gone on and there's more that's going to happen," Lofgren said.

The lower chamber approved the bill on a vote of 41 to 9, with many Democrats not voting and Republicans lending a pair of crucial votes.

The bill now goes back to the state Senate for approval of amendments before heading to Gov. Gavin Newsom's desk. The tech-friendly governor has not publicly weighed in on the bill, but has previously voiced concerns about overregulating the state's homegrown tech industry and is facing pressure from Silicon Valley to veto it.

Wiener, after the vote in the Assembly, told reporters he's grateful for the diverse support on a "very, very reasonable piece of pro-innovation, pro-safety legislation," adding that he expects Newsom to seriously consider it.

"Newsom is a pro, and he is used to having tough bills where he gets a lot of pressure on both sides," Wiener said. "And so I know that the governor is very used to this, and he will do what he always does — which is to look at the facts and make a thoughtful decision."