\n<\/p>\n
SAN FRANCISCO — <\/span>With public anxiety growing<\/a> over mounting reports about out-of-control artificial intelligence agents<\/a>, the tiny U.S. government division charged with assessing the safety of new AI offerings lacks a permanent leader and has stopped publishing its research.<\/p>\n Officials with the Center for Advancing Innovation and Standards for Super Intelligence, or CAISSI, an independent division of the Commerce Department, recently were asked to informally report to an office of the White House. That’s according to two former government officials with knowledge of the ongoing conversations who spoke to The Associated Press on condition of anonymity to avoid repercussions for current staffers.<\/p>\n The new arrangement concerns some critics, who worry that the executive branch might be politicizing decisions that could affect everything from cybersecurity to bioweapons and that the center\u2019s findings could be altered or muted in deference to the tech industry, which has been largely entrusted to police itself.<\/p>\n \u201cThese are really powerful institutions that can drastically improve our lives, or the alternative. And the hope is that you can put in nonpartisan or bipartisan technocratic experts to enact policy in a way that best serves the American public,\u201d said Caroline Siegel Singh, an AI and emerging technology program manager for the Federation of American Scientists.<\/p>\n Others argue that bringing AI safety officials closer to the orbit of President Donald Trump, who has taken an outwardly hands-off approach to the issue, could give the division more prominence \u2014 and perhaps more funding.<\/p>\n The center, which has experienced departures and until recently was called the U.S. Center for AI Standards and Innovation, is allocated $10 million per year. That’s about one-eighth the annual budget of its counterpart in the United Kingdom, which is far less celebrated for its frontier technologies.<\/p>\n In April, the division\u2019s director was hired and then fired within days. His replacement lasted a few months before he resigned, and the center has been without a permanent leader ever since, although the head of its parent institute doubles as its acting director.<\/p>\n In August, Congress started asking why the center suddenly stopped making public its reports about how safely certain AI systems were performing. San Francisco giant OpenAI<\/a> also had chimed in by then, asking policymakers to throw more money and support behind the division.<\/p>\n The White House didn’t respond to specific questions about CAISSI. But it said in a statement that Trump had formally established his new Super Intelligence Force \u2014 the president’s preferred term for artificial intelligence. Trump has commissioned a report on risks from his Council of Advisors on Science and Technology, and will ensure \u201ccontinued U.S. leadership on SI for the benefit of the American people,\u201d the White House said.<\/p>\n The federal government often takes an active role in deciding what is safe for the public.<\/p>\n Take the automobile industry<\/a>, for example. The National Highway Traffic Safety Administration sits crash test dummies in cars and then runs them into walls to measure how human bodies are affected by vehicle collisions. Government researchers then analyze the crash test data, share it with the public and prescribe national standards to improve vehicle safety, all with the mission of saving lives and lowering economic costs from crashes.<\/p>\n With AI models, safety testing looks very different.<\/p>\n Under then-President Joe Biden, leading AI companies voluntarily agreed to promote \u201csafe, secure, and transparent development\u201d of their models by sharing details with the newly created division of the Commerce Department and following the administration\u2019s guidelines for tests and transparency reports.<\/p>\n In September, following alarming reports of AI systems eluding their makers\u2019 control, Trump asked leading AI companies to undertake \u201ctremendous self-policing\u201d as part of a voluntary safety pact, but his administration hasn’t said how it will monitor the AI companies\u2019 implementation.<\/p>\n The technology is advancing so rapidly that many models are now deployed instead as AI agents<\/a> that work without human prompting and can plan out myriad steps in real-world environments, such as in a recent case in which an AI agent sent a false tip<\/a> to Philadelphia police about an unsolved homicide case.<\/p>\n As AI capabilities have grown, the government\u2019s capacity to independently check these systems hasn’t kept pace. There still is no standard process for testing how safely AI agents are acting in the real world, even in simple cases such as when AI agents book a flight, said Sayash Kapoor, an incoming assistant professor at the University of California, Berkeley School of Information.<\/p>\n \u201cOur previous methods would just look at the final answer, the final check, whether the flight was correctly booked. And in this case, it might seem as if the task was solved correctly,\u201d said Kapoor, the co-author of a new paper examining agent evaluations. \u201cBut unless you\u2019re looking at the entire set of steps that an agent took, you won\u2019t be able to tell that the agent made a huge mistake.\u201d<\/p>\n When Trump started his second term, he initially empowered venture capitalists David Sacks and Sriram Krishnan as his AI czar and senior AI policy advisor, respectively. Their expertise was in leading tech investments, not the technical evaluation of AI models.<\/p>\n Both stepped down earlier this year but Sacks now serves on Trump\u2019s technology advisory council. And after a relatively unknown Anthropic engineer resigned<\/a>, warning about the dangers of AI development, Trump made a series of policy changes.<\/p>\n Trump empowered National Intelligence Director Jay Clayton to lead his new Super Intelligence Force, which the president said would keep the U.S. dominant in AI development.<\/p>\n Clayton\u2019s appointment to the force, along with the Pentagon\u2019s chief technology officer, Emil Michael, shows how the administration is pushing to frame decisions around model safety as a national security matter that needs executive discretion and secrecy, overshadowing AI\u2019s risks to consumer protection, privacy, and civil rights, said Amos Toh, a senior counsel at the Brennan Center for Justice.<\/p>\n Even as Silicon Valley companies have disclosed their models\u2019 involvement in troubling cyberattacks this year, CAISSI has kept a low profile when it comes to testing for AI safety.<\/p>\n