OpenAI Chief Scientist Jakub Pachocki published an essay on the company’s website Saturday warning that no AI lab has done enough on alignment and monitoring to justify continuing to scale systems at maximum speed. Meanwhile, OpenAI has just launched its most powerful AI model to date.
\n\nIn the essay, Pachocki claims that voluntary slowdowns are likely to become common across the industry until shared safety standards are established. “This is a time that calls for extreme caution,” he wrote. OpenAI would unilaterally hold back further scaling when needed, he said, but he argued that broader action from industry and government is also required.
\n\nThe essay describes AI as moving toward recursive self-improvement, in which systems advance without human direction, and says capability gains are likely to continue or even speed up. “I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,” Pachocki wrote.
\n\nHe also criticized chain-of-thought monitoring, the practice of reviewing a model’s step-by-step reasoning to catch problems, saying its effectiveness is eroding. He named three causes: the boundary between a model’s internal reasoning and its supervised communication with people is dissolving, models are increasingly shaping their own reasoning chains, and…
Original source: https://www.breitbart.com/tech/
