The scandal over the flood of intimate images on Elon Musk’s X created non-consensually by its Grok AI tool has underlined how the artificial intelligence industry is “too unconstrained”, according to a pioneer of the technology.
Yoshua Bengio, a computer scientist described as one of the modern “godfathers of AI”, said tech companies were building systems without appropriate technical and societal guardrails.
Bengio spoke to the Guardian as he appointed the historian Yuval Noah Harari and the former Rolls-Royce chief executive Sir John Rose to the board of his AI safety lab.
X has announced it is stopping Grok from manipulating pictures of real people to show them in revealing clothes such as bikinis, including for premium subscribers, after a public and political backlash.
Asked what the furore showed about the state of the AI industry, Bengio said the situation across the sector was “not completely a free for all” but needed to be addressed.
“It is too unconstrained and, because frontier AI companies are building increasingly powerful systems without the appropriate technical and societal guardrails, this is starting to have more and more visible negative effects on people,” Bengio said.
Part of the solution was better governance, he said, including placing moral heavyweights on company boards. As well as Harari and Rose, Bengio has appointed Maria Eitel, the founder of the Nike Foundation – a philanthropic wing of the multinational sports group – as the chair of his safety lab, LawZero, which launched last year.
Bengio earned the “godfather of AI” moniker after winning the 2018 Turing award, seen as the equivalent of a Nobel prize for computing. He shared it with Geoffrey Hinton, who later won a Nobel, and Yann LeCun, a former chief AI scientist at Mark Zuckerberg’s Meta.
The former Swedish prime minister Stefan Löfven will be the first member of the NGO’s global advisory council. Harari, the author of Sapiens: A Brief History of Humankind, has been a prominent voice of caution on AI development and recently published a book, Nexus, outlining his concerns.
“It’s not only a technical discussion for companies building frontier AI systems,” said Bengio. “It also comes down to what choices are made about AI that we consider to be morally right.”
Bengio, a professor of computer science at the University of Montreal, has secured $35m (£26m) of funding for LawZero. It is building a system called Scientist AI that will work alongside autonomous systems – knows as AI agents – and flag potentially harmful behaviour.
“The whole construction of the board has been guided by the idea that we need a group of people who are extremely reliable in a moral sense, who can help us keep to LawZero’s mission of delivering technical solutions for trustworthy, highly capable, safe-by-design AI systems as a global public good,” said Bengio.
Last month, Bengio warned against granting AI rights, saying it was showing signs of self-preservation – a key area of concern for AI safety campaigners – and humans should not be impeded from pulling the plug on such systems.
