Yann LeCun’s big bet to build intelligent machines

Melanie Mitchell, an artificial intelligence researcher at the Santa Fe Institute, is also excited to see a whole new approach. “We really haven’t seen it come out that much from the deep learning community,” she says. She also agrees with LeCun that big language models can’t be the whole story. “They lack memory and internal models of the world which are actually very important,” she says.

Natasha Jaques, a researcher at Google Brain, thinks that language models should still play a role, however. It’s odd that language is totally absent from LeCun’s proposals, she says: “We know that big language models are super efficient and part of a whole bunch of human knowledge.”

Jaques, who works on ways to get AIs to share information and abilities with each other, points out that humans don’t need to have direct experience of something to learn more. We can change our behavior simply by being told something, like not touching a hot pan. “How can I update this model of the world proposed by Yann if I have no language? she asks.

There is also another problem. If they were to work, LeCun’s ideas would create powerful technology that could be as transformative as the Internet. And yet, his proposal does not discuss how his model’s behavior and motivations would be controlled, or who would control them. It’s a strange omission, says Abhishek Gupta, founder of the Montreal AI Ethics Institute and lead AI expert at the Boston Consulting Group.

“We should think more about what it takes to make AI work well in a society, and that requires thinking about ethical behavior, among other things,” Gupta says.

Yet Jaques notes that LeCun’s proposals are still much more ideas than practical applications. Mitchell says the same: “Certainly there is little risk that it will soon become human-level intelligence.”

LeCun would agree. Its aim is to sow the seeds of a new approach in the hope that others will learn from it. “It’s something that’s going to take a lot of effort from a lot of people,” he says. “I’m putting that out there because I think ultimately that’s the way to go.” At least he wants to convince people that big language models and reinforcement learning aren’t the only way forward.

“I hate to see people wasting their time,” he says.

James G. Williams