Yann LeCun

LeCun in 2018

Yann André LeCun (originally spelled Le Cun; born 8 July 1960) is a French-American computer scientist working primarily in the fields of machine learning, computer vision, mobile robotics and computational neuroscience. He is the Silver Professor of the Courant Institute of Mathematical Sciences at New York University and Vice-President, Chief AI Scientist at Meta.

LeCun received the 2018 Turing Award, together with Yoshua Bengio and Geoffrey Hinton, for their work on deep learning. The three are sometimes referred to as the "Godfathers of AI" and "Godfathers of Deep Learning.

Quotes

• "The analogy I've been using is the fact that perhaps an equivalent event in the history of humanity to what might be provided by Generalization of AI assistant is the invention of the printing press. It made everybody smarter."
  • "#416 - Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI", on Lex Fridman Podcast, found on DeepCast (7 March 2024)

• "AI is going to bring a new renaissance for humanity, a new form of enlightenment, if you want, because AI is going to amplify everybody's intelligence."
  • "20VC: Yann LeCun on Why Artificial Intelligence Will Not Dominate Humanity, Why No Economists Believe All Jobs Will Be Replaced by AI, Why the Size of Models Matters Less and Less & Why Open Models..." on The Twenty Minute VC (20VC), found on DeepCast (15 May 2023)

ylecun (2014-05-15). AMA: Yann LeCun. r/MachineLearning.

• The direction of history is that the more data we get, the more our methods rely on learning. Ultimately, the task use learning end to end. That's what happened for speech, handwriting, and object recognition. It's bound to happen for NLP.
• [AI progres is very dependent on Moore's law.] The one thing that allowed big progress in computer vision with ConvNets is the availability of GPUs with performance over 1 Tflops.
• Don't get fooled by people who claim to have a solution to Artificial General Intelligence, who claim to have AI systems that work "just like the human brain", or who claim to have figured out how the brain works (well, except if it's Geoff Hinton making the claim). Ask them what error rate they get on MNIST or ImageNet.
• My problem with sticking too close to nature is that it's like "cargo-cult" science... I don't use neural nets because they look like the brain. I use them because they are a convenient way to construct parameterized non-linear functions with good properties. But I did get inspiration from the architecture of the visual cortex to build convolutional nets.
• Many of the papers that make it passed the review process are [good but boring] papers that bring an improvement to a well-established technique... Truly innovative papers rarely make it, largely because reviewers are unlikely to understand the point or foresee the potential of it.
• I try to stay away from all methods that require sampling. I must have an allergy of some sort. That said, I am neither Bayesian nor anti-Bayesian... I think Bayesian methods are really cool conceptually in some cases... but I really don't have much faith in things like non-parametric Bayesian methods...
• Every reasonable ML technique has some sort of mathematical guarantee. For example, neural nets have a finite VC dimension, hence they are consistent and have generalization bounds... every single bound is terrible and useless in practice. As long as your method minimizes some sort of objective function and has a finite capacity (or is properly regularized), you are on solid theoretical grounds.

Interview with TIME magazine, 2024-01-26. Perrigo, Billy (2024-02-13). Meta’s AI Chief Yann LeCun on AGI, Open-Source, and AI Risk (in en). TIME.

• [Large language models] require enormous amounts of data to reach a level of intelligence that is not that great in the end. And they can't really reason. They can't plan anything other than things they’ve been trained on. So they're not a road towards what people call “AGI.”
• I don't like to call [human intelligence] AGI because human intelligence is not general at all.
• The vast majority of human knowledge is not expressed in text... LLMs do not have that, because they don't have access to it. And so they can make really stupid mistakes. That’s where hallucinations come from.
• [Can a commercial entity] produce Wikipedia? No. Wikipedia is crowdsourced because it works. So it's going to be the same for AI systems, they're going to have to be trained, or at least fine-tuned, with the help of everyone around the world. And people will only do this if they can contribute to a widely-available open platform.