Eye On A.I. · Oct 9, 2026 · 44m · 30m saved
With Ilya Sutskever, Co-Founder and Chief Scientist of Safe Superintelligence Inc., previously Chief Scientist at OpenAI
Ilya Sutskever said there is "quite a high chance" that training a model on human feedback will stop it from making things up altogether.
The standard criticism of large language models is that they only learn statistical patterns in text and so cannot know anything about the world. Sutskever said the opposite: a model has to learn about the world that produced the text in order to predict that text well.
"I think that learning the statistical regularities is a far bigger deal than meets the eye." — Ilya Sutskever
Ilya Sutskever, Co-Founder and Chief Scientist of Safe Superintelligence Inc., on Eye On A.I., began working with Geoffrey Hinton at the University of Toronto at 17, helped build the neural network that won the 2012 ImageNet image-recognition competition, and was OpenAI's chief scientist through GPT-3 and GPT-4. The interview was recorded in 2023, while he was still at OpenAI and before the boardroom fight that briefly removed Sam Altman and led to Sutskever's departure in 2024.
The full interview is covered here so you can skip it. 44 minutes of audio, 14 minutes of reading.
Here are the 11 arguments that matter.
Key Takeaways
Predicting text well requires understanding the world that produced it, which is why Sutskever rejects the view that models only learn patterns
He said a good enough model will have a "shocking degree" of understanding
Human feedback training could remove hallucinations completely, he said in 2023, and his answer to doubters was "let's find out"
Rich Sutton's bitter lesson overstates the case for pure scaling: it matters what is scaled
Seeing is not required to learn about color: a text-only model already ranks purple closer to blue than to red
Today's models can already predict complex, uncertain outputs such as the next page of a book, contrary to a central claim in Yann LeCun's world-model proposal
The people correcting ChatGPT are not working alone: AI tools do most of their work
The cost of training is not the question; what matters is whether the result is worth it
AI could let citizens tell systems how to behave in far more detail than a vote allows
1. Hinton's Lab at 17
The host opened by saying the conversation was recorded before Sutskever left OpenAI. Since then, he said, Sutskever has started Safe Superintelligence Inc., a lab with one mission and no product, which the host said is now valued at $32 billion.
Sutskever started working with Geoffrey Hinton at 17, in 2003, when computers were assumed to be unable to learn
He was born in Russia, grew up in Israel and moved to Canada as a teenager. He said he was drawn to AI partly because consciousness disturbed him and AI looked like a way to understand it. Moving to Canada let him join the University of Toronto right away, where Hinton was a professor.
"Today we take it for granted that computers can learn. But in 2003, we took it for granted that computers can't learn." — Ilya Sutskever
The biggest AI achievement of the time was IBM's Deep Blue chess computer. Sutskever said it relied on searching through possible moves and a simple way to score positions, with no learning, so it did not look applicable to the real world.
He set out to make "a very small but real contribution" to a field he thought was hopeless
Asked whether he wanted to understand the brain, as Hinton did, he said his motive was to understand how intelligence works at all. Back then nobody knew neural networks were good for anything, he said, and he could tell that many claimed contributions to AI would come to nothing.
