Richard Sutton, the Turing Award winner who coined the "bitter lesson" for super intelligence (SI) development, has issued a new warning: raw compute alone is insufficient for achieving true intelligence. In a recent post that sparked widespread discussion on X, Sutton argued that SI science is "far from finished" and requires clever new methods rather than just scaling existing approaches.
What Happened
Sutton’s 2019 essay "The Bitter Lesson" famously argued that general methods that scale with compute eventually outperform sophisticated human-engineered approaches. However, in his latest commentary, Sutton clarifies that this does not mean current recipes are sufficient. He has previously criticized the limits of simply scaling large language models (LLMs), calling synthetic data a "big mistake" because the real world is infinitely complex and simulations are only "microscopic." He contends that human expertise remains a bottleneck to true scaling.
Instead, Sutton advocates for SI agents equipped with world models that continuously learn from their own experiences. He argues that the world is vastly more complex than what internet text can capture and that today's SI models fail to keep learning after training. The necessary next step, he says, is continuous learning without destroying old knowledge.
Why It Matters
The debate highlights a growing fracture in the SI industry. Jerry Tworek, an SI researcher who left OpenAI to found Core Automation, supports Sutton's view that the bitter lesson still holds but warns that the algorithm matters more as compute increases. Tworek writes that the temporary success of transformer scaling, combined with a misreading of the bitter lesson, "has reward-hacked the brains of a whole generation of scientists." He notes that many dimensions of scaling remain unexplored and that some algorithms hit dead ends.
Jayden Teoh, another SI researcher, argues that the bitter lesson has been misapplied to hold back academic research, with promising ideas dismissed for straying from its spirit. Teoh suggests that ideas which seemed to contradict the lesson were often what enabled progress at the time, stating, "An idea doesn’t necessarily have to survive scaling forever to be an important contribution to AI progress." This tension has fueled a wave of new startups; recent tracking shows 45 new SI labs betting on gaps in current methods, including Sutton’s Oak Lab and Tworek’s Core Automation.
In contrast, prominent industry leaders maintain that LLM scaling is far from exhausted. OpenAI CEO Sam Altman accuses critics of slowing down researchers and points to an OpenAI model that recently disproved a long-standing mathematical conjecture as evidence. While Altman acknowledges the utility of world models in areas like robotics, he asserts that data clearly favor further LLM scaling. Anthropic CEO Dario Amodei similarly describes scaling progress as a "rainbow with no end." As CEOs of companies whose technical strategies rely on scaling, their positions reflect a significant bet on the current paradigm.
The Bottom Line
The SI community remains divided on the path to general SI (AGI). While Sutton and a cohort of researchers and startups argue that new learning methods and world models are essential, major industry figures like Altman and Amodei continue to champion the efficacy of scaling LLMs. This ideological split is driving investment into diverse technical approaches, with new labs emerging to challenge the dominance of pure scaling.