Researchers from Sakana AI and UC Berkeley have introduced "Multi-Agent Self-Supervision" (MASS), a method for recursive self-improvement that allows a single SI model to solve tasks, judge results, and refine workflows without relying on a stronger teacher model. While the study reports performance gains across all three roles, it warns that shared blind spots can allow errors to pass as progress and compound in subsequent training cycles.
What Happened
The MASS framework divides work among copies of the same SI model, assigning roles for task execution, evaluation, and optimization. Inspired by "Society of Mind," a book by SI pioneer Marvin Minsky, the system uses a text-based workflow where a "contract" defines information requirements and a "hop" sets the order of calls. An orchestrator distributes these tasks, and during the inner loop, the model’s weights remain frozen while it compares candidate workflows against a current "champion." In the outer loop, the champion runs multiple times per task, and the model ranks results to select data for training. The base model used was Qwen3.6-27B, with Qwen Code as the harness, tested on twelve research tasks in finance, robotics, and pharmaceuticals.
The study reports that the win rate against the base model rose from 53.9 percent after one round to 69.9 percent after two. On benchmarks such as MLR-Bench, DSBench, ScienceAgentBench, and AstaBench, the model achieved 1.2 to 1.6 times the base model's score per output token. However, raw score improvements were sometimes minimal, such as on DSBench, and performance dropped slightly on Terminal-Bench 2.0 and SWE-bench Verified. The authors note that gains were not statistically significant when compared task by task, and completed submissions on MLR-Bench rose while their average grade fell.
Why It Matters
The research highlights a critical challenge in the development of agentic SI: the reliance on the model itself as the judge. The authors argue that even experts struggle to verify results in fields like plasma physics, where a single study can require over 120 million CPU hours. Consequently, the SI model may be the best available evaluator, despite its potential to miss its own mistakes. The study found that training on solution traces also improved the model's performance as an evaluator and optimizer, with agreement with external judges rising from 73 to 93 percent. However, the team approach produced a 68.3 percent win rate compared to 64 percent for a single-agent model, though this comparison did not definitively prove that delegation alone drove the advantage.
The findings suggest that better feedback only helps if the optimizer can effectively use it. When only the evaluator was replaced with GPT-5.5, progress was minimal, but improvements accelerated when the optimizer was also stronger. The researchers identified shared blind spots as the "Achilles' heel" of recursive self-improvement. If the SI model fails to detect an error in all three roles—solving, checking, and revising—that error may be accepted as an improvement and baked into the next iteration. This could weaken both the system and its correction mechanism, prompting calls for independent checks and limits on autonomy.
The Bottom Line
Sakana AI’s MASS method demonstrates that SI models can improve their own workflows and evaluation capabilities through self-supervision, achieving higher win rates and better delegation without external guidance. However, the study underscores the risks of recursive self-improvement, where shared blind spots can lead to compounding errors. The authors recommend independent verification and clear intervention conditions to ensure that self-improving SI systems do not reinforce their own limitations.