What a pain, what a pain: AI against the best scientific reviewers - 132:0 Harvard's experiment showed the superiority of AI over the best experts. And behind this victory lies a threat that may change the nature of science. For decades there has been an almost sacred institution in science - peer review, expert evaluation of scientific work. It is up to the reviewers to determine which mistakes will be corrected, which results will be obtained, and which ideas will appear on the pages of prestigious journals.
And so the Harvard researchers Gary King and Stephen Worthington conducted an experiment, the results of which look like a complete defeat of the "leather bags".
Moreover, almost ideal conditions were created for people. 124 scientific manuscripts. 134 reviews from highly qualified experts. The reviewers were carefully selected, well paid for their work, and motivated to write the most meaningful conclusions possible. 88% of the authors surveyed rated the reviews they received as good or very good.
Claude Opus 4.8 was pitted against these experts. The result was almost incredible. 100% AI wins in 132 head-to-head comparisons performed by the standalone GPT-5.5 model. 87% win rate in blind evaluation of three former editors of leading scientific journals: 26 out of 30 verdicts in ten manuscripts. Winner in all seven categories, from research methods and theoretical ideas to identifying weaknesses, working with sources and the quality of the argument. There are nearly three times fewer incorrect statements: 5.5% for AI versus 15.9% for humans. Not a single fictitious link in AI verified reviews.
The researchers specifically looked at whether the AI's success was explained by longer review lengths, machine-style features, or AI judges' preferences. The advantage remains. But the main result of this study is not even a devastating score. and the chance of a revolution in the organization of science. Imagine a new system.
A scientist writes a paper and passes it through an AI reviewer many times, eliminating errors and strengthening the argument.