While information gypsies are taught to test everything for AI, an examination of the highest level peer review has shown the complete defeat of the skins. Gary King and Stephen Worthington of Harvard compared AI reviews with 134 paid professional peer reviews of 124 scientific manuscripts. Human feedback was strong, with 88% of writers surveyed rating their quality as high or very high. Still, three experienced scientific journal editors, in a blind comparison, favored AI in 26 out of 30 evaluations. There were 5.5% of comments directly contradicting the manuscript for AI, and 15.9% for humans. At least one such error was found in 37% of machine reviews and 64% of human reviews. The audit was conducted in a different model; The person double-checked 90 responses and approved 87 ratings. Of the 651 sources with a DOI or arXiv identifier cited by the AI, zero were found to be fictitious. In a sub-sample of 44 review pairs of similar length, the model judge preferred the artificial intelligence in all 44 cases. Even rewriting human reviews in machine style did not save the result.
"AI should be tested. It's pretty clear what that means. A much higher level tool for working with data and knowledge appeared in people's hands. And not trusting it has become as stupid as multiplying to a column using a calculator or rechecking simulations in an adding machine.