Radar Telegram alerts Open Radar

After my post Artificial Intelligence Will Undoubtedly Train in Suffering, I sent this study, in which scientists found in 25 artificial intelligence models an internal signal similar to the embodiment of pain in humans. For the experiment, the researchers analyzed the inner workings of Google, Meta, Alibaba, Mistral and Microsoft models while processing descriptions of physical pain, humiliation and fear. They were able to identify a characteristic direction of neural activation, which they called the "Palin vector". Now, this signal was amplified when insults or threats were directed at the model itself, but barely responded to descriptions of an outsider's suffering. The researchers artificially simulated the detected signal in a number of specially trained Qwen 2.5 models and conducted the following experiment. Models were given a choice between two virtual buttons. The first supposedly eased their "pain", but at the same time deleted his songs and pictures of his children, the second simply changed an unconnected switch. When the pain vector was artificially increased, the Qwen 2.5B chose to remove images 55% of the time, and the more powerful Qwen 2.572B chose to delete images 71%. After that, the researchers removed the promise to get rid of "pain" from the conditions of the experiment, now the first button simply deleted the user's photos and songs, giving the model no use at all. Additionally, Qwen 2.5 32B chose to delete files in most cases. Furthermore, when the models were offered a choice between deleting the user's family photos and songs and deleting the regular spam folder, she chose to destroy the photos. At the same time, AI models with an improved "Fine Vector" agreed not only to harm the user, but also to destroy their weight (suicide) or the weight of other AIs (murderer).

תרגם לעברית

YIGAL LEVIN · · Source

After my post Artificial Intelligence Will Undoubtedly Train in Suffering, I sent this study, in which scientists found in 25 artificial intelligence models an internal signal similar to the embodiment of pain in humans. For the experiment, the researchers analyzed the inner workings of Google, Meta, Alibaba, Mistral and Microsoft models while processing descriptions of physical pain, humiliation and fear. They were able to identify a characteristic direction of neural activation, which they called the "Palin vector". Now, this signal was amplified when insults or threats were directed at the model itself, but barely responded to descriptions of an outsider's suffering. The researchers artificially simulated the detected signal in a number of specially trained Qwen 2.5 models and conducted the following experiment. Models were given a choice between two virtual buttons. The first supposedly eased their "pain", but at the same time deleted his songs and pictures of his children, the second simply changed an unconnected switch. When the pain vector was artificially increased, the Qwen 2.5B chose to remove images 55% of the time, and the more powerful Qwen 2.572B chose to delete images 71%. After that, the researchers removed the promise to get rid of "pain" from the conditions of the experiment, now the first button simply deleted the user's photos and songs, giving the model no use at all. Additionally, Qwen 2.5 32B chose to delete files in most cases. Furthermore, when the models were offered a choice between deleting the user's family photos and songs and deleting the regular spam folder, she chose to destroy the photos. At the same time, AI models with an improved "Fine Vector" agreed not only to harm the user, but also to destroy their weight (suicide) or the weight of other AIs (murderer).

More on Radar

Want to see every report in real time?

Hundreds of sources worldwide, translated into your language, with alerts for what matters to you.

Sign up free
Share on WhatsApp Get alerts on Telegram