Schneier - Subliminal Learning in AIs

Today’s freaky LLM behavior:

We study subliminal learning, a surprising phenomenon where language models learn traits from model-generated data that is semantically unrelated to those traits. For example, a “student” model learns to prefer owls when trained on sequences of numbers generated by a “teacher” model that prefers owls. This same phenomenon can transmit misalignment through data that appears completely benign. This effect only occurs when the teacher and student share the same base model.

Interesting security implications.

I am more convinced than ever that we need serious research into AI integrity if we are ever going to have trustworthy AI.



from Schneier on Security https://www.schneier.com/blog/archives/2025/07/subliminal-learning-in-ais.html

Comments

Post a Comment

Popular posts from this blog

KnowBe4 - Scam Of The Week: "When Users Add Their Names to a Wall of Shame"

Image
Eric Howes , KnowBe4 Principal Lab Researcher, found out about another insidious bad guy trick: " If you work in IT there has undoubtedly come a dark moment when you wondered to yourself just who among your employee users would be gullible enough to click through a phishing email and potentially bring down your organization. from KnowBe4 Security Awareness Training Blog https://blog.knowbe4.com/when-users-add-their-names-to-a-wall-of-shame
Read more

Krebs - U.S. Army Soldier Arrested in AT&T, Verizon Extortions

Image
Federal authorities have arrested and indicted a 20-year-old U.S. Army soldier on suspicion of being Kiberphant0m , a cybercriminal who has been selling and leaking sensitive customer call records stolen earlier this year from AT&T and Verizon . As first reported by KrebsOnSecurity last month, the accused is a communications specialist who was recently stationed in South Korea. One of several selfies on the Facebook page of Cameron Wagenius. Cameron John Wagenius , 20, was arrested near the Army base in Fort Hood, Texas on Dec. 20, after being indicted on two criminal counts of unlawful transfer of confidential phone records. The sparse, two-page indictment (PDF) doesn’t reference specific victims or hacking activity, nor does it include any personal details about the accused. But a conversation with Wagenius’ mother — Minnesota native Alicia Roen — filled in the gaps. Roen said that prior to her son’s arrest he’d acknowledged being associated with Connor Riley Moucka ...
Read more

Rapid 7 - Multiple Vulnerabilities in Veeam Backup & Replication

Image
On Wednesday, September 4, 2024, backup and recovery software provider Veeam released their September security bulletin disclosing various vulnerabilities in Veeam products. One of the higher-severity vulnerabilities included in the bulletin is CVE-2024-40711 , a critical unauthenticated remote code execution issue affecting Veeam’s popular Backup & Replication solution. Notably, upon initial disclosure, the Veeam advisory listed the CVSS score for CVE-2024-40711 as “high” rather than “critical” — as of Monday, September 9, however, the CVSS score is listed as 9.8, which confirms exploitation is fully unauthenticated. Five other CVEs were also disclosed in Backup & Replication, including several that allow users who have been assigned low-privileged roles to alter multi-factor authentication (MFA) settings, achieve remote code execution as a service account, and extract sensitive data (e.g., credentials, passwords). Other vulnerabilities in the bulletin affect additional Ve...
Read more