Schneier - Detecting AI-Generated Text

There are no reliable ways to distinguish text written by a human from text written by an large language model. OpenAI writes:

Do AI detectors work?

  • In short, no. While some (including OpenAI) have released tools that purport to detect AI-generated content, none of these have proven to reliably distinguish between AI-generated and human-generated content.
  • Additionally, ChatGPT has no “knowledge” of what content could be AI-generated. It will sometimes make up responses to questions like “did you write this [essay]?” or “could this have been written by AI?” These responses are random and have no basis in fact.
  • To elaborate on our research into the shortcomings of detectors, one of our key findings was that these tools sometimes suggest that human-written content was generated by AI.
    • When we at OpenAI tried to train an AI-generated content detector, we found that it labeled human-written text like Shakespeare and the Declaration of Independence as AI-generated.
    • There were also indications that it could disproportionately impact students who had learned or were learning English as a second language and students whose writing was particularly formulaic or concise.
  • Even if these tools could accurately identify AI-generated content (which they cannot yet), students can make small edits to evade detection.

There is some good research in watermarking LLM-generated text, but the watermarks are not generally robust.

I don’t think the detectors are going to win this arms race.



from Schneier on Security https://www.schneier.com/blog/archives/2023/09/detecting-ai-generated-text.html

Comments

Post a Comment

Popular posts from this blog

Krebs - NY Charges First American Financial for Massive Data Leak

Image
In May 2019, KrebsOnSecurity broke the news that the website of mortgage title insurance giant First American Financial Corp. had exposed approximately 885 million records related to mortgage deals going back to 2003. On Wednesday, regulators in New York announced that First American was the target of their first ever cybersecurity enforcement action in connection with the incident, charges that could bring steep financial penalties. First American Financial Corp. Santa Ana, Calif.-based  First American [ NYSE:FAF ] is a leading provider of title insurance and settlement services to the real estate and mortgage industries. It employs some 18,000 people and brought in $6.2 billion in 2019 . As first reported here last year , First American’s website exposed 16 years worth of digitized mortgage title insurance records — including bank account numbers and statements, mortgage and tax records, Social Security numbers, wire transaction receipts, and drivers license images. The d...
Read more

KnowBe4 - Scam Of The Week: "When Users Add Their Names to a Wall of Shame"

Image
Eric Howes , KnowBe4 Principal Lab Researcher, found out about another insidious bad guy trick: " If you work in IT there has undoubtedly come a dark moment when you wondered to yourself just who among your employee users would be gullible enough to click through a phishing email and potentially bring down your organization. from KnowBe4 Security Awareness Training Blog https://blog.knowbe4.com/when-users-add-their-names-to-a-wall-of-shame
Read more