Researchers caution that Anthropic’s reported 97 percent scam-detection rate may apply primarily to complete scam conversations, including fraudulent investment pitches—not the early relationship-building stage examined in their study. That initial phase often relies on subtle, less suspicious language. The researchers also found that Claude, using Anthropic’s latest chatbot model, was willing in many cases to impersonate a human.
The potential efficiency of AI-powered scam relationship-building contrasts with a broader survey of scam workers, which found that large language models are generally used only as supplemental tools to refine fraudulent messages. Researchers believe the disconnect may be explained by the economics of the scam industry: Human trafficking can still be cheaper and more profitable than automating operations with AI.
Workers held in scam compounds are frequently forced to work without pay or receive minimal wages that keep them trapped in debt bondage. In some cases, trafficking victims are also ransomed for additional payments after spending time in a scam compound. “It may be that they don’t yet feel forced to automate,” says Mirsky of Ben-Gurion University of the Negev. “It’s not only free labor, they also get money for those people as well.”
Even so, the study suggests that scam organizations could soon increase their use of artificial intelligence, says Erin West, a former Santa Clara County, California, prosecutor who now leads the anti-scam group Operation Shamrock. “We should be in great fear of what this study is showing,” West says.
Widespread adoption of AI chatbots could reduce human trafficking within the scam industry, but it may also make online fraud much more difficult to detect and disrupt. Automated scams would eliminate the need for large compounds across Southeast Asia where trafficking victims are housed and forced to work.
“If one of their weak points is getting the people and having to maintain and feed and monitor these people, now they don’t have to do that,” West says. “Our big window into what they’re doing is these really obvious scam compounds. Now, they can do this in somebody’s two-bedroom apartment.”
This story originally appeared on Wired.com.
Source: arstechnica.com