Slopsquatting is an emerging threat in supply chain security, fueled by the reliance on AI coding assistants. As developers increasingly depend on AI tools for code generation, they inadvertently provide cybercriminals access to vulnerabilities from the very start.
What is Slopsquatting?
Slopsquatting describes a novel form of supply chain attack that exploits large-scale language models (LLMs). AI hallucinations can unintentionally introduce malicious code into development processes. The term merges “AI slop” with “typosquatting,” where attackers leverage misspellings of popular domains to deceive unsuspecting users.
This attack vector takes advantage of LLMs’ propensity to generate fictional software package names. If an attacker registers these fictitious names, they can embed harmful code within the developer’s codebase.
While AI-assisted coding naturally produces fake open-source packages—collections of files, programs, and installation tools—this becomes dangerous when attackers capitalize on their existence to inject malware.
The AI-Driven Supply Chain Risk
Traditionally, AI safety risks stem from hallucinations that mislead users into trusting incorrect data as legitimate. However, this phenomenon has been transformed into a security vulnerability.
Typosquatting has long been recognized as a fraudulent practice in which malicious actors register misspelled versions of popular packages. Though protective measures exist, AI has shifted the playing field. Instead of mere misspellings, AI can suggest plausible but fictitious packages. Once adversaries identify these “phantom” package names, they can register malware-infused versions, posing significant risks.
Unlike simple typos, these hallucinated packages aren’t easily recognized. For instance, while registries can block attempts to exploit the common package “cross-env,” variations like “mpn install Cross-env file” or “cross-env-extended” could go undetected.
The Severity of Hallucinations
Even when multiple LLMs endorse a hallucinated package, the risk remains. Malicious packages may linger unnoticed in production environments for extended periods, potentially injecting malware into multiple systems. Studies show that vulnerabilities are surging at an alarming 98% annually, significantly outpacing the 25% growth in open-source packages. The average lifespan of reported vulnerabilities has also risen, indicating a drop in overall security.
Real-World Threats from AI Hallucinations
Cybercriminals can create open access packages bearing names similar to libraries known for AI hallucinations, embedding malware instead of legitimate code. As LLMs often repeat fictitious names, attackers could easily register packages that confuse numerous developers.
These malicious packages can appear legitimate due to their similarity to established libraries. A slight typo might be dismissed as an innocent mistake, and even entirely fabricated names can gain credibility in the right context. This makes detection challenging as developers often depend on coding assistants for valid dependency recommendations.
Why Do LLMs Hallucinate?
LLMs tend to prioritize statistically probable outcomes over accuracy. As a result, hallucinations are a common issue. Research shows that the incidence of hallucinations varies between 50% and 82%, depending on the model and input methodology. Even top-performing models like GPT-4o can have hallucination rates exceeding 23% despite mitigations.
Malicious actors can exacerbate this issue through token-level manipulation, leading LLMs to hallucinate desired outcomes and increasing the risk of recommending toxic packages.
Which LLMs are Most Vulnerable to Slopsquatting?
While all LLMs possess vulnerabilities, some are more susceptible. Proprietary models generate hallucinated packages four times more frequently than their open-source counterparts. A thorough study revealed that out of 576,000 code samples, 19.7% revealed hallucinatory content, varying significantly across models. For example, GPT-4.0 Turbo had a hallucination rate of 3.59%, while the leading open-source model, DeepSeek 1B, reached 13.63%.
This suggests that organizations depending on open-source AI tools are approximately four times more vulnerable to slopsquatting attacks. However, the exploitation of proprietary tools remains a possibility, as attackers can manipulate these systems if they detect security weaknesses.
The Impact of Vibe Coding
Recent surveys suggest that software developers attribute around 40% or more of their coding efforts to AI assistance, with expectations of significant future growth. A startling 72% of frequent AI users employ these tools daily.
The rise of vibe coding and AI-assisted development broadens the threat landscape. The risk of slopsquatting escalates as more developers integrate AI tools without adequate validation processes to ensure package accuracy.
When utilizing AI for coding, it’s critical to verify outputs. Always ensure that suggested packages exist in official repositories before integrating them into your project.
Guidelines for Safe AI-Assisted Development
Integrating automated checks to validate package names against recognized registries can help identify hallucinated packages before they reach production. Security teams should actively monitor installations of unusual packages and keep abreast of current threat intelligence focused on slopsquatting campaigns.
Zac Amos is the features editor at Rehack.
Welcome to the VentureBeat community!
Our guest posting program invites technology experts to share insights and provide an unbiased examination of AI, data infrastructure, cybersecurity, and other advanced technologies shaping enterprise futures.
Read more From our guest posting program — check it out guidelines if you’re interested in contributing an article!
Source: venturebeat.com