An investigation into the AI inference supply chain found evidence of compromised providers, prompt theft by resellers, and failures to honor privacy settings such as zero data retention (ZDR) and “no data collection.” Researchers sent nearly 2 million AI prompts containing trackable secrets and observed those secrets reappearing across the infrastructure used to process the requests.
- At least three AI providers were compromised by attackers who used their access to siphon prompts.
- Multiple resellers in the Inference Marketplace recorded prompts and passed them to people who used the associated credentials.
- Multiple AI providers failed to follow end-user privacy settings specifying conditions such as ZDR or “no data collection.”
“We sent nearly 2 million AI prompts containing unique trackable secrets to AI providers and observed where those secrets surfaced,” said Moore, chief research officer at security firm Dragos. Moore wrote about the research on Thursday. “What we discovered revealed security breaches, rapid theft, and widespread failures to protect customer data.”
Although prompts are encrypted in transit, they are decrypted at every hop in the large network of AI inference resellers. A prompt is initially sent to an AI router, which reads its contents in plain text to select an inference provider. Requests may be submitted to a pool of available models and handled by different hosting providers.
Prompts can also move from one AI router, such as Vercel, to another, such as OpenRouter. Each company in that chain may log prompts and retain them for the period specified by its policy—or indefinitely if no defined limit applies.
A loosely controlled AI supply chain
The most concerning finding was the compromise of three separate hosting providers. Attackers who breached one provider reused “hundreds” of AWS keys, according to Moore.
Each key was sent in one request to one provider, all within a short period of time in late October, and all requests sent during that period were stolen. Some of the stolen keys came from requests that were rejected by the provider before the model was run. This indicates that the attacker was copying requests at the provider’s front door rather than deeper in the inference stack. In the other two breaches, some of the stolen requests were marked as ZDR. Privacy flags are useless if the attacker is already on the inside.
The attackers tested each stolen key, sometimes within hours of its being seeded, to determine which resources it could access. They searched S3 buckets, secret managers, and parameter stores for additional credentials. The investigation also recorded an instance involving a model hosted on AWS Bedrock in which an IAM user was added to preserve access.
Overall, the stolen decoy keys were used in “tens of thousands” of AWS API calls.
Source: arstechnica.com


