Uncovering Vulnerabilities: The Rise of AI Supply-Chain Attacks in Release Pipelines

Four supply-chain incidents recently impacted OpenAI, Anthropic, and Meta within a span of 50 days. These incidents included three attacks initiated by adversaries and one case of self-inflicted packaging failure. Surprisingly, none of these incidents targeted the model itself, but rather exposed a common vulnerability in the release pipelines, dependency hooks, CI runners, and packaging gates. This vulnerability had never been addressed by any system card, AISI evaluation, or Gray Swan red-team exercise before.

One of the incidents involved a self-propagating worm named Mini Shai-Hulud that managed to publish 84 malicious package versions across multiple @tanstack/* npm packages in just six minutes. This worm exploited vulnerabilities in the release pipeline, GitHub Actions cache, and OIDC token extraction to hijack TanStack’s trusted release pipeline. Despite the packages having valid SLSA Build Level 3 provenance, the attack successfully infiltrated the system without phishing any maintainer passwords or intercepting 2FA prompts.

Following this incident, OpenAI confirmed that two of its employee devices were compromised, leading to the exfiltration of credential material from internal code repositories. Consequently, OpenAI took immediate steps to enhance its cybersecurity measures, including revoking macOS security certificates and mandating desktop users to update their systems by a specific date.

These incidents shed light on a critical architectural finding – the lack of coverage for release pipelines in existing model red teams. The security community emphasized the importance of addressing this gap in every AI vendor questionnaire to prevent similar incidents in the future.

The article further details other supply-chain incidents involving LiteLLM poisoning, Anthropic Claude Code leak, and downstream propagation of the TanStack worm. Each incident highlighted different vulnerabilities in the release pipeline, underscoring the need for a comprehensive approach to cybersecurity in AI development.

In response to these incidents, OpenAI launched a cybersecurity initiative called Daybreak, aimed at bolstering defenses against such attacks. However, the rapid succession of supply-chain breaches exposed a significant gap in security protocols, necessitating a reevaluation of current practices.

The article concludes with an action plan for security directors, urging them to address the identified vulnerabilities in their CI pipelines and enhance their cybersecurity measures to prevent future attacks. It stresses the importance of proactive identification and closure of workflow gaps to mitigate the risks posed by sophisticated supply-chain attacks.

In summary, the series of supply-chain incidents that targeted leading AI companies underscore the need for a holistic approach to cybersecurity that covers not only the models but also the release pipelines and associated components. By proactively addressing these vulnerabilities, organizations can better protect their systems from malicious attacks and safeguard sensitive data.