The Vulnerability Landscape: Exploring the Attack Surface

Boards of directors are increasingly pushing for enhanced productivity from large-language models and AI assistants. However, the same capabilities that make AI valuable – such as browsing live websites, retaining user context, and integrating with business applications – also increase the vulnerability to cyber attacks.

A recent report by Tenable researchers uncovered a series of vulnerabilities and attack methods dubbed “HackedGPT”. These findings demonstrated how indirect prompt injection techniques could facilitate data exfiltration and malware persistence. While some vulnerabilities have been addressed, others are still reportedly exploitable, as stated in an advisory released by the company.

To mitigate the inherent risks associated with the operation of AI assistants, it is crucial to implement robust governance, controls, and operational procedures that treat AI as a user or device. This approach necessitates stringent auditing and monitoring of the technology to enhance security.

The research conducted by Tenable highlights the potential failures that can transform AI assistants into security liabilities. Indirect prompt injection involves embedding instructions in web content that the assistant accesses during browsing, leading to unauthorized data access. Another attack vector involves utilizing a front-end query to introduce malicious instructions.

The business implications of these security issues are significant, requiring organizations to implement incident response protocols, undergo legal and regulatory assessments, and take steps to mitigate reputational damage.

Previous research has also shown that assistants can inadvertently leak personal or sensitive information through injection techniques. It is essential for AI vendors and cybersecurity experts to promptly address and patch any emerging vulnerabilities to safeguard against potential breaches.

As the capabilities of AI assistants continue to expand, so do the associated failure modes. Treating AI assistants as live, internet-facing applications rather than just productivity tools can enhance resilience against security threats.

In practice, governing AI assistants effectively involves the following key steps:

1. Establish an AI system registry to track all models, assistants, or agents in use, including their owners, purposes, capabilities, and data access domains.
2. Separate identities for humans, services, and agents to ensure accountability and enforce least-privilege policies.
3. Constrain risky features based on context, making browsing and independent actions opt-in per use case.
4. Monitor AI assistants like any internet-facing application, capturing structured logs and alerting on anomalies.
5. Build human expertise to recognize and respond to injection symptoms, promoting a culture of security awareness and incident response readiness.

For IT and cloud leaders, decision points include assessing the browsing and data writing capabilities of assistants, implementing auditable delegation mechanisms, maintaining a registry of AI systems, governing connectors and plugins, testing for vulnerabilities before deployment, and verifying vendors’ responsiveness to patching issues.

The risks associated with AI assistants include hidden costs, governance gaps, security vulnerabilities, skills shortages, and evolving security postures. Organizations must invest in training to bridge the gap between AI/ML and cybersecurity practices and remain vigilant against emerging threats.

In conclusion, executives should treat AI assistants as sophisticated, networked applications with the potential for security breaches and unpredictable behavior. Implementing governance measures, separating identities, constraining risky features, logging activities, and practicing containment procedures can enhance the efficiency and resilience of AI systems while minimizing the risk of security incidents.