Many security leaders I speak with regularly mention that they are dealing with shadow AI projects. They have implemented an acceptable use policy, listed approved tools, blocked unauthorized ones at the proxy level, and conducted training on avoiding sharing sensitive information with chatbots. Despite these efforts, they acknowledge that there is still more work to be done. However, most of them lack a clear vision of what the next steps should be.
This gap where the real risk resides is not merely a policy issue but an identity challenge. While having a policy is a good starting point, it is not sufficient.
A Familiar Story
A decade ago, a similar situation was referred to as shadow IT. Employees were adopting unapproved SaaS applications faster than IT could keep track, leading some companies to resort to outright bans. However, this approach proved ineffective as it only drove shadow IT further underground, making it harder to manage. The successful companies took a different approach by acknowledging the demand, gaining visibility into the tools being used, understanding the data access, and bringing it under control. Policy played a role, but active management and visibility were the key elements that made a difference.
Shadow AI presents a comparable scenario, with one crucial difference that alters the response strategy. With shadow IT, the focus was on gaining control of the application. In contrast, with shadow AI, the emphasis is on controlling the identity.
Here’s why this distinction is critical: when an employee integrates an AI service using their corporate credentials, the real risk lies in the access granted to that tool. By signing in with their corporate identity, the employee unknowingly provides a third-party AI service with persistent access to corporate data through OAuth tokens with broad permissions. Even after the employee stops using the tool or changes roles, this access remains active, often unnoticed by existing policies.
Looking Beyond Policies
It is essential to delineate where policy ends and where the subsequent layer of action must begin, as the gaps become apparent upon closer examination.
Policy does not monitor authentication processes. When a developer logs into an AI coding assistant with their corporate identity, this interaction is not documented in the acceptable use policy. The connection occurs between identity and service, highlighting the need for monitoring identity behavior.
Delegation is another aspect invisible to policy, yet it plays a crucial role in shadow AI scenarios. OAuth transactions go unnoticed by policy layers, as users grant AI services delegated authority to act on their behalf across various systems without policy oversight.
Furthermore, policy cannot observe the actions of AI tools post-connection, leading to compounded risks. AI agents operating under corporate identities behave differently from their authorizers, making decisions rapidly and accessing systems in ways not anticipated by policy documents.
Lastly, policy limitations are evident when addressing incidents, as they lack the capability to revoke unauthorized access promptly. Identifying problematic connections requires the visibility assumed by the policy but often missing in reality.
The Necessity of Visibility
If shadow AI is fundamentally an identity problem, the response must focus on where identities interact. This necessitates developing a visibility layer that complements existing policies. This layer should provide:
– Discovery: Identifying all corporate identities, human and non-human, connecting to AI services, sanctioned or not, to grasp the scope of the issue.
– Grant Visibility: Scrutinizing every OAuth token granted to an AI service, evaluating the authorizer, permissions, usage, and necessity of the access.
– Runtime Behavior: Monitoring the actions of AI identities post-connection to understand their actual activities, bridging the gap between static permissions and real-time behavior.
– Identity Layer Action: Enabling swift revocation of unauthorized access directly at the source when irregularities are detected, ensuring containment where the access originates.
Challenges Ahead
While current shadow AI scenarios predominantly involve human-initiated connections, the landscape is evolving towards automated agents accessing AI services on behalf of corporate identities. This shift introduces complexities beyond policy management, emphasizing the need for proactive visibility into identity-AI interactions.
The Path Forward
Shadow AI represents a familiar identity risk accelerated by AI adoption. Organizations must recognize that policy alone is not sufficient and should view it as the foundation for a more operational approach. Building visibility into how identities engage with AI services, monitoring OAuth grants critically, and attributing and containing AI identities are essential steps in mitigating the risks posed by shadow AI.
Lessons learned from managing shadow IT emphasize the importance of visibility and proactive management in addressing evolving challenges such as shadow AI. The policy serves as a starting point, but the true test lies in the operational strategies developed to navigate this rapidly changing landscape.
Jason Martin is the Co-Founder and Co-CEO of Permiso Security, a leading identity security provider offering advanced solutions to detect and respond to threats targeting human and non-human identities in cloud environments. With a background as the Executive Vice President of Products and Engineering at FireEye/Mandiant, Jason has played a significant role in shaping product strategies and engineering initiatives. Beyond his executive roles, he is actively involved in the cybersecurity community, organizing cybersecurity conferences and contributing to the field through various works. Additionally, Jason provides advisory services to companies like NightDragon.
Contact Jason via email or on LinkedIn, X, and visit the Permiso website: https://permiso.io
