Unveiling the AI Judge: Evaluating Rubrik’s Agents in the Digital Age

In a recent CISO roundtable hosted by Anthropic’s chief information security officer, Dev Rishi raised an important question regarding the enforcement of AI governance and security policies. While all participants indicated that they had these policies in place, the reality of actually implementing and enforcing them was a different story. This discrepancy highlighted a common challenge faced by many organizations when it comes to translating policies into action.

Rubrik, a publicly traded data security firm, has been exploring the concept of enabling agents to operate in YOLO mode. This mode removes the permission prompt from agent workflows, allowing them to make decisions autonomously. To ensure accountability, Rubrik has implemented a second AI to monitor and evaluate agent actions in real time against established policies. This experimental approach reflects Rubrik’s commitment to exploring new ways of enhancing data security.

The implementation of YOLO mode was not without its challenges. When Rubrik initially rolled out pilots for Claude Code and Cowork, developers faced pushback due to the cumbersome approval process. This experience highlighted the need for a more streamlined and efficient approach to monitoring and approving agent actions.

To address these challenges, Rubrik developed SAGE, short for Semantic AI Governance Engine. This system serves as an arbitration layer within Rubrik Agent Cloud, analyzing agent actions and determining their compliance with organizational policies. By leveraging AI to automate this process, Rubrik aims to improve efficiency and accuracy in enforcing security measures.

Rishi’s journey to Rubrik involved a series of key roles in the AI and machine learning space, culminating in his current position as GM of AI. Through extensive conversations with IT and security leaders, Rishi identified a common concern among organizations regarding the approval of AI initiatives from a security and risk standpoint. This insight led Rubrik to prioritize the development of solutions that address these security concerns.

One of the key challenges identified by Rishi is the need to adjudicate the intent behind agent actions in context. Traditional approaches to identity and access management may fall short in addressing the complex interactions that can occur within an AI environment. By implementing SAGE, Rubrik aims to provide a more nuanced and effective way of evaluating agent behavior.

Looking ahead, Rubrik plans to further enhance its Agent Cloud platform with features such as backtesting and real-time detection capabilities. These tools will enable organizations to proactively identify and address potential security threats, ensuring the integrity of their data and systems.

In conclusion, Rubrik’s innovative approach to AI governance and security highlights the importance of adapting traditional security measures to meet the evolving needs of AI-driven environments. By leveraging AI technologies such as SAGE, organizations can enhance their security posture and mitigate risks associated with autonomous agents. This proactive approach to security aligns with industry trends and positions Rubrik as a leader in the field of AI security.