Revolutionizing Performance Management: The Future of Evaluations in the Age of AI

Artificial intelligence (AI) and data officer Xavi Amatriain from Expedia Group shared insights at the recent VB Transform 2026 event in Menlo Park, emphasizing the importance of encoding product requirements and security measures into the product design document (PRD) before coding even begins. This approach, according to Amatriain, involves using evaluations, including red teaming evals, to outline what the product should accomplish.

Amatriain, who previously held a leadership role at Google, highlighted the shift towards AI-assisted or AI-generated code as the future of technology development. He stressed the significance of focusing on evaluations to guide the development process.

The discussion at VB Transform also touched on the evaluation gap in AI deployment, with many enterprises allowing production deployment without human review. However, only a small percentage fully trust automated evaluations, leading to potential risks in deployment.

Guardrails, which are artificial business rules put in place to guide decision-making, were also a topic of debate at the event. Amatriain cautioned against excessive use of guardrails, as they can interfere with the feedback loop and bias user interactions. Instead, he advocated for a three-layer governance approach at Expedia, starting with principles, followed by processes and tools, and then automation.

Amatriain emphasized the importance of specialized agents over a monolithic intelligence system, suggesting that a composition of specialized agents is more effective in achieving specific tasks. This approach allows for individual agents to be evaluated and secured independently before being integrated into the larger system.

In addressing real-time travel queries, Amatriain described Expedia’s system of blending different approaches based on latency requirements. He highlighted the need for users to have the final decision-making authority, emphasizing the importance of maintaining user agency in the decision-making process.

Regarding security, Amatriain underscored the need for proactive measures in AI system design to prevent threats from other AI systems. He stressed the importance of early consideration of security principles to avoid the need for reactive guardrails.

Overall, Amatriain’s insights shed light on the evolving landscape of AI development and deployment, emphasizing the importance of thoughtful design, governance, and security measures in the technology industry.