“Regulators Shift Focus to AI Governance Amid Control Gaps”

Australia’s financial regulator has issued a caution to financial institutions regarding the governance and assurance practices of AI agents. This warning comes as banks and superannuation trustees are increasingly implementing AI in their internal operations and customer interactions.

The Australian Prudential Regulation Authority (APRA) recently conducted a targeted review of several large regulated entities to evaluate the adoption of AI and associated prudential risks. The review revealed that AI was being utilized across all entities, but there were variations in the maturity of risk management and operational resilience. While boards showed a keen interest in leveraging AI for productivity and enhancing customer experience, many were still in the process of strengthening their management of AI-related risks.

APRA expressed concerns about the overreliance on vendor presentations and summaries and highlighted that boards were not consistently scrutinizing risks such as unpredictable model behavior and the impact of AI failures on critical operations.

The regulator emphasized the importance for boards to enhance their understanding of AI to effectively shape strategy and oversight. It recommended that AI strategies align with an institution’s risk appetite and incorporate monitoring mechanisms and defined procedures to address errors.

APRA noted that regulated entities were experimenting with AI in various areas including software engineering, claims triage, and loan application processing. Other identified use cases included fraud prevention, customer interaction, and scam detection.

Although some institutions were treating AI risks similarly to those of other technologies, APRA pointed out that this approach failed to account for the unique behavior and biases of AI models. The regulator identified gaps in monitoring model behavior, change management, and decommissioning processes, stressing the need for inventories of AI tools and assigned ownership of AI instances.

Cybersecurity emerged as another area of concern, with APRA noting that AI adoption was altering the threat landscape by introducing new vulnerabilities such as prompt injection and insecure integrations. Some entities were found to be inadequately adjusting their identity and access management practices to accommodate non-human elements like AI agents.

APRA recommended the implementation of controls on agentic and autonomous workflows, including privileged access management, configuration, and patching. It also called for security testing of AI-generated code to mitigate potential risks.

Furthermore, the regulator highlighted the risk of institutions becoming overly reliant on a single AI provider without a clear exit plan or substitution strategy. APRA emphasized the presence of AI in upstream dependencies that entities may be unaware of, underscoring the importance of thorough risk assessment.

Identity and Access

The focus on identity and permission controls is also reflected in the ongoing standards work by the FIDO Alliance. The group has established an Agentic Authentication Technical Working Group to develop specifications for agent-initiated commerce.

FIDO emphasized the need for service providers to verify the authorization of actions performed by software and proposed new authentication models to accommodate delegated actions. Vendors such as Google and Mastercard have presented their solutions to FIDO for review, including Agent Payments Protocol and Verifiable Intent framework.

The Centre for Internet Security has published AI security companion guides mapping CIS Controls to AI environments, covering prompt and sensitive-data issues and secure access by software tools and non-human identities.

(Photo by julien Tromeur)

See also: Google warns malicious web pages are poisoning AI agents

Interested in learning more about AI and big data from industry experts? Explore the AI & Big Data Expo events in Amsterdam, California, and London, part of the TechEx series alongside the Cyber Security & Cloud Expo. Visit the website for more details.

AI News is brought to you by TechForge Media. Discover upcoming enterprise technology events and webinars here.