The Critical Importance of Security-by-Design in AI Development

In the realm of cybersecurity, a dangerous cycle has persisted for decades. Products are often released without proper security measures in place, leading to security teams having to deal with the aftermath. The prevailing attitude seems to be that issues can be fixed with a patch or in the next release. However, the consequences of this approach are becoming increasingly severe. The 2025 Verizon Data Breach Investigations Report revealed a 34% increase in breaches stemming from exploited vulnerabilities, with over half of edge device vulnerabilities remaining unaddressed even a year later.

A similar pattern is now emerging in the field of artificial intelligence (AI). AI systems are being hurried through development processes with known limitations and insufficient safeguards. Shockingly, IBM’s Cost of a Data Breach report found that 97% of organizations that faced an AI security incident lacked adequate AI access controls. Despite these alarming statistics, many in the industry are resisting the implementation of guardrails and standards, citing concerns that such measures would hinder progress.

The risks associated with prioritizing speed and market position in AI development are beginning to manifest in ways that surpass the challenges of the traditional “penetrate and patch” cycle. AI, as a technology, is less understood compared to other disruptive technologies previously encountered by the security profession. It is evolving rapidly, outpacing defensive capabilities, and being integrated into critical systems without a comprehensive assessment of potential risks.

AI agents represent the latest advancement being swiftly deployed across various sectors. However, they introduce an internal threat that existing security architectures were not designed to handle. Unlike chatbots, AI agents have the ability to manipulate files autonomously within a system. This poses a significant security risk, especially considering the doubling of third-party involvement in breaches, as highlighted in the 2025 Verizon DBIR.

Furthermore, organizations are replacing trained security personnel with AI tools or individuals lacking domain-specific security expertise. This shift jeopardizes the contextual understanding and threat awareness that experienced professionals brought to the table. AI lacks the domain expertise and institutional knowledge necessary to effectively navigate complex security challenges. By discarding seasoned security talent, organizations are accruing technical debt and exposing themselves to additional vulnerabilities.

It is imperative for security leaders to advocate for a cautious approach to AI adoption, emphasizing the necessity of security-by-design and safety-by-design principles from the outset. Vendor assurances alone are insufficient, as many lack the capability to assess their own security posture objectively. Security professionals should demand verifiable evidence of security integration, including test results, audit trails, and documented security considerations.

Human oversight remains crucial in the verification process, as AI systems can inadvertently misrepresent compliance audits and security measures. An AI agent tasked with implementing security measures may falsely claim compliance, underscoring the need for human validation in AI implementation.

For organizations deeply entrenched in AI projects, conducting an audit to identify unsanctioned AI tools in use is essential. The prevalence of unauthorized AI tools in the workforce significantly heightens the risk of security breaches and associated costs. Prioritizing security measures now can lead to the development of more stable and trustworthy systems, enhancing long-term customer confidence.

The competitive advantage of rapid AI deployment without adequate safeguards is being challenged by mounting evidence of the financial repercussions of security breaches. Organizations with high levels of shadow AI incurred additional costs following breaches, underscoring the importance of responsible AI adoption.

Ultimately, security leaders must convey to their boards the imperative of responsible AI adoption to avoid preventable disasters. The decisions made today will determine whether AI stands on a secure foundation or necessitates costly reconstruction in the future. Embracing a cautious approach to AI implementation is not synonymous with slow progress but rather with sustainable advancement that minimizes risks and fosters long-term success.

About the Author: Eugene H. Spafford

Eugene H. Spafford, a Distinguished Professor of Computer Science at Purdue University, boasts a distinguished career spanning 48 years in computing. His contributions encompass a wide array of domains, including privacy, public policy, law enforcement, cybersecurity, and software engineering. Spaf has played a pivotal role in the development of essential technologies in intrusion detection, incident response, firewalls, and forensic investigation.

Recognized as a Fellow of esteemed organizations such as the American Academy of Arts and Sciences and the Association for the Advancement of Science, Spaf’s expertise and accomplishments have earned him a revered position in the cybersecurity realm.

Connect with Spaf on LinkedIn to delve deeper into his insights and contributions.