AI security is the new zero-day, and we’re not ready
Industries are rushing forward with AI adoption. AI is being embedded into financial systems, healthcare innovations, critical infrastructure, and national security operations.
But the way we deal with AI security risks is not keeping pace. We still view AI security as if it’s like traditional software, where vulnerabilities are identifiable ‘code’, which can be ‘patched up’ and reported to public databases. But AI security is radically different. Unless we reform our security apparatus fast, this is going to cause major problems.
For years traditional software security has relied on something called the Common Vulnerability Exposure (CVE) process. In simple terms, once a vulnerability is reported and validated, it’s ranked in terms of the impact or damage it could cause. This rating system helps to get the word out quickly, responsibly, and accurately so the security community can prioritise workarounds or patching to avoid exposure to a breach. Up until now it has been a reasonably successful programme for reporting vulnerabilities.
But businesses view AI security through the same lens as traditional software security and therefore assume the same CVE process will work for AI threats. They won’t.
AI security is different. In traditional software we deal with static lines of code. Bugs and exploits can be recorded in a fairly simple way via the CVE process. AI systems, such as large language models (LLM) do not break in the same way traditional software does. They’re not defined as just static lines of code but by learned behaviors, emergent properties, and training data that is often as unclear as the models themselves.
A security flaw in AI is therefore not just a misplaced line of code, or hardcoded secret (although this is an age-old problem), it is an unpredictable shift in behavior, an exploited blind spot in the training data, or a subtle manipulation that alters outputs without ever triggering traditional alarms. The CVE system – of which I am a big fan – is not designed to deal with this.
Hidden AI threats create tunnel vision
All AIs are hidden behind fairly traditional looking Application Programming Interfaces (APIs). This is the primary way services and applications can actually access the models. However, APIs can be a point of vulnerability. According to the US Cybersecurity and Infrastructure Security Agency, APIs account for a growing volume of (reported) AI security issues.
The ‘Open Worldwide Application Security Project’ helpfully documents various AI vulnerabilities. However, seven of the Large Language Model top ten vulnerabilities can only be leveraged by exploiting an API security vulnerability. This means common API security concerns, like hard coded credentials or security keys, can expose access to an LLM, as researchers at Truffle Security discovered.
Is this surprising? Not really. Much of the world’s LLM training data is underpinned by the massive common crawl dataset, a snapshot of the internet. It stands to reason the poor insecure coding habits of a generation, exposed on the internet would make its way into LLM architecture. While training data is only one behaviour source, it’s a good example of the tunnel vision on AI security thinking.
To make matters worse, actually reporting AI vulnerabilities can be difficult. Models are constructed from layers of open-source components, third-party integrations, and proprietary datasets, which makes it difficult to determine who is responsible when something goes wrong.
Many AI projects lack even the most basic security reporting structures, leaving researchers with no clear path to responsible disclosure. When vulnerabilities are reported, many vendors dismiss them outright, claiming they don’t fit into the traditional definition of a security flaw. AI vulnerabilities exist in the grey space between software, data, and behavior, but we insist on using a CVE framework that doesn’t recognise the new and novel AI vulnerabilities.
Timebomb waiting in the AI supply chain
AI models are dynamic, evolving, and constantly interacting with new inputs, yet security teams are still approaching them as if they are static pieces of software. Without accounting for model drift, feedback loops, and adversarial manipulation, vulnerabilities will only increase over time.
Fixing vulnerabilities in some AI systems is extremely difficult. The root cause of an AI security issue is rarely a simple bug that can be patched. It is embedded in the model itself, tied to training data that cannot be easily rewritten, or lurking in complex interactions that make a straightforward fix impossible.
The ‘traditional approach’ of deploying patches and issuing updates is surely inadequate. But I see an industry that is still clinging to a binary view of cybersecurity; systems are either vulnerable or they are not. AI does not operate in this way. The challenge is not just fixing what is broken but defining what secure behavior even looks like in a system that learns, adapts, and changes over time.
The trouble is that vulnerabilities that do not receive a CVE identifier are often ignored, despite being active AI security risks. AI vendors therefore risk choosing secrecy over disclosure, treating security research as a publicity issue, rather than a necessity for the greater good.
Without standardised security reporting and an AI bill of materials (known as AIBOM), AI enabled software risks being filled with hidden components, untracked dependencies, and undocumented modifications, making it nearly impossible for cybersecurity teams to assess whether their AI services and systems are affected.
The failure is systemic, but it is not inevitable
We will continue to fail to detect AI specific threats, partly because of the lack of tools to do so, but more so because of a gulf in the skillset. Traditional cybersecurity scanners, software engineers, data architects, and testing teams are not set up, skilled or designed to catch adversarial inputs, model inversion attacks, or data poisoning attempts. The attack surface is expanding, but teams are being left blind to the AI threats that exist beyond the capabilities of conventional security tools or AI API issues.
Even with the release of the UK AI security code of practice (which in my view should be mandatory for critical industries, rather than voluntary) we are repeating the same mistakes that led to the massive security failures of the past. Without structured learning from past vulnerabilities, without clear definitions of AI security vulnerabilities, and without a willingness to adapt existing CVE reporting frameworks to this new reality, I fear things will get worse before they get better. If they get better at all.
AI security needs a radical shift. Above all we should abandon the idea that AI can be secured using the same processes built for traditional software. That means vendors should allow independent security testing, remove legal barriers that prevent vulnerability disclosure, and integrate security into the AI pipeline from the start. (One relatively simple change would help: AI products should ship AI Bill of Materials AIBOM which would document the dataset, model architecture and the dependencies.)
More generally, we will need more collaboration. Cybersecurity teams should champion ‘secure by design’ methodology for the engineering community and build AI threat modeling. And the industry as a whole must develop AI-specific security tools, establish clear frameworks for AI vulnerability reporting, and demand transparency from AI vendors.
This is not just a technical challenge, it’s a strategic imperative. AI security cannot be an afterthought. Without independent scrutiny, modern and novel AI security vulnerabilities will remain undiscovered, until they are exploited in real-world attacks. And as these systems become integrated into more aspects of our lives, the costs could be devastating.
