Why AI coding makes zero trust an AppSec requirement

Feeling comfortable about the safety of your software supply chain because your organization has invested in SBOMs, signing, and provenance? You shouldn’t.

In the AI coding era, those tenets of the traditional trust model aren’t enough to ensure software safety. That’s because that model fails to answer an important security question: What is the code capable of doing?

CodeHunter CEO Ken Ammon, writing for Help Net Security, summarized the trust vacuum.

“[T]raditional trust models are insufficient when AI generates, modifies and deploys code faster than human review can keep up. We can no longer trust software based primarily on what it is, where it came from or whether it resembles something seen before.”
—Ken Ammon

Brett Smith, a software developer at SAS, said trusted sources, signatures, SBOMs, and provenance prove identity and origin, but they don’t prove intent or safety.

“A binary can be signed by a trusted vendor and have a perfect provenance chain and yet still be malicious if the build pipeline was compromised or contains a zero-day vulnerability.”
—Brett Smith

Here’s why zero trust is now essential to application security (AppSec).

[ Join webinar: Independently Verified SBOMs: Why Trust Is No Longer Enough ]

It’s now trickier to spot malicious code

SBOMs, signing, and provenance answer important questions, but AI coding brings new questions that they don’t address, said Jeff Williams, CTO and co-founder of Contrast Security and founder of OWASP.

“An SBOM tells you what components are present. A signature establishes who signed an artifact and whether it was modified. Provenance tells you where it came from and how it was built. None of that tells you whether the developer made a mistake, whether a trusted contributor or build environment was compromised, or whether the software will behave dangerously in your environment.”
—Jeff Williams

We need to recognize that dangerous code can have a perfectly documented chain of custody. “This becomes even more obvious with agentic systems,” Williams said. “The same legitimate model, tools, and components can produce safe or dangerous behavior depending on the goal, context, data, and sequence of decisions made at runtime.”