AI Hackers Are Faster Than Your Pen Test — Eli Cohen, Snyk
Eli Cohen of Snyk argues that AI-generated code and AI-powered attackers have broken the traditional security model. He calls for continuous offensive security built on AI pentesting on every code change, agent red teaming, and a multi-agent system that validates exploits. He compares SAST, DAST, and human pentesting, and closes with four questions to ask any AI pentesting vendor.
Why the old security model is breaking
Eli Cohen, co-founder of Helios (now part of Snyk, where he works as field CTO on AI security), opens with a simple premise: we are producing far more code, and much of it is insecure. He cites data showing 218% more lines of code per developer, with more than 85% of developers using coding agents. He also says 62% of LLM-generated output is insecure or broken. Put together, insecure code ships faster than teams can remediate it, and the security backlog keeps growing.
Attackers are on the same curve. Eli argues that AI helps them chain low-severity findings into critical vulnerabilities, especially in application-context bugs, which is where LLM-generated code tends to be weakest. He cites an average of 24 to 34 minutes for a successful AI attack, with the fastest at about four minutes, and says 43% of MCP servers have vulnerabilities.
SAST, DAST, and pentesting: what each one misses
Eli uses the three classic approaches to show their gaps.
- SAST (static scanning) is cheap and can run on every code change. It catches injection classes such as SQL injection and cross-site scripting, but it cannot see runtime problems like authorization or configuration issues.
- DAST (dynamic testing) exercises the running application with large payload sets. It finds issues static analysis cannot, such as BOLA (broken object level authorization, where user A can read user B’s data), but it struggles with business logic.
- Human pentesting is the gold standard for business logic because a skilled tester understands what the application is supposed to do and tries to break it. Eli points out that it is expensive, slow, and usually done once or twice a year. The output is a PDF report that developers must triage and then retest, which leaves most of the year untested. He describes this as a model that no longer fits a world of continuous deployment.
Continuous offensive security and agent red teaming
His central argument is that LLMs are good at the thing human pentesters were valued for, which is reasoning about business context. The shift is to move that capability into continuous, automated testing. Snyk’s offering, Evo, runs AI pentests on every PR and every delta, and tells developers what to fix and how to fix it. Eli says the system also validates findings, proving exploitability so that teams are not left with a pile of unverified flags. This is aimed at the industry’s false-positive problem.
He also introduces agent red teaming. The idea is to simulate multi-step attacks such as prompt injection, data exfiltration, and goal hijacking against agents, and to combine this with dynamic testing for authorization flaws. In his framing, attackers already run continuously, 24/7, so defenders need the same posture.
How the agent team works
Eli stresses that the system was built from the ground up around LLMs rather than by bolting an LLM onto existing scanners. It is a team of specialized agents:
- Orchestrator: the “brain” that follows a dynamic plan and directs the sub-agents.
- Recon: the “intelligence unit” that gathers data on endpoints, APIs, and fingerprints, then formalizes an architecture for the next stage.
- Vulnerability hunters: some target known vulnerability classes, while others explore the application in the wild to understand business logic and where an attacker could get to.
- Judge (exploit validation): reviews every candidate finding and decides whether it is actually exploitable. This reduces false positives and cognitive load.
- Remediation and reporting: make findings actionable, since a report developers cannot act on is of little use.
Context is everything, and no single tool is enough
Eli repeats that an LLM is only as good as the context it receives. Snyk feeds results from its static and dynamic scanners into the AI pentesting layer, so the model knows where to focus. He argues that the best results come from combining tools rather than choosing one. DAST remains the strongest option for exhaustive XSS payload testing, while pentest-style reasoning is the right tool for business logic. He also says that richer context makes the LLM work better and cheaper.
Four questions for any vendor
He closes with four questions to ask any AI pentesting vendor, whether it is Snyk or a competitor, or an in-house build:
- Does it run continuously, or is it a point solution run every so often?
- Does it genuinely reason about application context?
- How much additional context can it ingest and enrich, given that richer data improves results and lowers LLM cost?
- Does it prove that a bug is exploitable, or does it just flag it?
His closing message is that the goal is to find bugs faster, better, and cheaper, so developers and security teams can focus on what matters.
Note: a few figures in the transcript, including the firewall breach numbers, are garbled and have been left out.