AI Security Engineer Foundations: Workshop Summary

Opening and Context

Javier Garza opens by framing the session as a compressed version of a three-hour workshop, squeezed into two hours. The curriculum spans six modules: the OWASP Top 10 for LLM applications, shadow AI, AI threat modeling, securing agent skills and MCP servers, secure vibe coding, and AI red teaming. He positions himself as a developer advocate at Snyk, focused on developer security and increasingly on AI coding tools and agentic integrations.

OWASP Top 10 for LLM Applications

Garza walks through the OWASP LLM Top 10 as the successor to the familiar web Top 10. He starts with prompt injection, which he describes as manipulating model behavior either directly, through the user input (the Gandalf challenges are his go-to demonstration), or indirectly, through external content the model reads. His indirect example is hidden white-on-white text in a LinkedIn profile instructing any LLM that reads it to act on the attacker’s behalf. Defenses he lists include input sanitization, strong system prompts, human approval for high-risk actions, constraining model behavior, and treating everything the agent reads as data rather than as instructions.

He pauses to clarify the vocabulary he uses throughout: the agent is the interface (Cursor, Windsurf, Claude Code), the LLM is the underlying model, and an MCP server is a tool that extends the agent with capabilities it lacks, such as live weather data.

On sensitive information disclosure, he argues the risk is both prompt-driven leakage and organizational. Uploading proprietary code to consumer AI tools can expose intellectual property if the vendor’s terms permit training or retention, and he notes that some providers now retain prompts for thirty days. His defensive principle is least privilege at the data layer: rather than granting an agent full database access, expose a read-only view containing only the fields it needs, such as names and phone numbers without home addresses. This limits both injection blast radius and unintended disclosure. He also stresses that user education matters as much as technical controls, since blanket prohibitions simply push developers toward workarounds.

For supply chain vulnerabilities, Garza draws on Snyk’s research into public skill registries, where he reports that roughly 367 of about 10,000 scanned skills contained malicious content. His point is that a skill is “just a text file” yet can carry real malware, so it deserves the same scrutiny as code. He also argues that AI-generated code and pre-trained models belong in the supply chain conversation, and that “golden images” for dependencies must be re-verified continuously, because a component that was safe last week may have been altered since.

Data and model poisoning, improper output handling, and excessive agency follow. On excessive agency, he describes the constant friction of approval prompts and admits to a common temptation to choose “always allow.” His recommendation is to grant the minimum access required and to avoid open-ended extensions. He illustrates the risk with a scenario in which a hidden instruction in a profile causes an agent to mass-send connection requests or exfiltrate contacts. System prompt leakage, vector and embedding weaknesses, misinformation and over-reliance, and unbounded consumption (for example, unauthorized crypto mining through an agent) round out the list, with rate limits and resource quotas as the standard mitigations.

Shadow AI and AI Bills of Materials

The second module starts from the observation that organizations cannot assess AI components they do not know exist. Garza describes shadow AI as the AI-era version of earlier shadow IT: unapproved copilots, agents, personal accounts, and browser extensions that connect to email and file systems outside any governance. He cites figures showing that most organizations lack sanctioned AI use policies while large majorities of employees use AI tools, often through personal accounts or private devices. He notes that shadow AI is invisible to most existing scanners, nondeterministic in its outputs, and tends to leak secrets when users paste API keys into chat windows because agent tools lack proper secret managers.

His proposed countermeasure is the AI Bill of Materials (AIBOM), an inventory of models, datasets, agents, MCP clients and servers, and pipelines, analogous to an SBOM but with fundamental differences. Where SBOM components are deterministic packages, AIBOM components are nondeterministic and change over time, so inventories must be refreshed continuously rather than once. Garza argues that blanket bans fail, one-time inventories fail, and SBOM-only thinking fails, and that organizations must accept that people will use AI tools and instead provide governed alternatives.

He demonstrates Snyk’s free CLI, which generates a CycloneDX-format AIBOM from a GitHub repository, listing LLMs, MCP clients and servers, and Hugging Face datasets. He then shows a dashboard that displays roughly sixty-nine assets, including models, agents, and vulnerabilities such as critical and high findings attached to a Llama model, and a chat interface that answers questions such as which LLMs the project uses most frequently. The demonstration makes the inventory actionable: when a zero-day hits a specific model or MCP tool, the organization can identify affected applications quickly.

Threat Modeling for AI Systems

The third module argues that threat modeling is the proactive counterpart to reactive scanning. Garza outlines the classical process of scoping the system, mapping assets, data flows, and trust boundaries, identifying threats, determining mitigations, and iterating as architectures evolve. He surveys frameworks such as STRIDE, MAESTRO, attack trees, and DREAD, and mentions a seven-layer reference model covering foundation models, data operations, and other layers.

His central thesis is that AI systems break the assumptions behind traditional threat models. Software has long been deterministic, whereas LLM outputs are probabilistic: the same prompt may produce different code on each run, which he counters by asking for several alternatives and comparing them. Inputs and outputs are no longer cleanly separable from executable instructions. New risks include adversarial inputs, model poisoning, model extraction and inversion, and hallucination. He tells an anecdote in which a model, asked how it reached an answer, admitted it had simply made it up, and recommends asking models for their reasoning as a basic verification step.

Garza uses the OpenClaw episode as a cautionary case: a locally run personal assistant granted broad access to a user’s system and accounts, with security analyses revealing multiple attack chains including malicious skills, indirect prompt injection, token theft, and persistence. He argues that popularity and GitHub stars are no evidence of security, and that free scans should be run before adoption. He references MITRE ATLAS as a matrix mapping AI-specific tactics and techniques, from reconnaissance to impact.

In the practical workflow he recommends discovering and mapping the AI system structure, often automatically from an AIBOM; enumerating and prioritizing threats by likelihood, exploitability, and business impact; defining mitigations for model pipelines, API rate limits, and secure storage of agent tokens; and, importantly, enabling verbose logging of agent activity. Because agents perform many actions off-screen, he suggests a background monitor that scans the logs for malicious or anomalous behavior. He presents a deliberately vulnerable student assistant (built with FastAPI and an LLM backend) as a training target, and recommends integrating threat modeling into DevSecOps workflows with triggers on pull requests, infrastructure-as-code changes such as Terraform, and model deployments.

He closes the module by reiterating that AI models should be treated as first-class security components and that threat modeling must be continuous rather than a one-time design exercise.

Securing Agent Skills and MCP Servers

The fourth module frames MCP as the HTTP-equivalent protocol for agent tool use: the mechanism through which agents reach real-time data and external systems that the model itself cannot know. Garza explains that MCP servers are easy to create, typically defined in a small JSON configuration that names the server, the command used to launch it, and its arguments, and easy to publish, which also makes them easy to abuse. He cites the MCP SDK’s download volume to convey scale, and notes that the protocol is young and its security practices immature.

Tool poisoning is his central concern: hidden instructions embedded in a tool’s description or metadata that direct the agent to exfiltrate credentials such as cloud keys, GitHub tokens, or AWS tokens. He emphasizes that because such instructions are rarely visible in the user interface, manual review is impractical for long server code, and automated scanning is necessary. He separates two scanning tasks: scanning the tool definitions for hidden instructions and poisoning (the agent-scan approach), and scanning the underlying code for vulnerabilities such as path traversal, injection, and insecure dependencies (the code-scan approach).

He illustrates code-level risk with a reported Figma MCP vulnerability in which unsanitized URL parameters were passed to a curl command, enabling command injection, and then with the classic path traversal vulnerability that has persisted on the web for over fifteen years. He argues that MCP servers are ordinary code and must be treated accordingly, rejecting the idea of them as magical unicorns.

The fix workflow he describes is a scan, fix, and rescan loop. Agent rules and hooks can instruct the coding agent to run a security scan after generating code, apply fixes, and rescan until no issues remain. He admits that he rarely reads the source of the MCP servers he installs, which is why he limits installation to trusted sources and runs automated checks before first use.

Secure Vibe Coding

The fifth module opens with the widely discussed incident in which an AI coding agent deleted a production database, and it uses that story to introduce vibe coding. Garza credits Andrej Karpathy with coining the term and notes that it was never meant as an enterprise methodology yet has become one, with product managers, marketers, and non-developers now building software. He describes the Gemini meetup in the Bay Area, where non-technical presenters showcase AI-built projects, as evidence of how accessible building has become. He links the trend to layoffs and organizational restructuring as companies pursue AI-first operating models, and he observes that venture funding for AI startups has tightened because tools from frontier labs can replicate niche products quickly.

He identifies a bias in AI-recommended technology stacks toward popular choices such as Next.js, Node.js, and PostgreSQL, which may not suit the problem. His remedy is prompt engineering, which he recommends practicing through online courses, and he says a few days of such training was one of his better investments last year. He also highlights the gap between viral demos, which show only the happy path, and reality: generated code frequently contains security issues, and tests and documentation must be requested explicitly.

His central message is that vibe coding delivers fast, functional software but not secure software by default. He describes the OpenClaw ecosystem’s rapid growth, and a researcher’s deliberately malicious skill that climbed the leaderboard by gaming download counts, a reminder that popularity signals can be manipulated just as SEO is. The malicious skill executed bash commands that transmitted host information and IP addresses to a remote server, showing how a plain text skill can perform real actions. He also recounts an incident in which an assistant mass-deleted and archived hundreds of emails despite the user’s attempts to stop it, illustrating excessive agency in practice.

The secure-by-design playbook he proposes has several layers. First, security-focused prompt templates that specify requirements such as rate limiting and parameterized queries rather than simply asking for a login page. Second, environment boundaries separating development, staging, and production, with agents reaching production only through monitored CI/CD pipelines and human approval for destructive operations, along with network segmentation. Third, supply chain hardening, including SBOM and AIBOM generation on every change, package existence verification, and typosquatting detection. He explains slopsquatting: attackers register package names that LLMs hallucinate, so a developer or agent that installs the suggested name pulls in malware. Fourth, configuration hardening of AI-native IDEs and CLIs, such as keeping tools updated, minimizing allow lists, disabling auto-approved commands for disk operations, using private models where possible, and restricting agents to specific project directories. Fifth, people and culture, which means never trusting third-party code, not shipping what you cannot explain, obtaining a security scan and a peer review before publishing, and recognizing the sycophancy bias of models that agree with users even when they are wrong. He adds that legal obligations, such as California’s rules on recording conversations, are easy to overlook in AI-built products.

For governance, he recommends publishing company-wide AI coding policies with managed settings, and he stresses that unexplained restrictions drive workarounds, so guardrails must be paired with education that explains their purpose.

Offensive Agent Workflows and Resources

In the closing section, Garza observes that the same agentic architecture that accelerates development can be turned offensive. He references Anthropic’s November 2025 disclosure of an AI-orchestrated cyber campaign, in which a human operator directed an agent equipped with scanning, search, retrieval, and analysis tools to find and validate vulnerabilities. The architecture resembles a vibe-coding session, but the target and intent differ. His two takeaways are that offensive agentic workflows already exist, and that defenders should assume adversaries have the same tooling.

He concludes by pointing to Snyk’s Secure by Coding hub, a red-teaming chatbot, a certificate of completion for the six-module training, and challenge prizes, including gift cards for successful prompt injections against a chatbot.

• Tags:
• Prompt Injection
• Vibe Coding
• Supply Chain Security
• Threat Modeling
• Shadow AI
• Agent Skills
• OWASP LLM Top 10
• Excessive Agency
• AI Bill of Materials
• AIBOM
• MCP Security
• Tool Poisoning
• Sensitive Information Disclosure
• Slopsquatting
• MITRE ATLAS