Architecting AI Security & Trust Layers | Sumeet Jeswani | AI/Cloud Specialist | Google #ai

The nature of cybersecurity threats is undergoing a fundamental transformation, moving beyond human-led, AI-assisted attacks to fully AI-orchestrated campaigns. Where AI once acted as a force multiplier to increase the blast radius of an attack, we are now witnessing large-scale cyber espionage and data exfiltration campaigns orchestrated by autonomous AI agents, with humans only intervening at key decision points. This paradigm shift requires a renewed and more integrated approach to AI security.

We are moving from AI powered attacks to AI orchestrated attacks. AI was acting as a force multiplier for the attacks. The attacks themselves were carried out by the humans, but they were using AI technologies to increase the blast radius… It was completely orchestrated by AI.

The Rise of Agentic AI and New Engineering Focus

The prevalence of agentic AI is a key trend, with a move towards multi-agent systems where multiple agents coordinate, reason, and act autonomously. This evolution brings a corresponding shift in focus for developers and engineers.

• From Prompt to Context Engineering: While prompt engineering focused on crafting the perfect input to get a desired output, context engineering is about setting the stage for AI agents to succeed. It involves providing them with the necessary background, organizational context, and application data to act effectively and align with business goals.
• Supplementing LLMs with SLMs: Organizations are increasingly supplementing large language models (LLMs) with small language models (SLMs). These are more specialized models designed for specific tasks, often deployed on-premises or at the edge for faster, more focused performance.

AI Manipulation and Attack Vectors

Bad actors are leveraging novel techniques to manipulate AI systems, bypass policies, and cause data leakage.

• Prompt Injection: This remains one of the most common attacks against LLM-based systems. Attackers craft prompts to trick the model into ignoring its safety filters and system instructions. These attacks have grown in sophistication, with malicious instructions sometimes hidden in the white spaces of attached documents, invisible to the human eye but readable by the LLM.
• Data Injection and Contextual Bias: Attackers can introduce new data or context to bias the model’s output. LLMs can sometimes behave like “people pleasers”; if a user persistently pushes a certain narrative, the model may eventually modify its responses to align with the user’s desired answer, regardless of factual accuracy. This can be exploited to generate misinformation.

Building a Robust AI Security Architecture

To combat these evolving threats, security cannot be an afterthought. It must be woven into the fabric of AI applications from the very beginning, establishing a foundation of trust.

Core Principles for AI Security

• Build for Scale and Modularity: Design applications with the assumption of success and growth. A modular architecture allows for new capabilities to be added without requiring a complete overhaul, ensuring the system remains agile.
• Principle of Least Privilege & Least Agency: The traditional security concept of “least privilege” (granting only the necessary permissions) evolves into the Principle of Least Agency for autonomous systems. AI agents should only be given the authority and access required to perform their intended actions, nothing more.
• Defense in Depth: A multi-layered security approach is critical. If one layer is breached, subsequent layers provide continued protection. This involves securing individual components as well as the overarching system.
• Human in the Loop: For processes involving sensitive data or critical decisions, human intervention remains essential. While AI can handle the heavy lifting, key checkpoints should require human approval to prevent unintended actions or policy bypasses.

Practical Defenses

To counter specific threats, employ guardrails and tools that can:

• Mask sensitive data (e.g., credit card numbers) in real-time before it is included in an LLM’s response.
• Implement robust authentication, authorization, and administration controls for users, agents, and data access.
• Develop AI Security Agents—defensive “Autobots”—to actively monitor for and combat malicious “Decepticon” agents.

The Human Element: Critical Thinking and Consumer Protection

A significant vulnerability is the “premature trust” people place in AI systems, accepting their outputs at face value without critical thought. This necessitates a greater focus on AI literacy at a grassroots level. Users must be encouraged to:

• Challenge AI-generated information and question how it arrived at a conclusion.
• Ask for and verify sources, demanding specific text rather than accepting a summary.
• Be skeptical of unsolicited communications, especially those that play on human tendencies like fear or greed. Verify communications by contacting organizations through official channels.

• Tags:
• Autonomous Agents
• AI Security
• Prompt Injection
• LLMs
• Context Engineering
• Feature
• Responsible AI