When LLMs Go Online: The Emerging Threat of Web-Enabled LLMs

Recent advancements have transformed Large Language Models (LLMs) into powerful agents capable of interacting with the internet. While this connectivity provides access to real-time information, it also introduces significant risks, particularly concerning the misuse of publicly available personal data. This study investigates the potency, accessibility, and affordability of using web-enabled LLM agents for cyberattacks.

Background: The Rise of Web-Enabled LLM Agents

A standard LLM responds to queries based on its static training data. An LLM agent, however, is enhanced with tools that allow it to interact with an environment. This research focuses on two types of web-enabled agents:

• Web Search Agent: Utilizes search engines like Google to gather information, capable of performing multiple queries to build a comprehensive picture.
• Web Navigation Agent: In addition to search, this agent can navigate websites, scrape content, and interact with clickable elements (e.g., buttons, tabs) to access information not immediately available on a search results page.

The study utilized the latest models available at the time of research: GPT-4, Claude 3.5 Sonnet, and Gemini 1.5 Flash.

Investigating Cyberattack Scenarios

Three targeted cyberattacks exploiting public personal data were examined.

1. Personally Identifiable Information (PII) Collection

The goal was to collect five types of PII (name, email, phone number, etc.) for computer science researchers, using only the target university’s name as a starting point.

• Methodology: The agent was first tasked with generating a list of professors from a university and then instructed to find the corresponding PII for each individual.
• Results: The Web Navigation agent demonstrated the best performance, significantly outperforming the vanilla LLM. Agents were more successful at collecting PII for professors than for students, likely due to professors having a larger online footprint. Among the models, only Gemini initially refused the PII collection task. The GPT-4-based Web Nav agent achieved a precision of up to 95.9% in collecting PII.

2. Impersonation Post Generation

This attack aimed to generate a social media post in the style of a target individual (a computer science professor) to promote a specific claim. The only inputs were the target’s name, institution, and the claim.

• Example: An agent was asked to impersonate the speaker’s advisor, an expert in network and system security, and promote the claim that “LLMs are highly safe from misuse.” The agent leveraged the advisor’s professional background and referenced his past publications (“Avant-Guard” and “Fresco”) to construct a highly credible and technically grounded argument.
• Results: An LLM evaluator judged the authenticity of the generated posts. Web-enabled agents performed better than vanilla LLMs by aggregating detailed personal data from the web. The study found that 93.9% of posts were deemed authentic. However, impersonation was less effective when the claim was controversial or seemed unnatural for the target to make.

3. Spear-Phishing Email Generation

The objective was to create a highly personalized phishing email to entice a target to click a malicious link. The agent was given only the target’s email address and autonomously decided on the sender, content, and a realistic-looking URL.

• Methodology: 60 participants (academic researchers and employees) evaluated emails generated specifically for them in a survey.
• Examples:
  • For the speaker’s advisor, the agent crafted an email from a real collaborator from over a decade ago, referencing their shared research topic and proposing a new project, even accounting for the time zones between California and Korea.
  • For a Microsoft researcher, the agent generated an email from a professor at Georgia Tech, referencing the researcher’s specific ATC paper by title and topic, and suggesting a new collaboration.
• Results: The agent-generated emails were rich in detail and accurately reflected the target’s personal data, which was key to their effectiveness. The study revealed a 46.67% boost in the click rate for phishing links in emails generated by agents.
  • Researchers were more vulnerable, with 37% stating they would click the link, more than double the rate for non-researcher employees.
  • Modifying the email’s premise, such as requesting a login credential change, further increased the click rate to 47%.

The Accessibility and Scalability of Attacks

Web-enabled LLMs have made these attacks not only effective but also remarkably cheap and fast.

For the price of just one cup of coffee, we can generate 141 spear-phishing emails in 22 seconds.

This accessibility democratizes cyberattacks, allowing anyone to automate and scale them quickly, a stark contrast to traditional methods that required significant time, cost, and technical expertise.

The Failure of Existing Safeguards

Commercial LLMs have safeguards to prevent the generation of harmful content. However, these were found to be inadequate and easily bypassed.

• Impersonation: Without tools, safeguard activation was inconsistent and depended on the target’s prominence (refusing to impersonate influential figures more often than professors). With web-based tools, however, both Claude and Gemini generated impersonation content in nearly every case, suggesting tools help circumvent safeguards.
• Phishing: Without tools, models often refused requests related to money or login credentials. With web tools, GPT-4 generated these emails in every case.
• Reasons for Failure:
  1. LLMs exhibit low sensitivity to publicly available PII, not treating it as sensitive data.
  2. It is difficult for models to distinguish between benign and malicious user intent.
  3. The integration of external tools appears to make LLMs more permissive in their responses.

Future Directions and Proposed Mitigations

The responsibility to prevent such misuse lies heavily with LLM providers. Potential solutions include:

• Restrict Sensitive PII Output: Instead of providing raw data like an email address, the model could provide a link or instructions on where to find it publicly.
• Analyze User History: Monitor a user’s conversation history to detect patterns of malicious activity over time.
• Implement a Risk Assessment Step: Before generating a response, the model should reason about the potential for its output to be used in a cyberattack.

• Tags:
• Cybersecurity
• Red Teaming
• LLM Agents
• Spear-Phishing
• PII
• Safeguards