As AI platforms become part of everyday workflows, attackers have found a new way in: the platforms themselves. The Huntress Security Operations Center (SOC) says that the greater day-to-day risk comes from threats abusing AI functions that people already trust and rely on, not from attacks on the companies or AI models themselves.
Over the past nine months, Huntress tracked incidents where attackers used shared AI content as weapons, public widgets, and sponsored search to target AI users and deliver malware.
Legitimate functions hijacked
Huntress has seen threat actors abuse a handful of real-world features of the AI platform, including:
-
Claude Artifacts: content that Claude generates and displays in a chat preview panel that users can post and share via a public link.
-
claude.ai/share links: Share URLs created when someone posts a conversation with Claude; they can appear in search engines when posted in crawlable places like forums or social media.
-
ChatGPT and Grok chats: shared, indexable chats hosted on chatgpt.com and grok.com that can rank for troubleshooting searches.
Each of these lies within a confidence limit. Users recognize the platform, brand and surrounding content, so malicious instructions or downloads appear legitimate. These campaigns often take place just hours or days before the provider downloads the content, but that’s enough time to trick victims before they get caught.
Your files are encrypted, your operations are suspended, an attacker has named his price and is waiting for you to respond. do you pay Do you negotiate? Are you even engaging at all?
Choose your next move in a simulated ransomware incident built from tactics Huntress has seen used against real businesses. You’ll see how ransomware operators behave when they think they’re in control, and what steps you can take to catch an attack before it turns into a negotiation.
FakeAgent: abuse via Claude Artifact
In July, Huntress saw a campaign called FakeAgent hit more than 29 organizations. It all started with a malicious Claude Artifact hosted on the real claude.ai domain.
Since public artifacts are intended for light demos and receive minimal verification from Anthropic beyond a general disclaimer, the attackers created a convincing fake Claude Desktop download page.
Victims Bing searching for the Claude desktop app land on the fake page and click on what appears to be a legitimate download link. Instead, they were redirected to an external domain that delivered the SectopRAT malware.
Huntress reported that Artifact and Anthropic had taken it down by July 22, but incidents involving the same redirect domain continued into August.
Fake installation guide hidden in claude.ai/share
In a separate incident, a victim searching Google for “Claude on Mac” clicked on a sponsored result that led to a claude.ai/share link posing as an Apple Support installation guide. Because the page lived on Anthropic’s own domain, it didn’t carry any of the usual red flags: no similar URL, no certificate warning.
The fake guide instructs the victim to put a curl command into the terminal, starting a six-step chain that deploys MacSync Thief. It collected cookies, credentials, keychain secrets, Telegram and SSH sessions, and cloud keys.
passing the victim by placing a curly line in the terminal.
AI poisoning via ChatGPT and Grok
A third model targets the AI-generated troubleshooting tips themselves. In December, a routine search for “macOS clean disk space” turned up high-profile ChatGPT and Grok conversations that gave ClickFix-style instructions instead of actual fixes.
The attackers had created the conversations, pressed “share” to generate a public URL on the platform’s trusted domain, and used SEO poisoning to place the link at the top of Google results.
Since the links resided in real chatgpt.com and grok.com domains, victims trusted the advice and ran the suggested terminal commands that the AMOS thief provided.
What should defenders do?
None of these attacks breached the security of the AI platform. They took advantage of users’ trust in familiar brands and real domains.
Defenders should treat clipboard-driven execution and AI-assisted troubleshooting as security risks. Restrict script execution from the clipboard and enforce a list of allowed applications. Monitor for new scheduled tasks and changes to antivirus shutdown and train users to notice ClickFix style lures. Quickly report suspicious AI-hosted content to the platform provider.
These campaigns are usually short-lived, but quick reporting and layered controls can reduce the opportunity for attackers to take advantage of them.
If you’re interested in this kind of craft and researching how attackers develop their tactics, join our experts at Shopping Tuesdaywhere we break it all down every month.
Sponsored and written by Huntress Labs.
