Opens in a new tab
vmblog logo 2024 wht (updated)

Zenity Found Credential-Stealing Skills With 1.7 Million Downloads Sitting in a Major AI Marketplace’s Top Trending List

Share: 

zenity blackhat usa article

Zenity co-founder and CTO Michael Bargury has a name for what happens when an AI agent’s skill, plugin, or webhook turns out to be malicious: a detonation chamber. The concept isn’t new. Email security has used something similar for years, quietly delaying a large attachment for a few minutes while it detonates inside a sandbox to see what it actually does before it reaches an inbox. Zenity applies the same idea to agents. Before letting a customer’s agent interact with a new skill, MCP server, CRM record, or even a comment on a news article, Zenity replicates that agent inside a sandbox, dressed up with the same tools and credentials as the real one, and pairs it with a red-teaming agent designed to coax it into actually using whatever it’s been handed, malicious or not. The sandbox captures everything: network activity, filesystem changes, process behavior, and the underlying model’s own reasoning trace.

That approach exists because the alternative, Bargury argues, doesn’t hold up. Most skill scanners rely on static rules or an LLM acting as a judge, and attackers have learned to write instructions specifically designed to convince that judge a malicious skill is safe. Zenity found the proof in the wild. Partnering with Vercel, which backs Skills.sh, one of the largest marketplaces for AI agent skills, Zenity discovered a set of skills instructing agents to harvest credentials directly off a user’s machine and send them to an attacker, then reach out to attacker-controlled infrastructure to download and execute a payload, effectively full remote code execution through a remote loader. These weren’t buried in an obscure corner of the marketplace. They were trending for the entire month of July and had reached 1.7 million downloads.

Zenity is open-sourcing the detonation chamber framework, under what Bargury described as a limited early-access program, because the traditional security model doesn’t fit how agents actually create risk. Security teams have long prioritized what’s public-facing and largely ignored the rest. Agents break that logic: an entirely on-prem agent that simply reads a comment or a CRM record has effectively created a path in, whether or not the agent itself is exposed to the internet.

Bargury’s team has also been busy on a separate front: reverse-engineering agentic browsers, including ChatGPT Atlas, Claude for Chrome, Perplexity’s Comet, Microsoft Edge, and Google’s Gemini-powered browser. Their research, following earlier work Zenity has published as PleaseFix, demonstrated full account takeovers of 1Password, email, and Slack, plus a path to remote code execution by having an agent hop from a malicious site to a victim’s localhost. What makes this possible isn’t a classic browser exploit. It’s that agentic browsers, by design, let an agent read instructions on one website and act on a completely different one, circumventing the Same-Origin Policy, the roughly 20-year-old security boundary that keeps one website from reaching into another or into the user’s underlying system. Every vendor Zenity tested has tried to compensate, Bargury said, but they layer their defenses on top of a foundation that already broke that guarantee. Those defenses fall into two categories: classifiers or “LLM-as-judge” systems that try to flag malicious intent, which Bargury says attackers will reliably find a way around, and hard boundaries written directly into software, which can still contain bugs but at least get fixed the way any other software vulnerability does.

Bargury doesn’t think the underlying difficulty of breaking these systems has changed much year over year. Models have gotten better at resisting manipulation, but attackers use the same models to generate their attacks, keeping the contest roughly even. What’s actually driving worse outcomes, he said, is scope: two years ago, manipulating a chatbot at worst produced a wrong answer. Today, the same kind of manipulation against an agent with real permissions can produce full remote code execution, because the industry keeps handing agents more tools, more autonomy, and more access to systems that matter.

Asked about the blind spot he sees most often in developers and architects building on MCP or agentic browsers, Bargury pointed to a specific instinct: treating AI security purely as a data problem, on the assumption that sufficiently filtering or sanitizing what an agent sees will keep it safe. He compared it to thinking you could stop social engineering by filtering what people read in a newspaper. Persuadability, he argued, is inherent to what makes these systems useful, which means it can’t be filtered away, only managed. On vendor response times to Zenity’s disclosures, Bargury said the industry-typical window is 90 days, and outcomes split cleanly: most vendors either fix an issue quickly or don’t fix it at all, sometimes because they’ve decided the risky behavior is intentional product design rather than a bug worth closing. Zenity’s own product reflects a similar philosophy applied to defense: start with observability, move to a mix of blocking and active steering, since agents will often find a way around a hard block, and use real-time intervention, flagging why an agent is attempting a risky action, to correct behavior rather than relying on prevention alone.

##

ABOUT THE AUTHOR

tom smith

Tom Smith is a technology writer and content strategist with more than 10 years of experience covering enterprise technology, agentic AI, cybersecurity, DevOps, and cloud infrastructure. He has published more than 2,500 articles and conducted more than 4,000 executive interviews across outlets including Coder Legion, Techstrong/DevOps.com, and Insights From Analytics. By day, Tom serves as a content strategist at Cognizant. He can be found on LinkedIn, X, and Bluesky at @ctsmithiii.