By Josh Lemos, CISO at GitLab
Finding security vulnerabilities has never been faster or easier. Linus’s Law, Eric Raymond’s famous dictum about open source software, states that “given enough eyeballs, all bugs are shallow.” Put simply, if enough people review a piece of code, someone will eventually spot the problems.
AI has turbocharged this principle, powering new tools that boost the speed and capacity to find vulnerabilities.
The question remains: who will find them first: your security team or threat actors?
Autonomous AI Security Testing: Current Reality, Not Future Concept
XBOW’s climb to the top of HackerOne’s US leaderboard marked a milestone for application security (AppSec). In just 90 days, its autonomous AI penetration tester identified over 1,060 vulnerabilities, surpassing the output of thousands of human researchers.
Unlike a lot of unskilled AI slop, these findings weren’t theoretical. Through bug bounty reports, XBOW helped companies resolve 130 critical vulnerabilities, with 300+ more vulnerabilities triaged and awaiting resolution.
What distinguishes XBOW’s success is its economies of scale. The system operates autonomously, requires no sleep, and analyzes thousands of targets simultaneously. While human researchers cherry-pick high-value targets, AI systems can methodically test entire attack surfaces. HackerOne reports that autonomous agents submitted more than 560 valid reports in 2025 alone.
Vulnerabilities that once required skilled security researchers to exploit can now be found at machine scale and speed.
AI-Powered Threat Assessment
JPMorgan Chase’s launch of its AI Threat Modeling Co-Pilot demonstrates how enterprise application security teams actively use AI to address the velocity constraints they face. Its Auspex system captures threat modeling tradecraft in specialized prompts that guide AI through system decomposition, threat identification, and mitigation strategies, enabling developers to tackle them through a self-service model.
Auspex combines generative AI with expert frameworks, industry best practices, and JPMorgan’s institutional knowledge. The system incorporates this context directly into AI prompts through a technique called “tradecraft prompting.” It processes architecture diagrams and textual descriptions, then chains prompts to generate threat matrices that specify scenarios, types, security categorizations, and potential mitigations.
Traditional threat modeling can take weeks or months. AI-based approaches, such as JPMorgan’s system, collapse this timeline to minutes while strengthening the quality of human analysis.
Security Teams With AI Integration
New AI capabilities illustrated by XBOW and Auspex offer AppSec teams an alternative to the traditional AppSec model, which consumes enormous resources during development while providing limited coverage.
Code review backlogs grow, security debt accumulates, and critical vulnerabilities slip into production because humans create bottlenecks in the software development lifecycle. A recent GitLab survey found teams lose 7 hours per week to inefficient processes.
AI shifts this equation. Security teams can now systematically redeploy resources away from manual, repetitive activities toward building security-engineered solutions that integrate AI directly into developer workflows.
A collection of validated, AI-powered strategies can help AppSec teams scale effectively:
Develop queryable security intelligence: Ingest every security bug, vulnerability report, and incident into structured data stores that support semantic search. This will transform historical security findings into embeddings that enable AI systems to identify similar patterns across codebases. When a new vulnerability class emerges, your AI can instantly query whether similar issues exist elsewhere.
Customize models for your environment: Rather than relying on generic commercial tools, your AppSec team should leverage RAG (Retrieval-Augmented Generation) approaches to augment LLMs with security anti-patterns and architectural standards specific to your organization. Recent research shows that combining static analyzers like PMD and Checkstyle with fine-tuned LLMs greatly improves code review accuracy while reducing false positives.
Incorporate AI into your developer toolchains: Security findings that arrive days or weeks after code creation create friction and require developers to engage in additional context switching. Instead, embed AI-powered analysis directly into your IDEs, CI/CD pipelines, and pull-request workflows. Developers will receive real-time security guidance as they write code, not after they’ve moved on.
Implement AI for scalable threat modeling: Following JPMorgan’s lead, deploy AI-powered threat modeling that can analyze every new system design, API specification, and infrastructure change. The goal isn’t perfection but breadth: AI-generated threat models that cover 100% of your systems outperform expert-reviewed models that address only 10%.
Use AI to enhance your Static Application Security Testing (SAST): Traditional SAST tools generate high volumes of false positives that desensitize developers and create triage overhead. AI can dramatically improve the accuracy of these tools by understanding code context, analyzing data flows, and identifying real vulnerabilities that pattern-matching tools miss.
AI-Era Security Strategy
Security teams face a pivotal moment. The old playbook of adding more engineers at code review doesn’t work when development moves this fast. AI can match the pace, protecting software as quickly as teams create it.
This shift demands deliberate action. Security leaders need to proactively redirect their teams’ focus, redesign workflows, and rethink what skills matter when humans and AI collaborate. The organizations that commit resources early get this right and emerge with stronger security, lower costs, and faster shipping cycles.
##
ABOUT THE AUTHOR
Josh Lemos is the Chief Information Security Officer at GitLab Inc., where he brings 20 years of experience leading information security teams to his role. He is responsible for establishing and maintaining the enterprise vision, strategy, and program to ensure information assets and technologies are adequately protected, fortifying the Gitlab DevSecOps platform and ensuring the highest level of security for customers.
A talented security practitioner and technology leader, Josh is widely recognized for his strategic vision, his ability to drive growth and innovation, and his passion for building and empowering teams. He believes in technology’s potential to transform the world and the need to secure it against emerging threats. Josh has led security teams at numerous high-growth technology companies including ServiceNow, Cylance, and most recently Block (formerly known as Square).
Josh’s commitment to securing technologies to make a positive impact in the world has been a common thread throughout his career. He serves as a mentor to aspiring information security professionals, and is active in supporting organizations that promote diversity and inclusion in the technology industry. Josh holds a B.S. in Computer and Information Systems Security from the University of San Francisco.
