
Pentesting with AI: How to Choose Vendors in an Accelerated Exploitation Scenario
The increasing speed of vulnerability exploitation, driven by AI, demands a more rigorous approach to selecting autonomous pentesting tools. Evaluating vendors in real-world environments is crucial to ensure coverage and security.
Artificial intelligence is reshaping the cybersecurity landscape, accelerating both the discovery and exploitation of vulnerabilities. With flaw exploitation routinely occurring even before patches are available, choosing AI-based pentesting solutions becomes a strategic decision that requires rigorous criteria, going far beyond sales demonstrations.
Acceleration of Exploitation and the Role of AI
Recent reports underscore the urgency of the scenario. The Mandiant M-Trends 2026 estimates that the average time to vulnerability exploitation is 'less than seven days,' indicating that attacks occur even before fixes are available. VulnCheck data for the first half of 2026 reveals that 23.43% of Known Exploited Vulnerabilities (KEVs) were hit on or before their public disclosure. The average time between a CVE's publication and its inclusion in the KEV list dropped from 120 to 80 days in 2026.
AI is a key factor in this acceleration. The Google Threat Intelligence Group (GTIG) identified, for the first time, a threat actor using a zero-day exploit believed to have been developed with AI, targeting a two-factor authentication (2FA) bypass in a server administration tool. AI has also been associated with the accelerated development of infrastructure suites and polymorphic malware, such as PROMPTSPY, signaling a shift towards autonomous attack orchestration.
CrowdStrike's 2026 research documents a 42% increase in zero-day vulnerabilities exploited before public disclosure, while the Verizon 2025 DBIR indicates that vulnerability exploitation now accounts for 20% of all breaches, a 34% year-over-year increase. AI is, in fact, reducing the skill and time required to transform security advisories into functional exploitation logic.
The Rise of AI Pentesting
Given this scenario, the adoption of AI-powered pentesting tools is on the rise. A March 2026 survey by Synack and Omdia with security leaders in the US revealed that 95% of organizations prioritize pentesting, but on average, only test 32% of their global attack surface. To close this gap, 87% of security leaders are planning, piloting, or already using agentic AI for penetration testing.
However, choosing the right vendor is complex. Product demonstrations can be misleading, and the true shortcomings of an AI tool only manifest in authenticated workflows, real application coverage, and operational security in production environments.
Beyond the Demo: The Importance of Validation in Real Environments
The difference between an intelligent scanner and an AI agent capable of reasoning about an application lies in context. While some agents start from scratch with each run, others use an existing site model, updating it between tests. This ability to maintain and use prior context is crucial, as the reconnaissance phase can consume 20 to 30 hours of a manual pentest, according to CyCognito's model.
A notable example is the discovery of an unauthenticated RCE (CVE-2026-68982) in Apache Airflow. A security researcher, aided by AI, found the flaw by testing how deployment parts trusted each other, rather than just reading the code, demonstrating the power of AI's contextual reasoning instead of superficial analysis.
To ensure effectiveness, it is fundamental that AI agents not only suspect patterns but prove exploitation, reproduce their findings, and provide verifiable evidence. A list of suspected issues is not a validated pentest. In a case study, an AI agent found a flaw on a major insurer's website where a one-time password request could send login codes to any email address. The agent proved the flaw by logging in with a code sent to its own inbox, then being stopped by safeguards.
Essential Criteria for Choosing a Vendor
The evaluation of AI pentesting vendors should focus on critical aspects that go beyond the surface. It is essential that the agent demonstrates the ability to:
- Reach authenticated flows: Navigate multi-factor logins and complex steps.
- Validate findings: Prove exploitation and reproduce vulnerabilities, providing clear evidence.
- Ensure production security: Operate with robust safeguards that function independently of the AI model, controlling what the agent can do and who can interrupt an execution.
- Measure coverage: Explain what was tested, what was ignored, and how coverage is quantified.
- Retest fixes: Rerun the exact attack after a fix, not just a generic rescan.
- Manage data: Clarify what data the agent retains or uses for training.
The best way to evaluate is through a 'bake-off' – a practical test on the client's own application, with defined test accounts and scope, comparing multiple vendors simultaneously. This allows scoring each criterion and identifying the agent that truly delivers results, not just promises.

CyberX Technical Analysis
The integration of Artificial Intelligence into pentesting represents a significant evolution, but also introduces technical complexities in evaluating its effectiveness and security. The acceleration of vulnerability exploitation, with the average time to exploitation becoming negative, requires security teams to adopt tools that can operate with the same speed and depth as adversaries.
Exploitation Mechanisms Accelerated by AI
AI contributes to the automation of reconnaissance tasks, code analysis, and exploit development. AI tools can identify complex patterns and logical flaws that traditional scanners or static analyses might miss. The ability to reason about the application's context, rather than just scanning for known signatures, allows for the discovery of more sophisticated vulnerabilities, such as 2FA bypasses or RCEs in component interactions.
Technical Challenges in Evaluating AI Tools
The selection of an AI-powered pentesting vendor should focus on proven technical capabilities:
- Context and Reconnaissance: The agent must be able to build and maintain a contextual model of the application, navigating complex flows, including multi-factor authentication and logged-in sessions. The ability to persist this knowledge across executions is crucial for efficiency.
- Exploitation Validation: The tool must go beyond identifying 'suspicions'. It needs to prove the exploitation of the vulnerability, reproduce the scenario, and provide detailed technical evidence (e.g., logs, screenshots, payloads) that allow for verification and remediation by the security team.
- Safeguards (Guardrails): In production environments, safeguards are critical. They must be independent of the AI model and ensure that the agent does not cause damage, exfiltrate sensitive data, or perform unauthorized actions. Transparency about how these safeguards operate and who can intervene is fundamental.
- Coverage and Scope: It is vital to understand how the agent measures attack surface coverage and whether it can test different user roles and complex attack chains.
- Retesting Fixes: The ability to re-execute the *same* attack after a fix is essential to validate the effectiveness of the remediation, rather than just performing a generic rescan.
Implications for Defensive Posture
The adoption of AI-powered pentesting enables a more proactive and continuous approach to vulnerability management. By automating complex tests, organizations can reduce the time between discovery and remediation, aligning with the speed of adversaries. However, reliance on AI requires security teams to develop new skills to audit and validate the results of these tools, ensuring they do not introduce new risks or false positives.
CyberX Technical Recommendations
- Conduct rigorous 'bake-offs' in controlled test environments, using your own applications and specific scenarios.
- Demand concrete proof of exploitation and vulnerability reproduction, not just suspicion reports.
- Verify the robustness of safeguards and the capacity for human control over the AI agent.
- Understand the reconnaissance methodology and the agent's ability to maintain application context.
- Integrate AI-powered pentesting results into your Secure Development Lifecycle (SDLC) and vulnerability management processes.
CyberX SOC Intelligence Bulletin
The increasing sophistication and speed of vulnerability exploitation, driven by AI, represent a significant operational threat to web applications. The ability of adversaries to rapidly develop and deploy zero-day exploits, such as the 2FA bypass identified by GTIG, demands a re-evaluation of detection and response strategies.
Assets and Exposure Conditions
Web applications, especially those with complex authentication flows (MFA, multi-step) and exposed APIs, are primary targets. The lack of pentesting coverage across much of the attack surface (68% on average) increases the risk of exposure to undiscovered vulnerabilities.
Investigation Signals and Telemetry
SOC teams should actively monitor for:
- Anomalous Reconnaissance Activity: Unusual URL access patterns, rapid enumeration of directories or parameters, login attempts with varied credentials or at high frequency.
- Authentication Bypass Attempts: Repeated 2FA failures, session token manipulation, or suspicious password reset attempts.
- Unexpected Payloads: HTTP requests containing strings or data structures that indicate injection attempts (SQL, XSS, Command Injection) or business logic manipulation.
- Unusual Error Logs: Application errors that may indicate successful exploitation attempts or failures in internal components.
MITRE ATT&CK
Relevant attack techniques include:
- TA0043 - Discovery: T1087.001 (Account Discovery: Local Account), T1087.002 (Account Discovery: Domain Account), T1046 (Network Service Discovery), T1082 (System Information Discovery).
- TA0001 - Initial Access: T1190 (Exploit Public-Facing Application), T1566 (Phishing) to obtain initial credentials.
- TA0002 - Execution: T1059 (Command and Scripting Interpreter) for code execution.
- TA0004 - Privilege Escalation: T1068 (Exploitation for Privilege Escalation) through web application vulnerabilities.
- TA0005 - Defense Evasion: T1027 (Obfuscated Files or Information) to hide payloads.
Fixes and Mitigation
- Continuous Vulnerability Management: Implement continuous pentesting programs, including agent-based AI solutions, to quickly identify and remediate flaws.
- WAF and API Gateways: Utilize Web Application Firewalls (WAFs) and API Gateways to filter malicious traffic and protect against known and zero-day exploits.
- Strong Authentication: Reinforce authentication with robust MFA and complex password policies.
- Monitoring and Logging: Ensure comprehensive logging of security events in web applications and APIs, with SIEM integration for anomaly detection.
- Prioritized Patch Management: Prioritize applying patches for KEVs and critical vulnerabilities, focusing on internet-exposed systems.
Response and Evidence Preservation
In case of detection of suspicious activity or exploitation, follow standard incident response procedures:
- Immediate Containment: Isolate compromised systems or block malicious IPs.
- Forensic Analysis: Preserve application, web server, WAF, and SIEM logs for detailed analysis.
- Remediation: Apply patches, correct configurations, and revoke compromised credentials.
- Communication: Notify internal and external stakeholders, as per incident policy.
No specific verifiable Indicators of Compromise (IoCs) were identified in the consulted sources, as the focus is on pentesting methodology and not on a specific incident.
Frequently Asked Questions
Why is choosing AI pentesting vendors so critical now?
The increasing speed of vulnerability exploitation, often occurring before patches are available, makes selecting AI-based pentesting solutions a strategic decision requiring rigorous criteria. AI accelerates both the discovery and exploitation of vulnerabilities, demanding tools that can operate with the same speed and depth as adversaries.
What are the key challenges in evaluating AI pentesting tools?
Product demonstrations can be misleading, and the true shortcomings of an AI tool only manifest in authenticated workflows, real application coverage, and operational security in production environments. Additionally, some agents start from scratch with each run, lacking the crucial ability to maintain and use prior context.
How should organizations evaluate AI pentesting vendors?
The best way to evaluate is through a 'bake-off,' which is a practical test on the client's own application with defined test accounts and scope, comparing multiple vendors simultaneously. This allows for scoring each criterion and identifying the agent that truly delivers results, not just promises.
What capabilities should an effective AI pentesting agent demonstrate?
An effective AI agent must not only suspect patterns but also prove exploitation, reproduce its findings, and provide verifiable evidence. It should demonstrate the ability to reason about the application's context, maintain and use prior context between tests, and identify complex patterns and logical flaws.


