OpenAI’s o3 artificial intelligence (AI) model has recently assisted a cybersecurity researcher in identifying a zero-day vulnerability in the Linux operating system. According to the researcher, the flaw lies within the Linux kernel’s Server Message Block (SMB) implementation, known as ksmbd. The vulnerability, which had remained undiscovered until now, is challenging to detect, as it encompasses scenarios involving multiple users or connections interacting with the system simultaneously. This particular bug is now designated as CVE-2025-37899, and a patch has been released to address the issue.
OpenAI’s o3 Uncovers Zero-Day Vulnerability
The application of AI models in the detection of zero-day vulnerabilities, which are previously undisclosed and often unexploited bugs, remains uncommon, despite advancements in technology that could facilitate this process. Many researchers tend to rely on traditional code auditing methods, which can be labor-intensive when analyzing extensive codebases. Researcher Sean Heelan elaborated on how OpenAI’s o3 model aided him in discovering the vulnerability effectively in a blog post.
Interestingly, the major vulnerability was not the primary target of the researcher. Heelan was initially testing the AI’s capabilities against another issue, identified as CVE-2025-37778, which is related to a “Kerberos authentication vulnerability.” This particular flaw falls under the “use-after-free” category, indicating that a system component deletes data from memory while other segments continue to access it, potentially resulting in crashes and security risks. The AI model successfully identified this known flaw in eight out of 100 attempts.
Upon establishing that o3 could detect an existing security bug within a substantial codebase, Heelan decided to input the AI model with the entire session setup command handler file, rather than merely focusing on a single function. This file encompasses approximately 12,000 lines of code and manages various types of requests. This approach can be likened to presenting the AI with a novel and asking it to locate a specific typo, which, if overlooked, could lead to a system crash.
Following 100 simulations with the complete file, o3 was able to identify the known bug only once. While Heelan noted a decline in performance, he acknowledged the significance of the AI’s success in detecting the flaw. Furthermore, in subsequent runs, the OpenAI model uncovered a previously unknown bug that had eluded the researcher’s attention.
This novel security vulnerability similarly impacts the SMB logoff command handler. It occurs when the system attempts to access a previously deleted file during user log-off or session termination, raising serious security concerns about potential system crashes or allowing attackers to execute code with extensive system privileges.
According to o3’s findings, this bug poses a substantial security threat. Heelan emphasized that the model demonstrated an understanding of a complex bug in a realistic context, competently articulating the vulnerability in its report.
Heelan noted, however, that o3 is not without limitations and maintains a high signal-to-noise ratio, indicating a notable number of false positives compared to true positives. Yet, he contended that the model approaches bug discovery with a human-like intuition, differentiating it from conventional security tools, which tend to operate in a more rigid fashion.
