A cybersecurity researcher recently utilized OpenAI’s o3 artificial intelligence (AI) model to identify a zero-day vulnerability within the Linux operating system. The vulnerability, located in the Server Message Block (SMB) implementation of the Linux kernel, is recognized as CVE-2025-37899. The flaw is reportedly challenging to detect due to its occurrence during simultaneous interactions by multiple users or connections to the system. A fix for this previously unreported security issue has already been released.
OpenAI’s o3 Identifies Zero-Day Vulnerability
While AI models are becoming increasingly adept at discovering zero-day vulnerabilities—bugs that are previously unknown and likely unexploited—their application in this realm remains relatively uncommon. Many researchers continue to rely on traditional code auditing techniques, which can prove laborious when sifting through large codebases. Sean Heelan, the researcher behind this discovery, explained how OpenAI’s o3 model facilitated the identification of the flaw more efficiently in a blog post.
Interestingly, the researcher was not primarily focused on the significant vulnerability he uncovered. Heelan had initially been testing the AI’s efficacy against a different issue known as CVE-2025-37778, characterized as a “Kerberos authentication vulnerability.” This issue also falls under the “use-after-free” category, where a component of the system releases memory while other segments continue to attempt using it, potentially leading to crashes and security vulnerabilities. The AI model successfully detected this flaw in eight of the 100 test runs.
After confirming that o3 was capable of identifying known security bugs from extensive code, Heelan decided to input the entire session setup command handler file into the AI model. This file spans approximately 12,000 lines and manages various types of requests. To illustrate, it’s akin to providing the AI with an entire novel and asking it to locate a specific typo, with the caveat that this typo could cause a system crash.
Upon instructing o3 to execute 100 simulations on this complete file, the model only detected the known vulnerability once. Heelan acknowledged the decline in performance but emphasized that identifying the bug, even once, was a significant accomplishment. More notably, during other runs, the OpenAI model uncovered a completely different, previously unknown vulnerability that Heelan had overlooked.
This new security flaw, similar in nature to the first, affects the SMB logoff command handler. It emerges when the system attempts to access a file that has been deleted during a user’s logoff or session termination, thereby causing potential system crashes or allowing attackers to execute code with elevated privileges.
The report from o3 indicated that this particular vulnerability poses serious risks, as it could lead to system failures or unauthorized access. Heelan praised the AI model’s capacity to comprehend intricate bugs in realistic scenarios, noting that it articulated the vulnerability effectively in its report.
While Heelan recognized that o3 is not without its flaws and exhibits a high signal-to-noise ratio—indicating a significant number of false positives—he highlighted its human-like approach to bug detection. This adaptability contrasts with traditional security tools, which often adhere to a more rigid methodology.
