On Monday, Anthropic introduced a new technology aimed at enhancing the security of artificial intelligence (AI) models against jailbreaking attempts. Known as Constitutional Classifiers, this system is designed to identify potential jailbreaking attempts at the input stage, thereby preventing the AI from producing harmful outputs.
Anthropic Launches Constitutional Classifiers
Jailbreaking in the context of generative AI involves using unconventional prompts to manipulate an AI model into ignoring its training guidelines, often leading to the production of inappropriate or dangerous content. While many AI developers have put measures in place to guard against such practices, the continuous emergence of novel jailbreaking techniques presents a significant challenge in ensuring the complete security of large language models (LLMs).
Some common jailbreaking strategies include lengthy and complex prompts that disorient the AI’s processing, using multiple prompts to circumvent existing safeguards, and employing unique capitalization methods to bypass defenses.
In a recent post outlining their findings, Anthropic detailed the development of Constitutional Classifiers as an added layer of protection for AI systems. The framework is comprised of two classifiers—input and output—that operate based on a predetermined set of guiding principles, referred to as a constitution. Notably, these constitutions are already leveraged to align the company’s Claude models.
How Constitutional Classifiers Work
Photo Credit: Anthropic
Constitutional Classifiers utilize these principles to delineate types of content that are permissible or prohibited. This constitution generates a wide array of prompts and model outputs from Claude, spanning various content categories. Additionally, the synthetic data is translated into multiple languages and reformulated in known jailbreak styles, creating a comprehensive dataset to test the model’s defenses.
This synthetic dataset is subsequently employed to train both classifiers. Anthropic also initiated a bug bounty program that brought together 183 independent jailbreakers tasked with attempting to breach the Constitutional Classifiers. A detailed discussion of the system’s efficacy is available in an accompanying research paper published on arXiv, which noted that no single prompt style universally worked across all content categories.
In automated evaluation tests, where Claude was subjected to 10,000 jailbreaking prompts, the system displayed a success rate of only 4.4 percent, compared to an alarming 86 percent for an unprotected AI model. The company also succeeded in reducing unnecessary refusals of benign queries while managing the additional processing requirements imposed by Constitutional Classifiers.
However, there are acknowledged limitations to this approach. Anthropic admits that the Constitutional Classifiers may not thwart every possible jailbreaking attempt and could be less effective against innovative techniques specifically crafted to exploit weaknesses in the system. For those eager to evaluate the system’s resilience, a live demo is available here, and will remain operational until February 10.
