On Monday, Anthropic revealed a new system aimed at safeguarding artificial intelligence (AI) models from jailbreaking attempts. This innovative approach, referred to as Constitutional Classifiers, is designed to identify and block jailbreaking attempts at the input stage, preventing the AI from generating harmful outputs.
Anthropic Launches Constitutional Classifiers
Jailbreaking in the realm of generative AI involves using unconventional prompt-writing strategies that can compel an AI model to violate its training protocols, potentially resulting in harmful or inappropriate content. While this challenge is not new, AI developers have implemented various strategies to guard against it. Nonetheless, the continuously evolving techniques employed by prompt engineers make it difficult to create a large language model (LLM) that is entirely immune to such threats.
Examples of jailbreaking techniques include the use of overly lengthy and complex prompts that can confuse the AI’s reasoning abilities. Other methods involve segmenting prompts to bypass safeguards or utilizing unusual capitalization to evade AI defenses.
In a blog post outlining the initiative, Anthropic explained that the Constitutional Classifiers serve as a protective layer for AI models. The system features two classifiers—input and output—each guided by a set of principles known as a constitution. Notably, Anthropic already employs such constitutions to align its Claude models.
Mechanics of Constitutional Classifiers
Photo Credit: Anthropic
With the introduction of Constitutional Classifiers, the defined principles outline the types of content that are permissible and those that are not. This constitution aids in generating numerous prompts and model responses from Claude, spanning various content categories. The synthetic data produced is further translated into multiple languages and structured in formats typical of jailbreaking techniques, thereby creating an extensive dataset that can challenge the model.
This data is subsequently utilized to train both the input and output classifiers. To test the system’s resilience, Anthropic launched a bug bounty program, inviting 183 independent testers to attempt bypassing the Constitutional Classifiers. A comprehensive explanation of the methodology is available in a research paper published on arXiv, with Anthropic asserting that no universal jailbreak was detected during the testing phases.
During automated evaluations involving 10,000 jailbreaking prompts, the AI firm noted a success rate of 4.4% with Claude, in stark contrast to the 86% success rate for an unprotected AI model. Additionally, Anthropic managed to reduce excessive refusals of benign queries while minimizing the processing power required for the Constitutional Classifiers.
Despite these advancements, Anthropic recognizes certain limitations. The company admits that the Constitutional Classifiers may not be foolproof against all universal jailbreaks and could be less effective against newly developed jailbreaking methods tailored to exploit the system. Those interested in evaluating the performance of this system can access the live demo here, which will remain available until February 10.
