Google is reportedly conducting comparisons between the responses generated by its Gemini AI and those produced by Anthropic’s Claude models. According to sources, the tech giant based in Mountain View has instructed third-party contractors involved in assessing Gemini’s outputs to also evaluate whether Claude’s responses outperform them. Engaging third-party artificial intelligence models for this kind of comparison generally requires prior authorization from the respective company, although it remains unclear if Google secured permission from Anthropic.
Improving Gemini with Claude’s Insights
In a TechCrunch report, it was detailed that contractors evaluating Gemini’s performance are tasked with comparing its replies to those given by Claude, which stands as a competitor. TechCrunch claims to have accessed internal communications indicating that Google had directed these contractors to follow this evaluative process.
According to the report, the contractors are allotted up to 30 minutes for each prompt to ascertain whether they favor Gemini’s or Claude’s answers. These evaluators typically possess expertise in specific fields, allowing them to assess the chatbot’s responses on various metrics such as accuracy, clarity, and thoroughness.
Since the initiation of this comparison practice, some contractors have reportedly observed references to Anthropic and Claude within Gemini’s outputs. In one instance, a response from Gemini stated, “I am Claude, created by Anthropic,” which raises questions about the integrity of the evaluation process. This anomaly suggests the possibility that developers might be using Claude’s outputs to enhance Gemini, particularly when Claude provides superior responses.
Anthropic’s commercial terms clearly state that those utilizing Claude are prohibited from creating any “competing product or service,” or from training “competing AI models” without obtaining explicit consent from Anthropic.
In a statement to TechCrunch, Shira McNamara, a spokesperson for Google DeepMind, clarified that while the division conducts comparisons of model outputs for evaluation purposes, Gemini itself is not trained using Anthropic’s AI models.
Recently, reports surfaced indicating that contractors assessing Gemini’s responses were now required to evaluate outputs even outside their areas of expertise. Previously, these evaluators had the option to skip questions where they lacked proficiency, but that choice is reportedly being eliminated.
