The media has released information that contractors who are working to improve Google’s artificial intelligence system called Gemini are comparing responses generated by this virtual product with outputs received from a competing AI model, Anthropic’s Claude.
The mentioned data was discovered by journalists in internal correspondence. Media representatives did not receive an answer from Google to the question of whether this company has permission to use Claude as part of Gemini testing.
Currently, players in the artificial intelligence industry are striving to develop the most efficient and productive AI models. To a large extent, this desire is associated with increased competition in the area of machine intelligence. It is worth noting that very often the level of effectiveness of artificial intelligence models is determined by comparing the performance of these digital products with those of their competitors. At the same time, as part of the relevant practice, most cases use data through industry benchmarks. Contractors evaluating responses generated by competitors’ digital products are not part of specific AI area practices. It is highly likely that many companies will be against such a technique. In this case, it means firms which artificial intelligence models are secretly used as a kind of prototype.
According to media reports, the contractors working on Gemini must evaluate every AI response they see. In this case, attention is drawn to criteria such as truthfulness and verbosity. As journalists reported, the contractors are given up to 30 minutes per prompt to determine which answer is better, Gemini’s or Claude’s.
Also, according to information released by the media, the contractors recently began noticing references to Anthropic’s Claude appearing in the internal Google platform they use to compare Gemini to other unnamed AI models.
In one of the internal chats, the contractors underlined that in responses generated by Claude security factor had a higher priority than in similar results demonstrated by Gemini. It was also noted that in some cases, Anthropic’s digital product did not respond to prompts that it identified as unsafe.
It is worth noting that the commercial terms of the developer of Claude’s service provide for a ban on using its artificial intelligence systems to elaborate a competing product or service. Moreover, it is not allowed to leverage the mentioned systems to train competing AI models. It is worth noting that all mentioned actions are prohibited without approval from Anthropic. Also noteworthy is the fact that Google is a major investor in the developer of Claude.
Shira McNamara, a spokesperson for Google DeepMind, which runs Gemini, did not respond to a journalist’s question about whether the company has permission to use Anthropic’s digital product.
The developer of Claude has not yet provided any comments on this news.
Shira McNamara stated that DeepMind compares model outputs for evaluations but it does not train Gemini on Anthropic models. It was also noted separately that any suggestion about the use of Claude in the framework of work on Gemini is inaccurate.
As we have reported earlier, Google Launches First of Its Gemini 2.0 AI Models.
