OpenAI is reportedly in the development stages of its next generation large language model (LLM), but it appears the company is facing significant challenges. A recent report indicates that the San Francisco-based firm is struggling to make substantial enhancements to its upcoming AI model, known internally as Orion. While Orion is said to excel in language-related tasks compared to its predecessors, it has shown underwhelming results in areas like coding proficiency. Moreover, OpenAI is encountering difficulties in gathering sufficient training data necessary for the effective development of its models.
Concerns Arise Over OpenAI’s Orion AI Model Performance
The Information reports that expectations for the Orion model are not being met, particularly in coding tasks. Citing anonymous employees within the organization, the report suggests that while there has been a noticeable improvement in language tasks, other areas remain lacking.
This performance issue is troubling, especially considering that Orion’s operating costs in OpenAI’s data centers are higher than those for older models like GPT-4 and GPT-4o. The cost-to-performance ratio could hinder the model’s attractiveness to businesses and individual users.
Moreover, the report indicates that the advancement from GPT-4 to Orion is not as significant as the transition from GPT-3 to GPT-4. This trend raises concerns, particularly as similar patterns have been observed in rival AI models developed by companies such as Anthropic and Mistral.
For instance, benchmark scores for Claude 3.5 Sonnet indicate that improvements in each new foundational model are becoming more incremental. Competitors have largely redirected their focus toward developing novel capabilities, such as agentic AI, which has drawn attention away from their performance limitations.
To address these challenges, the industry is reportedly shifting towards enhancing AI models after the initial training phase. This could involve fine-tuning outputs with additional filtering methods. However, this approach serves as a temporary fix and does not eliminate the underlying issues stemming from either the model architecture or data scarcity.
The data scarcity is primarily attributed to the availability of both free and licensed datasets. In response, OpenAI has reportedly assembled a foundations team dedicated to addressing the training data deficit. However, it remains uncertain whether this team will successfully secure the additional data needed to enhance and refine Orion’s capabilities.
