ElevenLabs, a New York-based artificial intelligence (AI) company, has launched an application programming interface (API) for its newly unveiled Voice Design feature. This announcement was made last week, coinciding with the introduction of an open-source initiative called X to Voice, which creates a distinctive voice for an X (formerly Twitter) profile by analyzing the user’s posts. Alongside this, the tool also provides an auto-generated text prompt based on the user’s profile analysis.
In a blog post, ElevenLabs elaborated on these innovative AI tools. The first, an API version of the Voice Design tool, can produce bespoke AI voices corresponding to text prompts provided by users. These voices are crafted based on user-specified attributes, including pitch, timbre, delivery speed, intonation, and more.
This functionality is now accessible through the company’s API, enabling developers to incorporate it into their applications and software solutions. Voice Design can be utilized by developers to create voices for their AI characters or can be made available to users, allowing them to generate new voice profiles for themselves.
ElevenLabs has provided two endpoints for developers. The first endpoint allows the generation of three distinct voice previews from a text prompt. The second enables developers to store these voice previews locally in their libraries. However, the company did not disclose pricing details for the API or the cost associated with individual requests for its AI model. Information regarding the AI model itself remains unspecified.
The second offering, X to Voice, serves as the company’s open-source project. It extends a testable feature accessible on a web client here. Users can input an X username, at which point the AI system automatically analyzes the profile, including both the bio and posts. Following this analysis, the tool generates a relevant text prompt.
This text prompt is then fed into Voice Design to produce a unique voice for the user’s profile. Testing of the feature by Gadgets 360 revealed that it typically takes between 30 seconds and one minute to generate the voice previews for a profile, resulting in three distinct voice samples. The AI voice delivers lines that reflect the user’s profile analysis.
In addition to the voice previews, the interface displays the text prompt used for voice generation. The feature also animates the profile pictures of users who have uploaded a close-up image of their face, synchronizing lip and mouth movements with the spoken words.
