Last week, the Massachusetts Institute of Technology (MIT) introduced an innovative approach to training robots that leverages generative artificial intelligence (AI) models. This new technique integrates data from various domains and modalities, creating a unified language that can be understood by large language models (LLMs). Researchers at MIT believe this advancement may lead to the development of versatile robots capable of performing a variety of tasks without the need for separate training for each individual skill.
MIT Researchers Develop AI-Inspired Technique to Train Robots
In an announcement via a news post, MIT elaborated on the groundbreaking methodology designed for robotic training. Traditionally, instilling a robot with the ability to complete specific tasks has been a labor-intensive process, requiring extensive simulation and real-world data. The necessity for comprehensive understanding of various environments is crucial, as a robot’s adaptability hinges on its knowledge of how to execute certain tasks in unique settings.
This necessitates collecting new sets of data for every distinct task, which are composed of numerous simulations and real-world scenarios. Following this, robots enter a training phase to refine their actions and eliminate errors, which has typically confined them to specific tasks. The concept of multi-purpose robots, reminiscent of those in science fiction, remains largely unrealized until now.
A new method developed by MIT researchers aims to overcome these obstacles. In a pre-print paper published online on arXiv, which has not undergone peer review, the scientists demonstrated how generative AI can help address this challenge.
The researchers unified data from various sources—including both simulated environments and real robotic interactions—as well as different forms of input, such as visual feedback from sensors and positional data from robotic arms, creating a cohesive language for an AI model to process. They also introduced a new framework known as Heterogeneous Pretrained Transformers (HPT) to facilitate this data integration.
Lirui Wang, the study’s lead author and a graduate student in electrical engineering and computer science (EECS), noted that the motivation for this innovative approach was inspired by AI models like OpenAI’s GPT-4.
The system incorporates a transformer model, akin to the GPT architecture, which sits at the core of their new approach, enabling it to process both visual data and proprioceptive inputs—related to the robot’s self-movement, force, and position.
According to the researchers at MIT, this newly developed method could present a more rapid and cost-effective alternative to traditional robotic training techniques. The reduced need for extensive task-specific data contributes significantly to this advancement. Their findings indicate that this approach improves performance by over 20 percent in comparison to training robots from scratch, both in simulations and real-world scenarios.
