A groundbreaking artificial intelligence (AI) model named LegoGPT has been introduced by researchers, enabling the creation of three-dimensional (3D) Lego designs. This innovative project aims to assess the capability of AI models to produce structures that adhere to real-world physics and maintain stability. The team has provided insights into the construction of the model and has also released the dataset for public use. Notably, the AI-generated Lego structures underwent testing by both humans and robots to verify their stability.
LegoGPT AI Model Built on LLaMA-3.2-Instruct
In a recent announcement, researchers from Carnegie Mellon University outlined the core functionalities of the LegoGPT model. This large language model (LLM) can transform text prompts into tangible Lego structures while ensuring they are stable and feasible for construction. The model is open-source and can be downloaded from GitHub under an MIT license.
Users can input prompts such as “streamline elongated vessel” or “backless bench with armrest,” and the model will generate designs that not only meet the specified criteria but are also capable of standing upright without collapsing.
The success of LegoGPT is attributed to two key elements: the foundational AI model and a stability analysis system. The base AI model is a fine-tuned version of Llama-3.2-Instruct, boasting one billion parameters, coupled with Gurobi, a mathematical optimization solver that conducts stability assessments on each generated structure.
In parallel to the development of LegoGPT, the researchers compiled a dataset named StableText2Lego, which includes over 47,000 Lego structures and more than 28,000 distinct 3D objects. Each entry is accompanied by comprehensive captions, design codes, and models.
To ensure the stability of the generated designs, the structures were put to the test using a dual-robot assembly, tasked with reconstructing the designs and determining their ability to remain upright. Human participants also attempted to recreate some of the designs to assess how less skilled hands influenced stability. According to the research findings, an impressive 99.8 percent of the generated structures successfully passed the stability testing.
