Researchers have recently introduced LegoGPT, an innovative artificial intelligence model capable of generating three-dimensional (3D) designs for Lego structures. This open-source initiative aims to explore whether AI can create designs that adhere to real-world physics while maintaining stability. Alongside their findings, the researchers have made their dataset publicly accessible to encourage further exploration and innovation in this area.
LegoGPT AI Model Developed on LLaMA-3.2-Instruct
According to a post from Carnegie Mellon University, the LegoGPT model leverages a large language model (LLM) to generate physically stable and buildable Lego designs based on user prompts. The model is accessible for public use under a permissive MIT license and can be downloaded from GitHub.
Users can request designs such as a “streamline elongated vessel” or a “backless bench with armrest.” The AI can produce designs that fulfill these descriptions and can be constructed upright without risk of collapse.
LegoGPT consists of two key components: a foundational AI model and a stability analysis system. The base model is a fine-tuned version of Llama-3.2-Instruct with one billion parameters, while the stability analysis is conducted using Gurobi, a mathematical optimization tool that assesses the stability of each generated structure.
In conjunction with developing the model, the research team curated a dataset named StableText2Lego, featuring more than 47,000 Lego structures encompassing over 28,000 unique 3D objects. Each entry includes detailed captions, design codes, and models to facilitate training.
To ensure the stability of the AI-generated structures, the researchers employed a dual-robot assembly to recreate the designs and determine whether they maintained an upright position. Additionally, human builders were engaged to assess the stability outcomes when less-skilled hands were used in the construction process. The findings revealed that an impressive 99.8 percent of the generated structures successfully passed the stability assessments.
