Researchers at Google introduced a groundbreaking artificial intelligence (AI) architecture last week, designed to enhance the long-term contextual memory of large language models (LLMs). Published by the Mountain View technology company, the research indicates that AI models employing this new architecture demonstrate a memory retention capability that closely resembles human cognition. The team moved away from traditional Transformer and Recurrent Neural Network (RNN) models to create a novel approach for teaching AI how to retain contextual data.
Titans Architecture Capable of Managing Over 2 Million Tokens
Leading the research effort, Ali Behrouz shared insights about the new architecture on X (formerly Twitter). He emphasized that this approach incorporates a meta in-context memory with attention mechanisms to enable AI models to retain pertinent information during computational tasks.
According to the research paper, available on the pre-print online journal arXiv, the Titans architecture can effectively expand the context window for AI models to encompass more than two million tokens. This challenge of memory retention has long posed difficulties for developers in the AI field.
Unlike AI, humans typically recall information and events by contextualizing them. For instance, if someone were asked about their attire last weekend, they could also recall related information, like attending a close friend’s birthday party. This allows for the seamless integration of short- and long-term memories to provide a comprehensive answer to follow-up questions.
Conversely, AI models generally rely on retrieval-augmented generation (RAG), adapted from Transformer and RNN frameworks. In this system, information is stored as neural nodes that an AI accesses to retrieve relevant data. After answering a query, however, the model discards the information to optimize processing resources.
This method presents two main challenges: AI models cannot retain information for the long term, which means users must re-establish context for follow-up queries; and these models struggle to effectively retrieve information that relies on long-term context.
With the Titans architecture, Behrouz and his colleagues aimed to create a system that allows AI models to build and sustain long-term memory while also optimizing memory usage through forgetting irrelevant data.
To achieve this, the researchers devised an architecture that encodes historical data into a neural network’s parameters. They introduced three variants: Memory as Context (MAC), Memory as Gating (MAG), and Memory as a Layer (MAL), each tailored for specific tasks.
Moreover, the Titans architecture incorporates a novel surprise-based learning system, which encourages AI models to remember significant or unexpected information related to a topic. These enhancements collectively improve the memory function within LLMs.
In the BABILong benchmark, Titans (MAC) shows outstanding performance, where it effectively scales to larger than 2M context window, outperforming large models like GPT-4, Llama3 + RAG, and Llama3-70B. pic.twitter.com/ZdngmtGIoW
— Ali Behrouz (@behrouz_ali) January 13, 2025
In a follow-up post, Behrouz noted that internal assessments using the BABILong benchmark revealed that Titans (MAC) models surpassed other leading AI systems, including GPT-4, Llama 3 + RAG, and Llama 3-70B.
