Microsoft has unveiled Mu, a groundbreaking artificial intelligence (AI) model capable of operating locally on devices. In a recent rollout, the tech company introduced several new features in Windows 11, including an innovative AI agents capability in the Settings menu. This feature allows users to express their desired actions in natural language, enabling AI agents to either navigate to the appropriate settings or execute the tasks autonomously. Microsoft has confirmed that this functionality is driven by the Mu small language model (SLM).
Microsoft’s Mu AI Model Powers Agents in Windows Settings
In a blog post, Microsoft elaborated on its new AI model, which is fully operational on compatible Copilot+ PCs through the device’s neural processing unit (NPU). The company has optimized Mu for minimal latency, reporting that it can respond at a speed exceeding 100 tokens per second to satisfy the user experience requirements associated with the Settings agent.
Mu employs a transformer-based encoder-decoder architecture featuring 330 million token parameters, making it well-suited for small-scale implementations. This architecture includes an encoder that transforms input into a concise fixed-length representation, which the decoder then analyzes and converts into an output.
Microsoft selected this architecture for its high efficiency, which is crucial given the computational limitations typically found in local devices. To align with the specifications of the NPU, the company adjusted layer dimensions and optimized parameter distribution between the encoder and decoder components.
Derived from Microsoft’s Phi models, Mu was trained on A100 GPUs utilizing Azure Machine Learning. Generally, distilled models, like Mu, offer enhanced efficiency compared to their original counterparts. Microsoft further boosted Mu’s performance through task-specific training and fine-tuning techniques known as low-rank adaptation (LoRA). Remarkably, Mu is reported to perform comparably to the Phi-3.5-mini model, despite being only one-tenth of its size.
Optimising Mu for Windows Settings
Prior to deploying the model for AI agents in Settings, Microsoft confronted the challenge of effectively processing input and output tokens necessary for managing a multitude of system settings. This challenge called for an extensive knowledge base and quick response times to execute user commands nearly instantaneously.
To overcome this, Microsoft significantly expanded its training dataset from 50 settings to hundreds and employed strategies such as synthetic labeling and noise injection to educate the AI on common phrasing used by users. Following training on over 3.6 million examples, the model reportedly attained a speed and accuracy that allows it to provide responses in under half a second.
One notable insight was that Mu performs optimally with more elaborate multi-word queries compared to shorter or ambiguous phrases. For example, a request like “lower screen brightness at night” offers more contextual information than simply typing “brightness.” In scenarios where search queries lack clarity, Microsoft continues to display conventional keyword-based search results.
Additionally, Microsoft recognized an issue related to language ambiguity; settings that could apply to multiple functions, such as “increase brightness” for either the device’s screen or an external monitor, proved challenging. Currently, the AI model prioritizes the most frequently used settings to mitigate this complication, a process that Microsoft remains dedicated to refining.
