Latest news with #NeuralProcessingUnit

Business Standard

Microsoft Mu model brings on-device AI agent to Copilot+ PCs: How it works

Microsoft has detailed a new lightweight language model called 'Mu,' which has already been deployed to power agentic AI Settings features on Copilot+ PCs. This AI model runs entirely on-device using the Neural Processing Unit (NPU), and according to Microsoft, delivers response speeds of over 100 tokens per second—fast enough to power AI agents in certain scenarios. For context, Microsoft added an AI-powered agent to the Settings app earlier this year, which lets users control their PCs using simple prompts like 'make my mouse pointer bigger' or 'how to control my PC by voice.' The agent can then guide users through solutions—or take the necessary actions automatically, with permission. This feature is currently available to Windows Insiders in the Dev Channel on Copilot+ PCs. How Microsoft's Mu model works Mu is a specially designed AI model that's small and efficient enough to run directly on a PC's NPU, instead of relying on cloud-based servers. Microsoft says this makes it faster and more private since your data doesn't leave the device. The model is built using a technique called encoder–decoder architecture. In simpler terms, it first reads your input (like a question or command), understands its meaning, and then produces a response based on that understanding. Microsoft says this approach is more efficient than traditional models because it separates the task of understanding the input from generating the output—saving both memory and processing time. Mu was also custom-built to take advantage of how NPUs work, using hardware-friendly tweaks and smart memory-saving tricks like 'weight sharing' to reduce its size without sacrificing performance. As a result, Mu is optimised to deliver quick, reliable results while using fewer resources. Mu model's training process Microsoft trained Mu using its Azure cloud platform and powerful NVIDIA A100 graphics processors. The training process involved exposing the model to hundreds of billions of text samples—mostly educational content—to help it understand grammar, language patterns, and general knowledge. Mu builds on techniques from Microsoft's previous small models, like the Phi family. Although Mu is much smaller—about one-tenth the size of Phi-3.5-mini—it delivers similar performance thanks to better efficiency and training techniques. To make sure Mu runs smoothly on a wide range of Copilot+ PCs, Microsoft also worked closely with chip makers like Qualcomm, Intel, and AMD. They used advanced techniques like model quantization, which shrinks the model further while preserving its abilities.