Microsoft's Phi-3 Shows Surprising Power for Small, Locally Run AI Language Models

Microsoft has released Phi-3-mini, a lightweight AI model that's easy to run locally on smartphones and laptops. With just 3.8 billion parameters, Phi-3-mini offers performance that rivals larger models, and represents a step toward more energy-efficient AI.

Share
Microsoft's Phi-3 Shows Surprising Power for Small, Locally Run AI Language Models

Microsoft has launched Phi-3-mini, a new lightweight AI language model that's simpler and cheaper to run than traditional large language models (LLMs) like OpenAI's GPT-4 Turbo. Its small size means it can run locally, enabling an AI model with capabilities similar to the free version of ChatGPT on a smartphone without needing an internet connection.

Parameter Count and Performance

In the AI field, the size of language models is measured by their number of parameters. Parameters are numerical values in a neural network that determine how the language model processes and generates text. They're learned during training on large datasets and encode the model's knowledge in quantified form. More parameters allow the model to capture more nuanced and complex language generation capabilities, but also require more computing power to train and run.

The largest language models today, like Google's PaLM 2, have hundreds of billions of parameters. OpenAI's GPT-4 is believed to have over a trillion parameters, spread across eight 220-billion-parameter models in a mixture-of-experts configuration. These models require powerful data center GPUs to run properly.

Optimized for Consumer Hardware

Microsoft's Phi-3-mini contains only 3.8 billion parameters and was trained on 3.3 trillion tokens. This makes it ideal for running on consumer GPUs or AI-acceleration hardware found in smartphones and laptops. Phi-3-mini has a context window of 4,000 tokens, but Microsoft has also introduced a 128K-token version called "Phi-3-mini-128K."

Microsoft has also developed 7-billion and 14-billion parameter versions of Phi-3, planned for release later, which the company says are "significantly more capable" than Phi-3-mini.

An Efficient, Robust AI Model

Phi-3's performance is said to "rival models like Mixtral 8x7B and GPT-3.5," according to a report from Microsoft. Mixtral 8x7B, from French AI company Mistral, uses a mixture-of-experts model, and GPT-3.5 powers the free version of ChatGPT.

AI researcher Simon Willison said Phi-3 "looks like a shockingly good small model if their benchmarks reflect what it can actually do." After testing the model on his MacBook, Willison commented, "I got it working, and it's GOOD."

Innovative Training and Environmental Impact

Microsoft's researchers achieved this efficiency by using carefully curated, high-quality training data, including from textbooks. "The innovation lies entirely in our dataset for training, a scaled-up version of the one used for Phi-2, consisting of heavily filtered web data and synthetic data," Microsoft writes.

Small, efficient AI models like Phi-3 could reduce the need for larger models in everyday tasks, potentially significantly reducing AI's environmental impact. Phi-3 is available on Microsoft's cloud platform Azure, as well as through partnerships with Hugging Face and Ollama.