Microsoft's New AI Model Phi-3.5 Outperforms the Competition

Microsoft's new small AI model Phi-3.5 outperforms competitors like Google and OpenAI in several areas, especially reasoning and math tests. The model offers powerful capabilities with potential for integration into IoT devices.

Share
Microsoft's New AI Model Phi-3.5 Outperforms the Competition

Microsoft has released the latest version of its compact language model, Phi-3.5, which proves to be a significant step forward compared to previous generations. Phi-3.5 beats smaller models from industry leaders like Google, OpenAI, Mistral, and Meta in several key areas, making it a standout player in AI technology.

Three Variants With Different Parameter Counts

Phi-3.5 comes in three different sizes: 3.8 billion, 4.15 billion, and 41.9 billion parameters. All versions are available for download and can be run locally with tools like Ollama. The model has particularly excelled at reasoning, where it was only outperformed by GPT-4o-mini among the leading small models. It also performed strongly on math tests, outperforming both Llama and Gemini.

Improved Model With Image Understanding

One of the big pieces of news with Phi-3.5 is that it now also comes in a vision-model version that can understand images, not just text. The model uses what's known as a "mixture of experts" (MoE), meaning learning tasks are distributed across different sub-networks for more efficient processing. This method proved superior to Gemini Flash 1.5, used in the free version of the Gemini chatbot, on several benchmarks. Although Phi-3.5 is smaller than Gemini, it matches both ChatGPT and Claude in performance.

Potential for On-Device AI in IoT Devices

One of the biggest advantages of small models like Phi-3.5 is their potential to be integrated into applications or installed on IoT devices, such as smart doorbells. This would enable features like facial recognition without needing to send data to the cloud, increasing both efficiency and security.

Performance and User Experience

Despite promising benchmarks, the smaller 3.8-billion-parameter version of Phi-3.5 proved somewhat less impressive in practice. When used on a laptop, the answers often felt overly long and awkwardly phrased. The model also struggled with certain simple tests, such as writing a sentence where the first letter of each word matches the last letter of the previous word. The larger mixture-of-experts model, not yet tested by the writer, is expected to solve many of these issues and deliver results of a quality similar to OpenAI's GPT-4o-mini.

Phi-3.5's architecture makes it particularly effective in STEM subjects and social sciences, where it maintains high performance while handling complex AI tasks across multiple languages.