Google has released Gemma 4, a family of open-weight AI models that the company describes as its most capable openly available release to date. Announced in late May 2026, the lineup spans dense and mixture-of-experts (MoE) architectures and is designed to handle the kind of multi-step reasoning and tool-calling tasks that power AI agents. Independent researchers have already begun benchmarking the models against top competitors from Microsoft and Alibaba, and the early results paint a nuanced picture of where Gemma 4 excels and where rivals still hold an edge.
What Gemma 4 actually ships with
The Gemma 4 family includes several model sizes. The flagship is a 27-billion-parameter MoE model that activates only a fraction of those parameters on any given query, a design that lets it punch above its weight in accuracy while keeping memory and latency manageable. Google also released smaller dense variants aimed at mobile and edge deployment, where every megabyte of VRAM counts.
Unlike its predecessor Gemma 3, which was limited to text and image inputs, Gemma 4 accepts text, images, audio, and video. That multimodal breadth matters for agentic workflows, where a model might need to read a screenshot, listen to a voice command, and then call an external API to complete a task. Google’s model card describes built-in support for function calling and structured output, two capabilities that let developers wire the model into larger software systems without extensive prompt engineering.
The models ship under Google’s Gemma license, which permits commercial use, fine-tuning, and redistribution with certain conditions. Developers planning production deployments should review the specific terms on Google’s Gemma page, as the license includes usage restrictions around harmful applications and requires attribution.
How it stacks up: the first independent benchmarks
The most detailed outside evaluation so far comes from a research preprint titled “Gemma 4, Phi-4, and Qwen3: Accuracy-Efficiency Tradeoffs in Dense and MoE Reasoning Language Models,” published on arXiv in late May 2026. The authors ran Gemma 4 variants through a battery of reasoning and language-understanding benchmarks alongside Microsoft’s Phi-4 and Alibaba’s Qwen3, measuring accuracy, inference latency, and VRAM consumption under controlled conditions.
Their findings highlight tradeoffs rather than a clean sweep for any single model family. According to the preprint, Gemma 4’s 27B MoE variant scored approximately 74% accuracy on the paper’s composite reasoning benchmark, compared with roughly 71% for the closest Qwen3 MoE configuration and about 69% for Phi-4’s dense model of similar activated parameter count. Latency told a different story: Phi-4 completed short-prompt inference in around 38 milliseconds per token on the tested A100 setup, while Gemma 4 MoE came in near 45 ms/token under the same conditions. VRAM usage favored the MoE design, with Gemma 4 27B MoE requiring roughly 18 GB at a 4,096-token context length versus approximately 26 GB for a comparably accurate dense Qwen3 variant. On certain individual benchmarks, specific Qwen3 configurations matched or exceeded Gemma 4, and Phi-4 showed competitive latency on shorter prompts. The paper frames the comparison explicitly as a map of tradeoffs, not a ranking, which is the honest way to read it.
For developers, the practical value of the preprint is its hardware-specific data. The authors report memory footprints at different context lengths and timing results across GPU configurations, giving teams a way to estimate whether a particular Gemma 4 variant will fit on their available hardware before they commit to integration work.
A standard caveat applies: arXiv preprints, hosted by Cornell University’s arXiv platform, are screened for basic quality but have not undergone formal peer review. The methodology is detailed enough for replication, which means other groups can verify or challenge the numbers as they run their own tests.
What MoE means for the rest of us
Mixture-of-experts is the architectural idea driving much of Gemma 4’s efficiency story, and it is worth a brief explanation for readers who do not spend their days tuning model configs. A traditional “dense” model activates every one of its parameters for every input. An MoE model splits its parameters into groups of specialists and routes each input to only the most relevant group. The result is a model that has a large total parameter count, giving it broad knowledge, but only uses a slice of that count on any single query, keeping compute and memory costs down.
This matters because it directly affects who can run the model. A dense 27-billion-parameter model might require a high-end data-center GPU. An MoE model of the same total size, activating perhaps 8 billion parameters per query, could fit on a prosumer card or even a well-equipped laptop. Google’s bet with Gemma 4 is that MoE lets it offer frontier-class reasoning in a package that more developers and researchers can actually afford to deploy.
The competitive landscape is crowded
Gemma 4 enters a field that has grown significantly more competitive since Google released Gemma 3 in early 2025. Meta’s Llama 4 family, Alibaba’s Qwen3, Microsoft’s Phi-4, and Mistral’s latest releases all target overlapping use cases: reasoning, code generation, and agentic tool use. Each family makes different architectural and licensing choices, and no single model dominates across all benchmarks and deployment scenarios.
What distinguishes Gemma 4 in this crowd is the combination of native multimodal input, MoE efficiency, and Google’s explicit focus on agentic capabilities like function calling. Whether that combination translates into real-world advantages over, say, a fine-tuned Llama 4 variant or a Qwen3 agent pipeline depends on the specific application, hardware budget, and integration requirements of each team.
Gaps that still need filling
Several important questions remain open. Google has not published a detailed technical report describing Gemma 4’s training data composition, red-teaming process, or alignment methodology at the level of detail that accompanied some earlier releases. Organizations deploying the models in sensitive domains, such as healthcare, finance, or education, will need to run their own safety evaluations rather than relying on the model card alone.
The arXiv preprint, while valuable, tests reasoning benchmarks under controlled conditions. It does not evaluate end-to-end agentic tasks like multi-step tool use, long-horizon planning, or interaction with live APIs and browsers. Those are the scenarios where “agentic workflows” either succeed or fall apart, and no published study has yet put Gemma 4 through that gauntlet.
Real-world deployment data is also missing. Benchmark accuracy on a research GPU cluster does not guarantee smooth performance when a model is quantized, served behind a load balancer, and hit with unpredictable user inputs at scale. Early adopters will be generating that data over the coming weeks, and their reports will matter more than any single preprint.
Practical next steps for teams evaluating Gemma 4 in June 2026
Gemma 4 has arrived with enough independent validation to justify serious evaluation and enough open questions to discourage blind adoption. The preprint’s benchmark numbers give teams a structured starting point for comparison, especially if reasoning accuracy and hardware efficiency are top priorities. Google’s own documentation provides the licensing and integration basics.
The most productive next step for any team considering the models is straightforward: match the preprint’s reported configurations to your available hardware, reproduce the benchmarks most relevant to your application, and then run targeted tests on your own data, including safety and robustness checks. The open-weight AI landscape moves fast, and the teams that build their own evidence base, rather than waiting for a consensus ranking that may never arrive, will be the ones best positioned to ship.
More from Morning Overview
*This article was researched with the help of AI, with human editors creating the final content.
