Google says its new Gemini model makes things up more than five times less often

Every chatbot user has met the confident wrong answer: the invented citation, the product feature that does not exist. Google is now claiming a large cut in that failure. Announcing Gemini 4 Hydrogen on Thursday, Nov. 12, the company said the model produces more than five times fewer fabricated claims than Gemini 3.8 Flash, the model it replaces as the fast, low-cost option.

Read next on Morning Overview: Neanderthals in Spain carved a wooden object that matches nothing archaeologists have ever found

The figure is Google’s own, measured against Google’s own earlier model, and the launch posts do not carry an independent test. It still arrives alongside a set of speed, price and safety numbers that show where the company thinks the model fits.

The fabrication and speed claims, with their baseline

The announcement came in a Google blog post by Koray Kavukcuoglu, senior vice president of Google DeepMind and the company’s chief AI architect. He described Hydrogen as Google’s daily driver for agentic workflows, a fast and cost-efficient model that sits below the frontier models Gemini 4 Carbon and Gemini 4 Argon. Google says Hydrogen cuts fabricated claims by more than 5x compared with 3.8 Flash and finishes tasks about 2.5x faster at its default high thinking level.

The comparison point matters. Five times fewer than an older small model does not mean five times fewer than a rival’s, and the post gives no absolute error rate in the material available here. 9to5Google repeated the claim and noted that the performance figures come from Google and its partners without independent verification.

Partners in early testing reported runs 2x to 2.8x faster and 12% to 23% lower token use on data-heavy workflows, according to the post. A separate honesty figure from Google’s internal agent evaluation put the model at 94%, a result only Google can reproduce.

The naming is part of the story. 9to5Google says Hydrogen replaces the Flash branding in the Gemini 4 family and follows the availability of Carbon earlier in the same week. In Antigravity, Google’s coding environment, the launch adds an Extra thinking tier alongside Low, Medium and High, which is where the highest-effort results are meant to come from.

Benchmarks, price and the million-token window

Hydrogen takes input of up to 1 million tokens and can output up to 1 million tokens for multimodal work. Google prices it at $1.50 per million input tokens and $7.50 per million output tokens, flat across the full context window, a figure repeated by Resultsense.

On the Artificial Analysis Intelligence Index, Google reports a score of 51 at high thinking, against 50 for Argon, and 54 at a new xhigh setting. On Terminal-Bench 4.1, a coding-agent test, Hydrogen scores 57%, compared with 18.2% for 3.8 Flash. Google also lists 58.8% on HLE-Diamond and first place on the Vals Tax Agent Bench at 52.3%.

Resultsense points out that most of the comparisons are against Google’s older Gemini 3.8 Flash and that the Terminal-Bench version differs between two of the claims, 4.1 for the speed result and 4.0 for the Orchestrator pairing result. A second roundup flags the same inconsistency and says independent testing across broader tasks is still needed before Hydrogen can be compared with rival small models.

Hydrogen has four thinking levels, low, medium, high and xhigh, with high as the default, so the 2.5x speed claim describes the default setting and not the cheapest one. That detail matters for anyone budgeting API spend, because the price per token stays the same while the amount of reasoning, and therefore the token count, changes with the level chosen.

Who gets Hydrogen, and how free users reach it

Google says the rollout is global. Developers get it through the Gemini API and AI Studio, Android Studio and Google Antigravity, where paid AI Pro and Ultra subscribers can use an Orchestrator mode that switches between Carbon and Hydrogen in one session. Google says that mode approached Carbon’s success rate on Terminal-Bench 4.0 while using about 24% less quota and finishing about 19% faster.

Consumers see it in the Gemini app model picker and in Google Search. Notebookcheck reports that free users cannot choose Hydrogen by hand. They receive it only through auto model selection when Google judges it the right model for the prompt, and most free prompts still go to Flash-Lite. With auto selection off, every free reply comes from Flash-Lite.

In Google Search, 9to5Google says Hydrogen is the default model for AI Mode and handles the hardest questions in AI Overviews. Notebookcheck adds a wrinkle for paying consumers: Google’s plans page lists Hydrogen as the lead model for Google AI Plus with no Pro model, while a help page still lists Pro, and Google emailed Plus subscribers that they would lose Pro models on Friday, Nov. 13.

Google also claims safety gains, including a 94% honesty rate on its internal agent evaluation, agentic overreach reduced by up to 81%, and false refusals on HealthBench Pro under 0.4%. Other safety claims: misaligned actions and task shortcutting down about 12x in tool-use and coding evaluations, and more than 2.3x more resilience to indirect prompt injection on the Gray Swan benchmark. An outside test of the five-fold fabrication claim, run on tasks Google did not choose, is the piece still missing.

This article was produced with the assistance of AI and reviewed by Morning Overview editors prior to publication.