Google Gemini 4 Carbon's Benchmark Results Slightly Lag Behind Claude but Clearly Offers Better Value

·by Henderson·AI
Google Gemini 4 Carbon's Benchmark Results Slightly Lag Behind Claude but Clearly Offers Better Value

According to reports, AI research and analysis firm SemiAnalysis recently released its cutting-edge model benchmark GDPx, which includes the performance of Google's new cutting-edge model Gemini 4 Carbon. The list covers major cutting-edge models from companies such as OpenAI, Anthropic, and Meta, and lists the scores and costs of each model.

Google Gemini 4 Carbon Performs Well in Benchmark Tests

The Google Gemini GDPx benchmark includes 206 tasks across various fields such as finance, materials science, cybersecurity, software engineering, law, and biology. The results show that Anthropic's Claude Opus 5.5 ranks first, with Google Gemini 4 Carbon closely following. The score difference between the two is small, but the cost difference is significant. Claude's cost per task is approximately US$51 (about HK$397), while Gemini's cost per task is slightly above US$16 (about HK$124).

Gemini's cost-performance ratio also surpasses that of the higher-ranked GPT-6 Astra, which scored lower than Gemini but has a cost of about US$54 (about HK$421) per task, higher than the two aforementioned models.

The report notes that if cybersecurity, biology, and alignment fields are excluded, the rankings would change. Due to the safety measures of Claude and GPT models, GPT-6 Astra rises to the top, with GPT 6.1 Sol and Claude Sonnet 5.5 also surpassing Gemini 4 Carbon. SemiAnalysis stated that this result aligns more closely with their subjective impressions of each model.

Google's existing cutting-edge model, Gemini 4 Argon, ranks eighth on both lists, ahead of Meta's Muse Wonder but behind Wonder Full. Argon's cost in this benchmark test is only slightly lower than that of Carbon. In the cybersecurity field, both of Google's models scored better than those from OpenAI and Anthropic, but lagged behind Meta's Muse series, which the benchmark test found to be significantly ahead.

Google released Gemini 4 Carbon at the end of October, claiming significant improvements, but has not yet confirmed a public launch date, making it still upcoming for most customers. Google's own benchmark tests indicate that this model outperforms GPT-6 Astra and Claude Opus 5.5 in several aspects.

H
About the author
Henderson