Google releases Gemini 4 Carbon AI model with 88.2% vulnerability detection capability

·by Henderson·Engineering
Google releases Gemini 4 Carbon AI model with 88.2% vulnerability detection capability
Key Points
  • Google launches the Gemini 4 Carbon model, focusing on identifying software vulnerabilities and generating security patches.
  • Carbon achieved an 88.2% detection capability score in the SEC-bench Pro test.
  • The model's success rate in generating secure code is more than twice that of Argon.
  • Google plans to expand Carbon's availability and introduce additional security measures.

Google has announced the launch of its new AI model, Gemini 4 Carbon, designed to identify software vulnerabilities, generate security patches, and handle complex engineering tasks that typically take days to complete. The model scored 88.2% on SEC-bench Pro, a cybersecurity benchmark for testing AI systems' ability to identify real software vulnerabilities. Additionally, Carbon achieved a score of 71% in CWE-bench for fixing security flaws and 54.8% in ExploitGym, indicating its ability to verify whether vulnerabilities can be exploited.

Google noted that these results place Carbon ahead of competing models across all three cybersecurity benchmarks. This announcement comes just two weeks after the launch of Gemini 4 Argon, Google's previous leading model. Carbon builds on that release with improvements in coding, reasoning, cybersecurity, and autonomous task execution.

Performance Advantages of the Carbon Model

Unlike traditional AI assistants, Carbon is designed to execute extended workflows that involve multiple steps, tools, and decisions. Google stated that its internal team observed the model running for days on complex engineering problems, including identifying the causes of software bugs and developing patches within large codebases. However, these extended operation reports were provided by Google and have not been independently verified.

Carbon's cybersecurity capabilities encompass three main stages: finding vulnerabilities, determining whether these vulnerabilities pose real risks, and developing remediation strategies. The model achieved a score of 88.2% on SEC-bench Pro, while its 54.8% score on ExploitGym reflects its ability to verify security vulnerabilities through controlled exploitation tests. The 71% result on CWE-bench measures its performance in generating usable patches to address underlying software weaknesses.

Google also reported that Carbon's success rate in generating secure code during internal testing is over twice that of Argon.

The Chrome team found that Carbon's performance in generating high-quality security patches surpassed that of a competing leading model by tenfold, although Google did not specify the name of the competing model. Besides cybersecurity, Carbon ranks first in the version 5.0 AI analysis index with a score of 59. Its WebDev Arena score also improved by 89 Elo points, reaching 1,768, placing it fourth on the leaderboard. Google stated that these improvements extend to scientific tasks, multimodal understanding, and full-stack web development.

Security Measures and Availability Plans

Google has also introduced additional security measures aimed at reducing unreliable behavior in long-term AI workflows. In internal testing, Carbon's unsupported claims were reduced by about half, and high-severity misalignment alerts decreased by nearly 80% in real-time coding environments involving Google teams compared to Argon. The company also reported a 0% success rate for attacks in the Gray Swan indirect prompt injection benchmark for Carbon, while Argon had a 0.7% success rate.

These results reflect performance under specific testing conditions and do not prove the model's immunity to manipulation.

Google initially restricted Carbon to a small group of cybersecurity defenders, with other trusted partners gaining access through its Fairwind program. The company is also participating in the U.S. government's voluntary pre-release model testing process. Although Google has not announced a specific release date, it plans to provide broader availability for developers, businesses, and consumers. Carbon will adopt an entry pricing model of US$2 per million input tokens (approximately HK$15) and US$10 per million output tokens (approximately HK$78), similar to Argon.

Cached input tokens will receive a 95% discount. The phased rollout reflects the dual-use nature of advanced cybersecurity AI, which can help defenders identify vulnerabilities but may also provide capabilities that could be misused.

ItemSpecification
SEC-bench Pro Score88.2%
CWE-bench Score71%
ExploitGym Score54.8%
WebDev Arena Score1,768

Market Potential and Challenges of the Carbon Model

The launch of Google's Gemini 4 Carbon model in the cybersecurity field demonstrates its strong capabilities in identifying vulnerabilities and generating patches, which are crucial for enhancing software security. As AI technology evolves, Carbon's efficient performance may attract more attention from businesses and developers. However, the performance data of the model has not been independently verified, which may impact its market acceptance. Additionally, Google has implemented extra security measures to address potential misuse risks, reflecting a commitment to the responsible use of AI.

H
About the author
Henderson