Why the Benchmark King Breaks Code in the FieldThe Real Reason Google Antigravity Isn't Catching On

1. Conclusion: Why the Benchmark King Breaks Code in the Field

"It broke it again."

At Pocholab, we have had at least three instances where we entrusted Google Antigravity with adding or modifying features, resulting in regressions in existing functionality that we then had to ask ChatGPT Codex or Claude Code to fix.

Having Antigravity write it and Codex fix it.

It's as if Google sent a new hire to the field, and OpenAI or Anthropic is doing the code review from behind.

Yet, we still use it sometimes.

"Maybe this time, it has evolved significantly."

There's a morbid curiosity to see it.

Is this just a matter of compatibility? Or is Google truly lagging behind in coding by AI agents?

To cut to the chase, Google is not significantly behind in the intelligence of its AI models. What it lacks is the ability to mold a smart model into a "developer product that can be safely entrusted with work."

What ultimately matters in AI coding isn't the highest benchmark score.

It's about not breaking today the system that was working yesterday.

2. What is Google Antigravity?

Google Antigravity is an agent-based development platform announced by Google in November 2025.

The initial vision for Antigravity was for an AI agent to traverse a code editor, terminal, and browser, plan on its own, write code, launch the application, and even verify its operation in the browser.

In addition to the standard IDE screen, a "Manager Surface" was provided to assign tasks to multiple agents, allowing users to review the work plan, screenshots, and recordings of browser operations as deliverables. Google's goal was not merely code completion, but a "development environment where humans manage AI agents."

With Antigravity 2.0, announced in May 2026, the focus shifted further toward a "command center for agents" that runs multiple agents in parallel. A CLI and SDK were added, and the individual agent features of the former Gemini CLI and Gemini Code Assist were consolidated into the Antigravity CLI.

If you just read the philosophy, it's quite advanced.

However, there is one problem here.

Besides Antigravity, Google has AI Studio, Jules, AI agents in Android Studio, Gemini API, and Gemini Enterprise Agent Platform. They are listed as separate products even on the official Google AI Pro page.

In other words, it's not that Google lacks AI coding technology.

There are too many technologies and products, making it hard for developers to understand "which one they should use every day."

3. Is Gemini 3.1 Pro High Really Weak?

Gemini 3.1 Pro itself is not weak.

According to Google's model card, it scored 80.6% on SWE-Bench Verified, which tests resolving real GitHub issues, and 54.2% on the more difficult SWE-Bench Pro Public. These are fully top-tier numbers for a coding model.

The "Gemini 3.1 Pro High" displayed in Antigravity is not a completely different model, but a setting that allocates high inference compute to Gemini 3.1 Pro. It uses more time and computational resources than Low to think deeply about complex problems.

So, why does a model that is strong in benchmarks cause regressions in actual repositories?

The answer is because model intelligence and agent product quality are different things.

The true capability of an AI coding product is roughly determined by the following multiplication:

Model Inference Capability
× Repository Understanding
× Instruction Retention
× Edit Scope Control
× Test Execution
× Failure Detection
× Fix/Rollback Capability

No matter how smart the model is, if it touches files it shouldn't, ignores existing design conventions, and reports "completed" even when tests fail, it is unusable as a product.

There's an interesting study on this.

In "SWE-Bench Mobile," which had AI agents fix iOS apps close to practical use, 22 combinations of agents and models were tested, and even the best configuration had a success rate of 12%. Furthermore, even using the same model, the performance varied by up to 6 times depending on the agent framework it was mounted on.

Even with the same brain, if the body and the way of working are different, the capability changes 6-fold.

This is a very important fact to consider in AI coding.

4. A High Score on SWE-Bench is Not a Guarantee

SWE-Bench is a useful metric, but it is not a silver bullet.

It is a test that provides a specific GitHub Issue and repository and sees if a test-passing fix can be created. In practice, however, the following circumstances are added:

• Undocumented internal rules
• Unnatural implementations due to past history
• Impacts spanning multiple services
• Data migration and backward compatibility
• Side effects on config files or CI/CD
• Lack of tests themselves
• Specialized usage by different customers

SWE-Bench Pro, which handles longer tasks spanning multiple files, consists of 1,865 issues from 41 active repositories. It is much harder than the standard Verified, and the differences between models become closer to reality.

Google's Gemini 3.1 Pro also scores 80.6% on Verified, but drops to 54.2% on Pro Public.

In other words, it is strong in neatly isolated fix problems, but its success rate drops significantly in real, complex changes. This is a limitation common to all current AI agents, not just Gemini.

AI coding benchmarks are like university entrance exams.

A person who gets a high score on an exam is not necessarily strong in the maintenance and operation of existing systems.

In a production environment, the "ability not to arbitrarily change places that were not asked for" is more important than the ability to solve difficult math problems.

5. Six Reasons Why Google Antigravity Isn't Catching On

5.1. Developers Want Consistent Performance, Not Just Top Scores

If you give an AI agent 10 tasks and it succeeds 9 times, but breaks authentication or payment features on the 10th time, you can't put that tool at the center of production development.

The value of a coding agent is better evaluated with the following formula:

Productivity Gain - Review Time - Incident Investigation - Recovery Work - Anxiety About Recurrence

Even if you can write code 30 minutes faster, if investigating a regression takes 2 hours, you're operating at a loss.

Claude Code and Codex are favored not because they are always right. It's because it's relatively easier to trace their reasons for changes, ask for fixes when they fail, and they possess a high capability to reread existing code and recover.

The experience at Pocholab, where regressions caused by Antigravity were fixed by Codex or Claude Code, clearly illustrates this difference.

In practice, the "ability to diagnose and revert code broken by humans or other AI" is just as important as the "ability to write correctly in the first place."

5.2. Google's Product Strategy is Fragmented

Claude Code can be described as "letting Claude develop in the terminal."

Cursor is "a code editor built around AI."

Codex is "OpenAI's coding agent that handles multiple development tasks locally, in the cloud, and in apps."

Devin is "an autonomous AI software engineer working in the cloud."

Each can be explained in a single sentence.

Meanwhile, at Google, Antigravity, Jules, AI Studio, Android Studio, the transition from Gemini CLI to Antigravity CLI, and the Gemini Enterprise Agent Platform are all running in parallel.

Antigravity itself, after debuting as an IDE-type product, rapidly expanded its scope to a standalone command center managing multiple agents, a CLI, and an SDK. Technologically it's a step forward, but to users, it looks like "starting the next vision before finishing the product."

Google is building multiple runways.

But what developers want to know isn't the grand future vision of the entire airport.

They want to know which flight to board tomorrow morning to reach their destination.

5.3. Rate Limits are Opaque, Making Task Planning Difficult

The Antigravity quota in Google AI Pro is officially described as a "high quota refreshed every 5 hours." However, weekly limits also exist, and consumption fluctuates not by prompt count, but by the amount of work the agent does, the complexity of processing, and Google's supply capacity.

It doesn't explicitly state "how many times you can use it" or "up to how many tokens," and limits are subject to change.

This is troublesome for users.

If you don't know "how many more hours you can make it work," it might stop in the middle of an important implementation. Even if it lasts a long time for simple tasks, it rapidly consumes the quota on tasks that require reading massive repositories, running tests repeatedly, and operating a browser.

In the Google community, there are multiple reports that using the Claude Opus family models on Pro or Ultra hits the limit in ten to tens of minutes, or with a small number of heavy tasks. However, these are user reports, not official tests under standardized conditions.

Even at Pocholab, there was a period when Claude Opus 4.6 could be selected on the ~$20/month plan, but we sometimes hit the rate limit for a certain period after about 30 minutes of work.

Being able to use it is different from being able to continuously use it for work.

Antigravity lists luxury cars on its menu, but they only have a small amount of gas in the tank.

5.4. Offering Unlimited Opus for $20 a Month is Inherently Difficult

The API price for Claude Opus 4.6 was $5 per 1M input tokens and $25 per 1M output tokens. Even higher prices apply to massive contexts exceeding 200k tokens.

An AI coding agent doesn't just process the short request text entered by the user.

Every time, it rereads the code, directory structure, Git diffs, test results, terminal output, and past conversations. If it fails, it retries again.

Therefore, if you let the expensive Opus run freely for a long time on a $20 monthly flat rate, it's highly likely that Google will operate at a loss.

It's not surprising that the limits are strict.

The problem is that it's difficult for users to grasp those limits in advance.

While the model list for Antigravity includes Gemini 3.1 Pro, Claude Sonnet/Opus 4.6, and GPT-OSS, the current plan description explicitly states "access to third-party models" as a perk of Ultra. The conditions of availability vary by time, region, and account, making it hard to understand the scope of usage at a glance even after reading the documentation.

They've succeeded in showing off high-performance models, but it hasn't translated into peace of mind regarding the pricing structure.

5.5. Google is Building a "Massive AI Foundation" Rather Than "One Great Tool"

Google's goal isn't simply to build an IDE like Cursor.

They are trying to build a massive AI ecosystem that runs agents in a secure sandbox on Google Cloud, calls them from SDKs, connects to enterprise platforms, and encompasses Android, search, Workspace, and video generation.

As a corporate strategy, it is rational.

However, building a massive platform and building a tool that programmers want to boot up every morning require different capabilities.

Cursor focused on the editor experience.

Claude Code focused on the terminal.

Codex focused on multiple agents and cloud execution.

From the start, Google tried to grab the IDE, CLI, SDK, cloud infrastructure, enterprise agents, and browser automation all at once.

This is a very Google-like strength, but simultaneously a very Google-like weakness.

5.6. For AI Development Tools, "Being Used by Peers" is a Feature in Itself

A large-scale repository study published in 2026 analyzed over 180 million GitHub repositories and reported that Claude Code was involved in 886,122 commits and 17,295 projects within the verifiable range. Note that this study inferred usage from configuration files and such, so it doesn't capture all usage.

Claude Code's npm package was also recording over 10 million weekly downloads at the time of the study. Since this includes updates, CI, and fetching from multiple environments, it's not identical to the number of users.

In July 2026, OpenAI announced that Codex is used by over 5 million people weekly. Cursor has self-reported being used by over 50,000 companies and 64% of the Fortune 500.

Because the measurement methods differ, direct comparisons cannot be made.

However, at the very least, Google has not announced a clear usage scale for Antigravity equivalent to Codex's 5 million weekly users.

Development tools have network effects.

The more users there are, the more configuration examples, prompts, Rules, Skills, troubleshooting solutions, blogs, videos, and internal know-how multiply. Research also confirms that the adoption of AI coding tools spreads significantly by seeing colleagues using them.

Claude Code and Cursor offer the reassurance that "someone else has already encountered this problem."

With Antigravity, there is still a lingering feeling of "Am I the guinea pig here?"

6. Is Google AI Pro Really a Bad Deal?

I've evaluated Antigravity strictly so far, but Google AI Pro as a whole is a different story.

As of August 2026, the Google AI plans for Japan are structured mainly as follows:

The storage for Google AI Pro has been increased from the previous 2TB to 5TB now. Furthermore, you can share the capacity with up to 5 family members, and those members can also use some AI perks at no extra charge.

Also, Google AI Pro includes features like:

• Expanded usage quotas for Gemini 3.1 Pro and Deep Research
• Gemini integration into Gmail, Google Docs, Sheets, etc.
• Gemini Notebook
• 1,000 monthly AI credits for Google Flow
• Expanded usage quotas for Antigravity, Jules, and AI Studio
• $10 of Google Cloud credits every month
• YouTube Premium Lite and Google Home Premium Standard in eligible regions

Some features have country, language, and age restrictions.

A simple calculation of current consumption means 1,000 Flow credits equal roughly 100 uses of Veo 3.1 Lite, 50 of Fast, or 10 of Quality. In reality, consumption changes depending on settings like generating multiple proposals at once.

For video creators, families that heavily use Google Drive, and people who use Gemini, Notebook, and Gmail integrations, ¥2,900/month is quite cheap.

If you look at Google AI Pro as the "price of an AI coding tool," it's half-baked.

However, if you view it as a comprehensive service bundling 5TB of cloud storage, video generation, research AI, office integration, family sharing, and coding assistance, it has overwhelming strengths that competitors lack.

Google AI Pro isn't a high-end electric screwdriver.

It's a home center where you can rent tools, a warehouse, cameras, an editing room, and a library all at once.

7. Comparison with Claude Code, Codex, Cursor, and Devin

7.1. Claude Code: Have It Read Existing Code and Fix It Carefully

Claude Pro costs $20 a month and includes access to Claude Code. The usage quota is shared between regular Claude and Claude Code, with 5-hour session limits and weekly limits. After reaching the limit, you can use it via additional credits, and a Max plan is available for heavier use.

The strength of Claude Code is that it's terminal-centric with little extraneous UI, making it easy to have it read existing code, formulate plans, and proceed while reviewing diffs.

Rather than flashy demos, it's strong at maintaining existing repositories, refactoring, and investigating incidents.

On the other hand, you can't use it for many hours every day on the $20 Pro plan. If you place it at the center of your operations, you need to consider the Max plan or pay-as-you-go.

7.2. ChatGPT Codex: Strong at Multiple Tasks and Recovery

ChatGPT Plus allows use of Codex for $20 a month. Pro offers 5x quotas for $100 and 20x quotas for $200, with additional credits available after limits are reached.

Codex integrates not just a CLI, but also a desktop app, cloud execution, parallel work by multiple agents, Skills, and Automations.

It is strong at long investigations, changes spanning multiple files, repeated testing, and diagnosing and fixing code created by other agents.

OpenAI's guideline for corporate users estimates the average Codex usage cost per developer at around $100 to $200 per month, assuming that full-scale use will cost more than the Plus tier.

It's not necessarily cheaper than Antigravity.

However, it's relatively easy to see "how much money you'll need based on how much you use it."

7.3. Cursor: The Most Refined "IDE with AI"

Cursor is $20 a month for Pro, $60 for Pro+, and $200 for Ultra.

According to official documents, Pro includes roughly $20 worth of model usage, Pro+ $70, and Ultra $400, transitioning to API-equivalent pay-as-you-go after the limits. They state that developers who use agents daily tend to spend about $60 to $100 a month, and power users over $200.

Cursor's strength is not the model alone.

• Codebase indexing
• Diff reviews within the editor
• Switching between multiple models
• Rules and Skills
• Integration of autocomplete and agent operations
• Clarity of usage and additional billing

It packages all of these into a single IDE experience.

Even if a specific model is temporarily underperforming, you can switch to another. It's not a product for choosing an AI model, but a product for utilizing different models to get work done.

7.4. Devin: Handing Over the Work Rather Than Writing Together

Devin starts at $20 a month for Pro, $200 for Max, and team plans start at a minimum of $80 per month. Pro has daily and weekly limits and a pay-as-you-go tier, while Max offers a larger weekly quota.

Devin's feature is not writing code line-by-line while conversing.

It's about delegating tasks with clear specifications—migrations, routine fixes, backlog processing, refactoring—to an independent worker in the cloud.

If used well, it can make progress while the human is doing other work.

On the other hand, for tasks with ambiguous requests or tasks requiring repeated detailed design judgments, the management cost of instructions and reviews increases.

8. Ultimately, Which Should You Choose?

It's difficult to recommend placing Antigravity as the star of daily development at this point.

However, it has value for the following uses:

• Drafts for new projects
• UI prototypes
• Functional verification involving browser automation
• Investigations using Gemini's long context
• Agent experiments combined with Google Cloud
• Small-scale changes that can be reverted if they fail

Conversely, if you're delegating authentication, payments, data migrations, infrastructure configs, or extensive refactoring, the following safety measures are required:

1. Always make it work in a dedicated branch or Git worktree
2. Save the baseline by running tests, linting, and type checking before making changes
3. Have it submit only a plan first to limit the edit scope
4. Do not let it change multiple features at once
5. Force small, incremental commits
6. Have a different model review it upon completion
7. Do not execute data deletions or migrations without human approval

The setup of having Antigravity implement and letting Claude Code or Codex review the diffs seems tedious, but it's actually rational.

It's not about making AI agents compete, but separating the AI that builds from the AI that doubts.

9. Conclusion: Is Google Truly Losing?

Google invented the Transformer, possesses TPUs, and holds the world's largest search engine, cloud, browser, smartphone OS, and video platform.

They have almost all the technological ingredients.

Yet in AI coding, they have not yet earned the same level of trust as Claude Code, Codex, or Cursor.

The reason is simple.

Google excels at expanding the possibilities of AI, but they struggle to narrow it down into a single tool that one developer can safely use every day.

Gemini 3.1 Pro High is smart.

The vision of Antigravity is also interesting.

The cost performance of Google AI Pro is extremely high when viewed as a comprehensive service.

Even so, if asked whether it can run system development continuously, stably, and with high quality, the answer at present is "still difficult."

What determines the winner in AI coding is not a model that writes brilliant code just once.

• Remembering instructions
• Not touching places it shouldn't touch
• Stopping when tests fail
• Explaining its own changes
• Finding the cause itself when it breaks things
• Working at the same quality again tomorrow

It's these somewhat unglamorous abilities.

Google Antigravity makes you feel the future.

But what the field demands is not a demo that makes you feel the future.

It's a colleague who won't break things today, and who you can trust to continue the work tomorrow.

That's why at Pocholab, we'll probably boot up Antigravity again in a few months.

Thinking, "Surely, it must have evolved this time."

And just to be safe, we'll keep Codex or Claude Code on standby right next to it.