AI Coding Agents: A Practical Field Report

What I learned from testing Junie, Claude Code, and Antigravity on real projects

During my recent holidays, I decided to test more intensively three AI coding agents: Junie from JetBrains, Claude Code from Anthropic, and Antigravity from Google. My goal was simple — understand what these tools can actually provide when working on real (already existing) software, not just demo applications.

Illustration image created using ChatGPT.

The Testing Ground

I structured my experiments around two types of projects. First, I tested all three coding agents on a Python + React application to enable direct comparison. In parallel, I explored using only Claude Code on Swift and SwiftUI applications to understand how it handles native iOS development.

First Impressions: The Setup Experience

All three tools support guideline files — markdown documents where you can define project-specific instructions and conventions. Junie uses .junie/guidelines.md, Claude Code looks for CLAUDE.md, and Antigravity expects GEMINI.md.

Junie and Claude Code facilitate things by offering to auto-generate these guideline files based on your project structure (but you’ve got the time to do your laundry while they works, and the projects weren’t really big (about 20k lines of code for the bigger ones)). Antigravity, however, doesn’t seem to offer this convenience, at least not that I’ve discovered yet.

From a user experience perspective, I found myself gravitating toward Junie’s interface. The integration with JetBrains IDEs felt natural — unsurprising given my longstanding preference for JetBrains over VSCode. The readability and workflow felt more polished for daily use.

On the pricing front, Antigravity’s free tier is generous in principle, though the free quota disappears quickly with actual use. Still, there’s value in being able to experiment without immediate cost.

As for the capacity to customise the agents to respect my coding conventions (ie when declaring a pydantic model in python to define a field alias to use camelCase instead of snake_case to reduce JSON size) the clear winner is Claude Code with a quite clear documentation on how to use hooks, skills and tools to extends it’s capacities.

The Promise: AI-Generated Suggestions

My first real test was asking each agent to analyze the codebase and generate a suggestions.md file with improvement ideas. This is where the tools genuinely shined.

The suggestions were surprisingly insightful. Some identified issues already on my todo list, validating their understanding of the code. Others proposed improvements I hadn’t considered but immediately recognized as valuable. This analytical capability alone makes these tools worthwhile.

The Reality: Implementation Challenges

When it came to actual coding tasks, my enthusiasm became more measured.

Speed Isn’t What You Think

Coding agents are not fast in the traditional sense. A significant portion of execution time goes into planning — determining which files to modify and how to structure changes. Even with detailed guideline files already in place, this overhead remains substantial.

For context, I asked an agent to replace all calls from MongoDbHandler.load_instance to MainDbHandler.load_instance. This straightforward find-and-replace task took longer with the AI than if I’d simply used my IDE’s search-across-files feature.

The real productivity gain comes from parallelization and asynchronous work. You can run multiple agents simultaneously on different projects or isolated environments, and they continue working while you’re otherwise occupied — whether that’s reviewing other code or, in my case, spending time with my kids.

The Quality Question

Testing the generated code revealed consistent patterns of issues. These aren’t necessarily mistakes you wouldn’t make yourself, but they’re mistakes you might avoid or catch faster writing code “by-hand”.

The “Any-as-a-Cure” Problem

Without strict guidance through hooks or explicit instructions, agents tend to fix symptoms rather than root causes. In TypeScript projects, I repeatedly saw what I call “acute-as-any disease” — when facing type incompatibilities, the agent would add as any casts to silence errors instead of addressing the underlying type mismatch.

Context Confusion

On my SwiftUI application, the agent generated perfectly valid UI improvement code — but placed it in the wrong class. I have multiple classes implementing the same protocol, and the changes went into an irrelevant implementation, corresponding to another screen. The code was quite correct; the location wasn’t.

Inconsistent Attention to Detail

Some generated code used deprecated features that should have been flagged. When adding new data to display, the agent forgot to include translations for new text, despite having properly translated text in previous edits to the same file.

This could probably have been avoided by specifying directly in the initial request or in the guidelines file to always remember to do the translations. But I am surprised that sometimes he remembers to do it himself and other times he doesn’t; it’s inconsistent behavior.

The Iterative Reality

Don’t expect working code on the first attempt. Agents typically follow an edit-test-fix cycle: they make changes, attempt to compile or run tests, then address any errors detected. This continues until no obvious issues remain. It isn’t really fast and can consume a portion of your credits/usage plan.

During my testing and preliminary reading/corrections of AI-generated code, I identified four recurring categories of problems:

1. Code Duplication

Agents create new files similar to existing ones when a simple modification to the original would suffice. They don’t always recognize opportunities to reuse existing code for new use cases. Do not underestimate this issue, the more big is your codebase the more likely AI is to make mistakes while working on it.

Sometimes things are even worse, you’ve got unused code you weren’t asking for and files that aren’t referenced anywhere.

2. Partial Edits

Changes are applied to some parts of the codebase but miss other locations where the same modification should occur. This inconsistency can be subtle and hard to catch and can be aggravated by code duplication.

3. Linting and Typing Issues

Good practices aren’t always respected. Type safety gets compromised. Linting rules may be violated or, worse, explicitly disabled to make code pass checks.

4. Runtime Errors

Logic errors that only manifest during execution, not compilation. These are the most dangerous because they can slip through initial testing.

Mitigating the Risks

The first two categories can be reduced through careful use of instructions and hooks. You’re essentially teaching the agent your project’s patterns and conventions.

For linting and typing issues, instructions can help agents detect problems, but prevention is harder. Will guidelines alone prevent an agent from taking shortcuts with type safety? My experience suggests only partially.

Runtime errors demand a robust testing foundation. You need solid unit tests, integration tests, and end-to-end coverage before letting AI agents modify existing code. Even then, risks remain. My recommendation: request limited, focused tasks and perform thorough reviews rather than asking for complete features in one go.

The Verdict: Junior Assistant, Not Senior Developer

AI coding agents are remarkable tools, but they’re best understood as junior assistants rather than autonomous developers.

Before integrating these tools into serious workflows, establish safeguards:

• Comprehensive test suites (AI can help you write these)
• Strict linting rules and enforcement
• Detailed guideline files with project-specific instructions
• Static analysis tools to catch issues quickly
• Code review processes that treat AI-generated code with appropriate scrutiny

The current state of AI coding agents presents us with two equally unfortunate extremes: not using them at all, or relying on them blindly. The productive middle ground requires treating them as capable but inexperienced team members who need clear guidance, oversight, and verification.

Used thoughtfully, they can accelerate certain tasks, generate useful suggestions, and handle routine modifications while you focus on architecture and complex problem-solving. Used carelessly, they’ll introduce subtle bugs, technical debt, and maintenance headaches.

The technology is indeed useful — just not in the way the marketing might suggest. Approach with realistic expectations, implement proper safeguards, and you’ll find these tools can meaningfully enhance your development workflow.

Although I experimented with LLMs and RAG early on, I have only recently begun to look into the actual usability of AI coding agents. One reason for this is that my past experiments, as well as the code I had the opportunity to read, were far from convincing and did not meet my criteria. Another reason is my distrust of marketing communications that embellish or even invent things that can be done while glossing over the real limitations. Having had the opportunity during my vacation to run agents on elements with no real risk while I was doing other things allowed me to reassess what can and cannot be achieved.

I provided Claude with my notes to write a first draft of this article before I proofread it to make sure it matched what I wanted to say. As for the illustration, I used ChatGPT to create one for me.