From Jira ticket to pull request: living with an AI coding agent for three months

For the last three months we've had an AI coding agent working against our own repositories. You give it a Jira ticket; it breaks the ticket into steps, writes the code, and opens a pull request. A human reviews and merges, same as any other PR.

This is the feature people are most excited and most skeptical about, in roughly equal measure, so here's a straight account of where it's landed.

The workflow

1. A ticket is tagged for the agent.
2. It reads the ticket — title, description, comments, linked tickets — and the relevant part of the codebase, and produces a plan: the files it expects to touch and the changes per file.
3. A human can approve the plan, tweak it, or reject it. This gate matters; more on that below.
4. It makes the changes, runs whatever checks it can, and opens a PR with a description that explains what it did and why.
5. Normal review. Nothing merges without a person.

What it's genuinely good at

Mechanical, well-specified changes. "Add a created_by field to this model, plumb it through the three endpoints that create the record, add it to the serializer." That's a forty-minute task for a person and the agent does a clean version of it in a few minutes. The PR is boring and correct, which is exactly what you want for that class of work.

Following an existing pattern. If the codebase already has six examples of "how we add a new integration," and the ticket is "add a seventh for tool X," the agent is good at matching the established shape. It reads the neighbours and writes something that looks like they wrote it.

The first draft of a test. Point it at a function and ask for tests and you get a reasonable set of cases, including a couple of edge cases a tired person might skip. You'll rewrite some of them, but starting from a draft beats starting from an empty file.

Tedious refactors with a clear rule. "Every call to this deprecated helper should use the new one; the signatures differ like so." Across thirty files, that's the kind of task where a human makes two typos and the agent makes zero.

What we still do by hand

Anything where the hard part is the decision. If the ticket is "make checkout faster" and the real work is figuring out which of four things to change, the agent can't do that thinking. It'll pick one and implement it competently, and you'll have spent review time on a change that solved the wrong bottleneck.

Changes that span the system's assumptions. A refactor that touches how auth works, or how data flows between two services, needs someone who holds the whole model in their head. The agent works locally and well; it doesn't have the global picture, and a PR that's locally correct can still be architecturally wrong.

Anything customer-facing and subtle. Copy, error messages, the exact behaviour of a form. The agent's version is always fine and rarely right, and the gap is judgment about the user, which it doesn't have.

The plan review is not optional. The times it's gone wrong, the wrongness was visible in the plan before a line of code was written — it misread the ticket, or picked the wrong file, or scoped too broadly. Approving plans carefully is where you catch that cheaply. Teams that skip that gate and review only the final PR spend more time, because now they're untangling code instead of correcting a sentence.

The numbers, roughly

Of the tickets we've routed to it, maybe half produced a PR we merged with light edits. A quarter needed real rework but still saved time versus starting cold. A quarter we ended up doing ourselves, usually because the ticket turned out to need a decision the agent couldn't make.

That's not "it does half our work." It's "it does a clean job on the class of work that's mechanical and well-specified, which is a real fraction of the backlog, and it frees up the people for the class of work that needs a person." The mistake is expecting it to be a junior engineer. It's more like a very fast, very literal contractor who does exactly what the ticket says — which is great when the ticket is good and a problem when it isn't.

Would we recommend it

If your backlog has a steady supply of small, clearly-written tickets — field additions, pattern-following, mechanical refactors — yes, with the plan-review gate treated as mandatory. If your tickets are mostly one-liners that expand into a week of discovery, the agent will amplify the ambiguity rather than resolve it, and you'll get less out of it.

We're keeping it. It's changed what we bother to write tickets for — a chore that used to sit in someone's head as "I'll do that eventually" now gets a two-line ticket, because a two-line ticket is enough for the agent to take a real first pass.

Try it on one ticket — pick something small and well-specified, review the plan closely, and see what comes back.