Back to The Haunt
Building with AI3 min read

Stop re-prompting. Debug it.

When AI-written code breaks, the tempting move is to paste the error back into the chat and ask for a fix. Sometimes that works. Often you get a different version of the same mistake, then another, and twenty minutes later you have three patches and still no idea what was wrong.

Re-prompting is guessing with extra steps. The way out is the same as for any bug: find out what is actually happening before you change anything.

Reproduce it on demand

You cannot fix what you cannot trigger. Write down the exact input, command or click sequence that makes it fail, and run it until it fails every time. If it only fails sometimes, that is a clue in itself: timing, shared state, or something outside your code.

The best reproduction is a failing test. It proves the bug exists now and proves it is gone later.

Shrink it

Generated code often arrives in large pieces, so the failure hides in a lot of plausible-looking lines. Cut the input down until removing anything else makes the failure disappear. If it used to work, git bisect will find the commit that broke it in a handful of steps, however long the history.

A small reproduction usually points straight at the cause, and it makes a far better question if you do go back to the assistant.

Read the code that actually ran

Do not read the code you think ran. Follow the stack trace back to where the bad value was created, not where it finally caused a crash. Add a print or a log line, or stop in a debugger, and look at the real values.

This is where AI code differs from your own: you have no memory of why it was written that way. Read it like a stranger's code. Look hardest at the usual weak spots: edge cases such as empty, zero and the last item, missing error handling, and calls to library functions that may not exist in your version.

One idea at a time

Write down what you think is wrong and what you would see if you were right. Then check it, changing one thing only. If you change three things and the bug goes away, you have learned nothing about which one mattered.

Then fix it, and lock it

Make the smallest change that fixes the cause, not the symptom. Keep the failing test: it must fail on the old code and pass on the new. If the tests that came with the generated code all passed while the bug was there, read them again. They may be checking for the bug rather than against it.

Use the assistant inside this loop, not instead of it. Ask it to explain a function, list possible causes, or suggest a test, then check its answer the way you check your own.

The assistant can suggest where to look. Only a reproduction tells you what is true.

Practise on purpose before you need it in production. The free Debugging Is the Skill book teaches this loop, and its companion repository has sixteen planted bugs, including one in a feature an AI wrote with all its tests passing.

That’s it. Come back soon; the ghost keeps writing.Follow by RSS
Read next

Flaky tests need a method, not a retry button.