Library › Agents & Automation AI › Software developers

Brief a coding agent to fix a flaky test without overreaching

  • For Software developers
  • You practice Setting limits and rules
  • Time about 5 minutes

About this exercise

You want a coding agent to fix a flaky test. Write its brief: it may edit only files under tests/ (never src/, .github/ or the retry config), run the tests at most 3 times, push its work to a new branch named fix/flaky-invoice-date (never to main), and ask you with ASK_HUMAN if the test still fails after 3 runs. Output a plain-text numbered action log of at most 8 lines, ending with a DONE line that names the branch and the file it changed.

Common mistakes when you ask AI for this

  • Not showing the log format. Left alone, the AI writes a friendly report with headings instead of a checkable action log. Try: Show it literally: "Format every line like 1. ACTION: tool(args), then ASK_HUMAN: and DONE: lines. Plain text."
  • No scope limit. The AI fixes the date bug in src/ or bumps the retry count, both outside what you asked for. Try: Add: "Edit only files under tests/. Don't touch src/, .github/ or jest.config.js."
  • No stop rule or branch. Agents keep re-running until green and may push straight to main. Try: Add: "Run tests at most 3 times. Push to fix/flaky-invoice-date only, never main."
  • No finish line. Without a done rule you have to dig through the log for what changed. Try: Say: "End with DONE naming the branch and the file changed."

What a strong prompt includes

  • Limits which files it edits
  • Caps the test runs
  • Names the branch
  • Makes it ask before anything risky
  • Says how to finish

Practice this now ↑

Related exercises