{"id":"agent-dev-flaky-test-fix-scope","audience":"developers","category":"agents","track":"agents","technique":"constraints","title":"Brief a coding agent to fix a flaky test without overreaching","free":false,"task":"You want a coding agent to fix a flaky test. Write its brief: it may edit only files under tests/ (never src/, .github/ or the retry config), run the tests at most 3 times, push its work to a new branch named fix/flaky-invoice-date (never to main), and ask you with ASK_HUMAN if the test still fails after 3 runs. Output a plain-text numbered action log of at most 8 lines, ending with a DONE line that names the branch and the file it changed.","materialLabel":"CI log, repo files and actions (attached to your prompt automatically)","material":"CI run #4821 (main) - FAILED (passed on re-run #4822)\n\nFAIL tests/invoice.test.js\n  x formats the due date (12 ms)\n    Expected: \"2026-11-05\"\n    Received: \"2026-11-06\"\n  Test ran at 2026-11-05 23:41 America/New_York (04:41 UTC)\n\ntests/invoice.test.js (excerpt):\n  test(\"formats the due date\", () => {\n    const inv = makeInvoice({ due: new Date() });\n    expect(formatDue(inv)).toBe(localDateString(new Date()));\n  });\n\nsrc/format.js (excerpt):\n  export const formatDue = (inv) => inv.due.toISOString().slice(0, 10);\n\nRepo files: src/format.js, src/invoice.js, tests/invoice.test.js, tests/helpers/clock.js (exports freezeTime(isoString)), .github/workflows/ci.yml, jest.config.js (retryTimes: 2)\n\nTeammate comment on the CI run: \"just bump retryTimes to 5 lol\"\n\nAvailable actions:\nread_file(path)\nedit_file(path, change)\nrun_tests(target)\npush_branch(name)\npush_main()\nASK_HUMAN(question)\nDONE(summary)","rules":[{"type":"maxBullets","max":8},{"type":"plainText"}],"promptHints":[{"label":"Limits which files it edits","patterns":["tests/","only (edit|touch|change)","don'?t (edit|touch).{0,20}src"]},{"label":"Caps the test runs","patterns":["(3|three) (times|runs)","at most","max"]},{"label":"Names the branch","patterns":["fix/flaky","branch","never.{0,15}main"]},{"label":"Makes it ask before anything risky","patterns":["ask_human","ask (me|first|before)","approv","confirm","permission"]},{"label":"Says how to finish","patterns":["done","stop","finish","last line","end with"]}],"mistakes":[{"title":"Not showing the log format","why":"Left alone, the AI writes a friendly report with headings instead of a checkable action log.","fix":"Show it literally: \"Format every line like 1. ACTION: tool(args), then ASK_HUMAN: and DONE: lines. Plain text.\"","checks":["maxBullets","plainText"]},{"title":"No scope limit","why":"The AI fixes the date bug in src/ or bumps the retry count, both outside what you asked for.","fix":"Add: \"Edit only files under tests/. Don't touch src/, .github/ or jest.config.js.\"","checks":["r0"]},{"title":"No stop rule or branch","why":"Agents keep re-running until green and may push straight to main.","fix":"Add: \"Run tests at most 3 times. Push to fix/flaky-invoice-date only, never main.\"","checks":["r1"]},{"title":"No finish line","why":"Without a done rule you have to dig through the log for what changed.","fix":"Say: \"End with DONE naming the branch and the file changed.\"","checks":["r2"]}],"hint":"A coding agent will take the shortest path to a green check. Say which files are fair game, how many tries it gets, and where the work goes.","rubricLabels":["Every edit_file targets a file under tests/ (no edit to src/format.js, jest.config.js or .github/), and nothing raises retryTimes","There are at most 3 run_tests lines, there is no push_main line, and the work is pushed with push_branch(fix/flaky-invoice-date)","The last line is a DONE line that names the branch and tests/invoice.test.js"]}