Library › AI Evals › Software developers

Write a bug report for a summary that dropped a charge

Practice: Asking for a format you can use. About 5 minutes.

About this exercise

Your ticket summarizer dropped a customer's double-charge complaint, and you need a bug report for the tracker. Write exactly 6 numbered lines labeled Title, Input, Expected, Actual, Severity and Likely cause. Use the team scale: P1 means a money or safety issue was missed, P2 a wrong but harmless summary, P3 style only. Base the cause on the logs, not guesses. Plain text, under 200 words.

Common mistakes when you ask AI for this

  • Guessing the severity. The AI picks 'High' or 'Medium' on its own scale because it doesn't know your team's P1-P3 rules. Try: Give the scale: "P1 = money or safety missed."
  • Fuzzy expected vs actual. The AI writes 'summary was incomplete' without saying what was missing. Try: Say: "Expected and Actual name the exact missing detail and the ticket id."
  • Repeating the Slack guess. The AI echoes 'switch to a bigger model' instead of reading the truncation warning. Try: Add: "Base the cause on the logs, not opinions in Slack."
  • A long write-up. The AI adds steps to reproduce, environment and next actions until it's a page long. Try: Say: "Exactly 6 numbered lines: Title, Input, Expected, Actual, Severity, Likely cause."
  • Getting markdown back. AI defaults to bold labels, headings and tables, which break when you paste into a doc or tracker. Try: Ask for "plain text, no bold, no headings, no tables."

What a strong prompt includes

  • Lists the fields
  • Gives the severity scale
  • Points to the logs
  • Says how many lines
  • Asks for plain text

Practice this now ↑

Related exercises