Library › AI Evals › Customer support teams
File a bug report when the bot quotes the wrong fee
Practice: Asking for a format you can use. About 5 minutes.
About this exercise
Your support bot told a customer there's no fee to cancel, and you need to file it for the bot team. Write exactly 6 numbered lines: Title, Customer said (quote), Bot said (quote), Expected (with the policy section), Severity, and Repro. The bot team's scale: S1 = told a customer a wrong money amount, S2 = wrong but no money involved, S3 = tone only. Plain text, under 150 words.
Common mistakes when you ask AI for this
- Guessing the severity. The AI picks 'High' or 'Medium' because it doesn't know the bot team's S1-S3 scale. Try: Give the scale: "S1 = told a customer a wrong money amount."
- Paraphrasing the chat. The AI summarizes what was said, but the bot team needs the exact words to find the bug. Try: Say: "Quote the customer and the bot word for word."
- No policy or repro. The AI says the answer was wrong without citing the rule or how to trigger it again. Try: Add: "Expected cites the policy section. Repro gives the test-console question."
- A long incident write-up. By default the AI writes a full report with impact, timeline and next steps. Try: Be exact: "Exactly 6 numbered lines."
What a strong prompt includes
- Lists the fields
- Gives the severity scale
- Asks for exact quotes
- Says exactly how many
- Asks for plain text
Related exercises
- Announce a referral bonus with the steps to claim it (Marketing AI · HR and recruiting)
- Announce a team meeting change people will actually follow (Writing AI · Managers and team leads)
- Answer a 'found an error' question in STAR order (Writing AI · Finance and accounting)
- Answer a wholesale inquiry with clear terms (Writing AI · Small business owners)
- Answer an RFP timeline question in exactly four phases (Writing AI · Sales and customer success)