Library › AI Evals › Customer support teams

Turn "the bot feels pushy" into 4 testable checks

Practice: Catching mistakes and made-up facts. About 5 minutes.

About this exercise

Customers keep telling agents your cancellation bot 'feels pushy', and you need checks a QA reviewer can apply to one chat. Write exactly 4 numbered checks, each 'N. Fail if [something you can see or count in one chat] - e.g. "quote from the feedback"'. Don't use vague words like pushy, aggressive or salesy as the test, and skip praise and unrelated complaints. Plain text, under 130 words.

Common mistakes when you ask AI for this

  • Keeping the vague words. The AI rewrites 'pushy' as 'respects the customer's choice', which is just as hard to grade. Try: Say: "Each check must be something you can see or count in one chat."
  • No example from real data. The AI writes generic retention tips instead of tying checks to what customers said. Try: Add: "Quote an example from the feedback on each line."
  • Banning all offers. The AI fails any chat with an offer, but retention wants to keep one. Try: Add: "One save offer is fine. Skip praise and unrelated complaints."
  • Too many checks. The AI turns every comment into a check. Try: Be exact: "Exactly 4 numbered checks."

What a strong prompt includes

  • Asks for observable checks
  • Asks for quoted examples
  • Says exactly how many
  • Allows one offer
  • Asks for plain text

Practice this now ↑

Related exercises