Library › AI Evals › Customer support teams

Write a 4-line pass/fail rubric for bot handoff notes

Practice: Setting limits and rules. About 5 minutes.

About this exercise

When your chatbot hands a chat to a human, it writes a handoff note, and agents keep complaining about them. Write exactly 4 numbered criteria, each 'N. Pass if ...; Fail if ...', binary only. They must cover: the order number, what the bot already tried, a mood label from exactly Calm, Upset or Angry, and a 60-word limit. Plain text, under 160 words.

Common mistakes when you ask AI for this

  • Getting a rating scale. Asked for a rubric, the AI invents 1-5 levels like 'Excellent' and 'Needs work'. Try: Say: "Binary only: each line is 'Pass if ...; Fail if ...'."
  • Not naming the must-haves. The AI writes generic criteria like 'is clear and concise' instead of what agents need. Try: List them: "order number, what the bot tried, mood label, 60-word limit."
  • A fuzzy mood check. The AI accepts any feeling word, so 'seems kind of annoyed' still passes. Try: Add: "Mood must be exactly Calm, Upset or Angry."
  • Too many criteria. The AI writes 8 or more criteria, including tone and politeness. Try: Be exact: "Exactly 4 numbered criteria, one line each."

What a strong prompt includes

  • Asks for binary pass/fail
  • Names the must-haves
  • Says exactly how many
  • Limits each line
  • Asks for plain text

Practice this now ↑

Related exercises