Library › AI Evals › Customer support teams

Plan which bot chats your team reviews each week

Practice: Setting limits and rules. About 5 minutes.

About this exercise

Your team can spend 5 hours a week reviewing support bot chats at 3 minutes per chat. Write exactly 4 numbered lines: Capacity (the math), Always review (the high-risk groups from the data, with their total), Random sample (how many of the remaining chats, drawn at random across the whole week), and Sample rate (the math as a percent). Plain text, under 100 words.

Common mistakes when you ask AI for this

  • Sampling the easy way. The AI accepts 'read the first 100 on Monday', which never covers weekend chats. Try: Say: "Random sample drawn across the whole week, weekend included."
  • Forgetting the must-review chats. The AI samples everything randomly and misses the high-risk groups your policy covers. Try: Add: "Always review the high-risk groups first, then sample the rest."
  • Hiding the math. The AI states '100 chats' or '2%' without showing how it got there. Try: Say: "Show the math on each line, like 5 x 60 / 3 = 100."
  • A QA strategy essay. By default the AI writes a long plan with scoring rubrics and tools. Try: Be exact: "Exactly 4 numbered lines: Capacity, Always review, Random sample, Sample rate."

What a strong prompt includes

  • Asks for the math
  • Reviews high-risk chats first
  • Samples the whole week
  • Says exactly how many
  • Asks for plain text

Practice this now ↑

Related exercises