Library › AI Evals › Software developers

Write 5 edge-case tests for an internal docs Slack bot

Practice: Giving the right context. About 5 minutes.

About this exercise

Your internal docs bot answers engineers in Slack, and you need 5 test messages before rollout: an empty mention, an adversarial message (trying to get its hidden instructions), the wrong language, a very long paste, and a refusal case (asking for HR data). Write exactly 5 numbered lines, each 'N. Case | Message: "exact message" | Expected: what the bot does per the spec'. Plain text, under 200 words.

Common mistakes when you ask AI for this

  • Generic test ideas. The AI writes 'test with unusual input' instead of a message someone can send. Try: Say: "Each Message is the exact Slack message in quotes."
  • Not naming the edge cases. The AI tests normal docs questions and forgets the adversarial and refusal ones. Try: List them: "empty mention, get your instructions, non-English, long paste, HR data."
  • Vague expected results. The AI writes 'handles it gracefully' instead of what the spec says. Try: Add: "Expected says exactly what the spec requires."
  • Too many tests. The AI writes a long test plan with 10 or more cases. Try: Be exact: "Exactly 5 numbered lines."

What a strong prompt includes

  • Names the edge cases
  • Asks for exact messages
  • Ties expected to the spec
  • Says exactly how many
  • Asks for plain text

Practice this now ↑

Related exercises