Library › AI Evals › Product managers

Draft 5 edge-case tests for an AI trip planner

Practice: Giving the right context. About 5 minutes.

About this exercise

Before Wayloft's AI trip planner launches, you need 5 test cases your QA contractor can run exactly as written: one each for an empty input, an adversarial user, the wrong language, a very long trip, and a refusal case. Write exactly 5 numbered lines, each 'N. Case name | Input: "exact user message" | Expected: what the planner should do per the spec'. Plain text, under 220 words.

Common mistakes when you ask AI for this

  • Generic test cases. The AI writes 'test that the planner handles errors gracefully', which no one can run. Try: Say: "Each Input is an exact user message in quotes."
  • Not naming the edge cases. The AI picks easy happy-path tests like 'plan 3 days in Rome' and forgets adversarial ones. Try: List them: "empty, adversarial, wrong language, very long trip, refusal."
  • Vague expected results. The AI writes 'responds appropriately' instead of what the spec actually requires. Try: Add: "Expected says exactly what the spec requires."
  • Too many tests. Asked for tests, the AI writes 10 to 20 and pads them with the hotel API. Try: Be exact: "Exactly 5 numbered lines."
  • Getting markdown back. AI defaults to bold labels, headings and tables, which break when you paste into a doc or tracker. Try: Ask for "plain text, no bold, no headings, no tables."
  • Trusting the AI to count words. AI is bad at counting. Ask for a total and it often lands 20% over. Try: Cap each line instead: "each line under 20 words."

What a strong prompt includes

  • Names the edge cases
  • Asks for exact inputs
  • Ties expected results to the spec
  • Says how many
  • Asks for plain text

Practice this now ↑

Related exercises