Library › AI Evals › Product managers

Pick the one success definition for Pedalbot a test can check

  • For Product managers
  • You practice Setting limits and rules
  • Time about 5 minutes
  • Level Beginner

About this exercise

Spokewise's kickoff notes list five ways to define success for Pedalbot. A usable definition passes three tests: a test set can check it, it has a number to hit, and it measures whether the answer was correct (not how it sounds or how fast it is). Write exactly 3 lines: "Pick: [one letter]", "Why: [one sentence under 20 words]", and "Rejected: [the other four letters, each with a reason of 3 words or fewer, like D (no number)]". Plain text, under 70 words.

Common mistakes when you ask AI for this

  • Picking by gut. The AI explains all five options at length and hedges between C and E. Try: Say: "Pick the one a test set can check, with a number, that measures correct answers."
  • A long Why. The AI writes a paragraph of reasoning instead of one short sentence. Try: Say: "Why is one sentence under 20 words."
  • Rejections without reasons. The AI lists the losers or writes a sentence for each one. Try: Add: "Rejected lists the other four letters, each with a reason of 3 words or fewer."
  • Bold labels and bullets. By default the AI formats a mini report with bold text. Try: Add: "Plain text, exactly 3 lines."
  • Trusting the AI to count words. The AI runs over word limits when it explains choices. Try: Cap each part: "Why under 20 words, each reason 3 words or fewer."

What a strong prompt includes

  • States the three tests
  • Asks for one pick
  • Sets the Rejected format
  • Says exactly 3 lines
  • Asks for plain text

Practice this now ↑

Related exercises