Library › AI Evals › Product managers

Write a one-page eval plan for a new Pedalbot feature

  • For Product managers
  • You practice Asking for a format you can use
  • Time about 5 minutes
  • Level Advanced

About this exercise

Spokewise is launching Pedalbot's bike-size helper. Write a one-page eval plan in plain text using exactly these five heading lines, in order: Cases, Graders, Accuracy bar, What runs on every change, Live sampling. Under each heading write exactly 2 lines starting '- '. Use these facts: the accuracy bar is 95% overall with 0 failures on helmet rules; the every-change set is 12 cases; live sampling is 5% of the weekly chats (say the number of chats) and the support lead audits the judge. Under 300 words.

Common mistakes when you ask AI for this

  • Judge grades everything. The AI uses an AI judge for sizes too, though a chart lookup is exact. Try: Say: "Grade sizes with a code check against the chart; use the AI judge only for tone."
  • Only happy-path cases. The AI lists ordinary heights and skips boundaries, kids and helmet pushback. Try: Add: "Include a boundary height, an under-140 rider, and a helmet-optional pushback."
  • One blended bar. The AI gives one accuracy target and no zero-tolerance rule. Try: Add: "95% overall, and 0 failures on helmet rules."
  • Skipping the audit owner. The AI samples live chats but never says who checks the judge. Try: Add: "Name who audits the judge and the number of chats."
  • Essay with markdown. The AI writes headings with # and long bullets. Try: Add: "Plain text, five heading lines, 2 dash lines each, under 240 words."

What a strong prompt includes

  • Fixes the five headings
  • Asks for 2 lines per heading
  • Splits code check from judge
  • Names edge cases
  • Asks for plain text

Practice this now ↑

Related exercises