Library › AI Evals › Managers and team leads

Sort 8 eval jobs into every change, nightly, or never waits

  • For Managers and team leads
  • You practice Setting limits and rules
  • Time about 5 minutes
  • Level Intermediate

About this exercise

You manage Pedalbot's eval pipeline. Every-change jobs must each finish in under 2 minutes and cost under $1 together. Anything that checks a reply before a rider sees it, or alerts on live traffic, never waits on a batch. Everything else runs nightly. Write plain text with three heading lines exactly 'EVERY CHANGE', 'NIGHTLY', 'NEVER WAITS ON A BATCH', each followed by '- Job name: reason under 8 words' lines, using every job once. End with the line 'Every-change cost: $X + $Y = $Z'. Under 190 words.

Common mistakes when you ask AI for this

  • Running everything every change. The AI puts the full regression on every change because more checking sounds safer. Try: Say: "Every-change jobs must be under 2 minutes each and under $1 together."
  • Treating live checks as batch. The AI schedules the live filter and burst alert nightly. Try: Add: "Anything checking a reply before the rider sees it, or alerting on live traffic, never waits on a batch."
  • Missing the cost line. The AI never adds up the every-change cost. Try: Add: "End with: Every-change cost: $X + $Y = $Z."
  • Long explanations. The AI writes a paragraph per job under markdown headings. Try: Add: "Plain text, reason under 8 words per job."

What a strong prompt includes

  • States the time limit
  • States the cost limit
  • Names the three groups
  • Uses every job once
  • Asks for the cost sum line

Practice this now ↑

Related exercises