Library › AI Evals › Managers and team leads

Make a launch call from an AI scheduler's eval results

Practice: Catching mistakes and made-up facts. About 5 minutes.

About this exercise

You have to decide whether your team's AI meeting scheduler launches Monday. Write exactly 4 numbered lines, one per test category, each 'Category: X of Y = Z% vs bar B% - Pass or Fail', then a last line 'Decision: Launch or Hold - reason'. Use only results from the current prompt version, and note that the ops director raised the time zones bar to 95% yesterday. Plain text, under 110 words.

Common mistakes when you ask AI for this

  • Using the old result. The table has two rescheduling rows, and the AI may pick the better-looking v5 one. Try: Say: "Use only v6 results."
  • Averaging it all together. The AI repeats the 93% overall number, hiding the one category that fails. Try: Add: "Judge each category against its own bar."
  • Hiding the math. The AI says 'rescheduling is a bit low' without numbers. Try: Show the shape: "Category: X of Y = Z% vs bar B% - Pass or Fail."
  • Using the old bar. The table still shows 90% for time zones, so the AI passes a category that now fails. Try: Tell it what changed: "The time zones bar is now 95%."
  • A long memo. The AI writes a launch memo with risks and next steps. Try: Be exact: "4 numbered lines, then one Decision line."

What a strong prompt includes

  • Uses only the current version
  • Compares each category to its bar
  • Asks for the math
  • Asks for a clear decision
  • Asks for plain text

Practice this now ↑

Related exercises