Library › AI Evals › Product managers
Make a per-language launch call from eval results
Practice: Catching mistakes and made-up facts. About 5 minutes.
About this exercise
Your AI support assistant is launching in new languages, and each needs at least 90% on the eval to ship. Japanese was pulled from this launch yesterday, so leave it out. Write exactly 3 numbered lines, one per remaining language, each 'Language: X of Y = Z% vs 90% - Ship or Hold', then a last line 'Summary: ship [languages], hold [languages]'. Plain text, under 80 words.
Common mistakes when you ask AI for this
- Including the pulled language. The AI doesn't know Japanese was pulled yesterday, so it grades all four. Try: Tell it: "Japanese is out of this launch. Leave it out."
- Hiding the math. The AI says German is 'a bit low' without the numbers. Try: Show the shape: "Language: X of Y = Z% vs 90% - Ship or Hold."
- Deciding on the average. The AI quotes the 88% overall and holds everything, or ships everything. Try: Add: "Decide each language on its own score."
- A launch memo. The AI writes a long memo with risks and the marketing teaser. Try: Be exact: "3 numbered lines, then one Summary line."
What a strong prompt includes
- Leaves out Japanese
- Gives the bar
- Asks for the math
- Asks for a per-language call
- Asks for plain text
Related exercises
- Alert sales to a competitor's price change, facts only (Research & AI Search · Product managers)
- Approve or return expense claims against the policy (Productivity AI · Finance and accounting)
- Approve part of an overtime request with the math (Writing AI · Managers and team leads)
- Catch a chatbot's mistake on your invoice total (Data AI · Freelancers and creators)
- Catch a chatbot's mistakes about a supplier's return policy (Research & AI Search · Small business owners)