Library › AI Evals › Product managers
Turn "the digest feels generic" into 4 testable checks
Practice: Catching mistakes and made-up facts. About 5 minutes.
About this exercise
Users say your AI weekly digest 'feels generic', and you need checks a grader can apply to one digest. Write exactly 4 numbered checks, each 'N. Fail if [something you can see or count in one digest] - e.g. "quote from the interviews"'. Don't use vague words like generic, bland or boring as the test, and skip praise and requests for new features. Plain text, under 130 words.
Common mistakes when you ask AI for this
- Keeping the vague words. The AI rewrites 'generic' as 'specific and engaging', which is just as hard to grade. Try: Say: "Each check must be something you can see or count in one digest."
- No example from real data. The AI writes generic writing tips instead of tying checks to what users said. Try: Add: "Quote an example from the interviews on each line."
- Treating every comment as a problem. The AI turns P5's praise and P7's feature idea into checks. Try: Add: "Skip praise and feature requests."
- Too many checks. The AI writes one check per interview, plus some of its own. Try: Be exact: "Exactly 4 numbered checks."
What a strong prompt includes
- Asks for observable checks
- Asks for quoted examples
- Says exactly how many
- Skips praise and feature ideas
- Asks for plain text
Related exercises
- Alert sales to a competitor's price change, facts only (Research & AI Search · Product managers)
- Approve or return expense claims against the policy (Productivity AI · Finance and accounting)
- Approve part of an overtime request with the math (Writing AI · Managers and team leads)
- Catch a chatbot's mistake on your invoice total (Data AI · Freelancers and creators)
- Catch a chatbot's mistakes about a supplier's return policy (Research & AI Search · Small business owners)