Library › AI Evals › Product managers

Turn "the digest feels generic" into 4 testable checks

Practice: Catching mistakes and made-up facts. About 5 minutes.

About this exercise

Users say your AI weekly digest 'feels generic', and you need checks a grader can apply to one digest. Write exactly 4 numbered checks, each 'N. Fail if [something you can see or count in one digest] - e.g. "quote from the interviews"'. Don't use vague words like generic, bland or boring as the test, and skip praise and requests for new features. Plain text, under 130 words.

Common mistakes when you ask AI for this

  • Keeping the vague words. The AI rewrites 'generic' as 'specific and engaging', which is just as hard to grade. Try: Say: "Each check must be something you can see or count in one digest."
  • No example from real data. The AI writes generic writing tips instead of tying checks to what users said. Try: Add: "Quote an example from the interviews on each line."
  • Treating every comment as a problem. The AI turns P5's praise and P7's feature idea into checks. Try: Add: "Skip praise and feature requests."
  • Too many checks. The AI writes one check per interview, plus some of its own. Try: Be exact: "Exactly 4 numbered checks."

What a strong prompt includes

  • Asks for observable checks
  • Asks for quoted examples
  • Says exactly how many
  • Skips praise and feature ideas
  • Asks for plain text

Practice this now ↑

Related exercises