Library › AI Evals › Product managers
Check an AI research summary against your interview notes
Practice: Catching mistakes and made-up facts. About 5 minutes.
About this exercise
An AI summarized 4 customer interviews into 5 findings for your roadmap review, and you want to check each against your notes first. Write exactly 5 numbered lines, one per finding, each labeled Supported, Overstated or Not in notes. For each Overstated line, add what the notes actually say in quotation marks. Judge only by the notes. Plain text, under 110 words.
Common mistakes when you ask AI for this
- Asking 'is it right?'. The AI says the summary is 'a fair reflection' and misses the made-up finding. Try: Say: "Label each finding Supported, Overstated or Not in notes."
- Letting 'all' slide. One customer cancelled over price, and the AI accepts 'all four say price is too high'. Try: Add: "Overstated means the finding claims more people or more strength than the notes show."
- No proof for corrections. The AI says a finding is too strong without showing what the notes say. Try: Add: "For Overstated, quote what the notes actually say."
- A rewritten summary. The AI 'helps' by rewriting the findings and adding recommendations. Try: Be exact: "Exactly 5 numbered lines, nothing else."
What a strong prompt includes
- Gives exact labels
- Makes the notes the judge
- Asks for a quote as evidence
- Says exactly how many
- Asks for plain text
Related exercises
- Alert sales to a competitor's price change, facts only (Research & AI Search · Product managers)
- Approve or return expense claims against the policy (Productivity AI · Finance and accounting)
- Approve part of an overtime request with the math (Writing AI · Managers and team leads)
- Brief a research agent to prep you for a sales call (Agents & Automation AI · Sales and customer success)
- Brief a research assistant to shortlist sources you can trust (Agents & Automation AI · College students)