Library › Research AI › Product managers

Check a chatbot's market size answer against analyst notes

Practice: Catching mistakes and made-up facts. About 5 minutes.

About this exercise

You asked a chatbot about the market for pet-sitting software, and you want to check its 4 numbers against the analyst report your company bought. You want one numbered line per number, under 50 words total, labeled Matches, Wrong, or Not in report, with the report's figure after any Wrong label. Use only the report, plain text, nothing else.

Common mistakes when you ask AI for this

  • Letting the AI check itself. Ask a chatbot if its numbers are right and it often defends them from memory. Try: Say: "Judge only by the analyst report."
  • Calling missing numbers wrong. The income figure isn't in the report, so it's unverified rather than wrong. Try: Add: "If the report doesn't include it, label it Not in report."
  • No fixed labels. The AI writes a market overview instead of a clear verdict per number. Try: Give them: "Matches, Wrong, or Not in report."
  • Trusting the AI to count words. AI is bad at counting. Ask for a total word limit and it often lands 10-20% over. Try: Cap each line: "each line under 8 words."
  • Getting markdown back. AI defaults to bold labels and headings, which show up as stray symbols in your strategy doc. Try: Ask for "plain text, no bold, no headings."

What a strong prompt includes

  • Gives exact labels
  • Names the source
  • Asks for the report's figure
  • Sets a length limit
  • Asks for plain text

Practice this now ↑

Related exercises