{"id":"data2-precision-recall-fraud-flags-numbered","audience":"managers","category":"data","track":"data","technique":"verification","level":"intermediate","title":"Explain a fraud filter's precision and recall in three numbered lines","free":false,"task":"Your fraud review manager asks how well the new filter works. Write exactly 3 numbered lines with the math shown: 1 how many of the flagged orders were truly fraud (precision), 2 how many of the real fraud orders were caught (recall), 3 what each means for the review team's workload in plain words. Plain text, under 90 words, percents to one decimal.","materialLabel":"Filter results (attached to your prompt automatically)","material":"Fraud filter pilot - October (Ridgeway Online Store), 1,000 orders reviewed\n\nResults compared with confirmed outcomes:\n- Flagged by the filter and truly fraud: 40\n- Flagged by the filter but not fraud: 60\n- Not flagged but truly fraud: 10\n- Not flagged and not fraud: 890\n\nDefinitions:\n- Precision = flagged and truly fraud / all flagged.\n- Recall = flagged and truly fraud / all truly fraud.\n\nContext:\n- The review team manually checks every flagged order; each check takes about 10 minutes.\n- Missed fraud costs the company about $120 per order on average.\n- The manager is not technical and asks, \"How many of my reviews are wasted, and how much fraud slips through?\"\n- All flagged orders = 40 + 60 = 100. All truly fraud = 40 + 10 = 50.","rules":[{"type":"numberedItems","count":3},{"type":"plainText"},{"type":"maxWords","max":90}],"mistakes":[{"title":"Mixing up the denominators","why":"The AI divides by all orders (1,000) instead of flagged or fraud totals.","fix":"Add: \"Precision = 40 / 100 flagged; recall = 40 / 50 truly fraud.\"","checks":["r0"]},{"title":"Jargon for the manager","why":"The AI explains false positives and negatives.","fix":"Add: \"Line 3 in plain words: 60 of 100 reviews wasted; 10 of 50 fraud orders slip through.\"","checks":["r1"]},{"title":"Skipping the working","why":"The AI gives percents only.","fix":"Add: \"Write each division with the numbers.\"","checks":["r2"]},{"title":"Leaving the format to chance","why":"The AI picks its own length and layout unless told the exact shape.","fix":"Add: \"Exactly 3 numbered lines. Plain text, no bold or headings.\"","checks":["numberedItems","plainText"]},{"title":"Trusting the AI to count words","why":"AI is bad at counting. Ask for a total word limit and it often lands over it.","fix":"Cap each part instead, for example \"each line under 15 words.\"","checks":["maxWords"]}],"promptHints":[{"label":"Asks for three lines","patterns":["3 numbered","exactly 3","start .{0,15}1\\."]},{"label":"Gives precision math","patterns":["40 ?/ ?\\(40 ?\\+ ?60\\)","40\\.0%"]},{"label":"Gives recall math","patterns":["40 ?/ ?\\(40 ?\\+ ?10\\)","80\\.0%"]},{"label":"Asks for plain words","patterns":["plain words","wasted","slip through"]},{"label":"Rules out notes","patterns":["no notes","nothing else"]}],"hint":"Precision and recall are two divisions. Put the numbers in the line shape, then ask for plain-language meaning.","mission":"How good is this fraud filter?","musts":["Three numbered lines","Both divisions shown","Plain-words impact"],"rubricLabels":["Line 1 shows precision as 40 / (40 + 60) = 40.0%","Line 2 shows recall as 40 / (40 + 10) = 80.0%","Line 3 explains in plain words that 60 of every 100 reviews are wasted on orders that aren't fraud and 1 in 5 fraud orders (10 of 50) slips through, and the reply is exactly 3 numbered lines"]}