{"id":"data2-average-of-averages-check-ai","audience":"analysts","category":"data","track":"data","technique":"verification","level":"intermediate","title":"Check an AI's average order value across two stores","free":false,"task":"An AI said the average order value across two stores is $65 and you suspect it averaged the two averages. Write exactly 3 labeled lines in plain text, under 70 words: Verdict (Right or Wrong plus the reason), Correct math (the weighted calculation written out) and Why (one sentence for a non-analyst). Use only the numbers given.","materialLabel":"The AI's answer and the store data (attached to your prompt automatically)","material":"AI's answer: \"Store A's average order value is $50 and Store B's is $80, so the overall average order value is ($50 + $80) / 2 = $65.\"\n\nStore data for October (Oakridge Hardware):\n- Store A: 900 orders, average order value $50\n- Store B: 100 orders, average order value $80\n\nNotes:\n- Average order value = total sales / number of orders.\n- Store A's total sales: 900 x $50 = $45,000.\n- Store B's total sales: 100 x $80 = $8,000.\n- The owner will quote the overall figure in a loan application, so it needs to be right.\n- Express the final figure in dollars with cents.","rules":[{"type":"plainText"},{"type":"maxWords","max":70}],"mistakes":[{"title":"Accepting the $65","why":"The AI says the simple average is fine.","fix":"Add: \"Check whether the stores should be weighted by order counts.\"","checks":["r0"]},{"title":"Math without the sums","why":"The AI gives $53 with no working.","fix":"Add: \"Correct math: ($45,000 + $8,000) / (900 + 100) = $53.00.\"","checks":["r1"]},{"title":"Jargon in Why","why":"The AI explains 'weighted means' at length.","fix":"Add: \"Why: one plain sentence, under 25 words.\"","checks":["r2"]},{"title":"Leaving the format to chance","why":"The AI picks its own length and layout unless told the exact shape.","fix":"Add: \"Plain text, no bold or headings.\"","checks":["plainText"]},{"title":"Trusting the AI to count words","why":"AI is bad at counting. Ask for a total word limit and it often lands over it.","fix":"Cap each part instead, for example \"each line under 15 words.\"","checks":["maxWords"]}],"promptHints":[{"label":"Gives the three labels","patterns":["verdict:","correct math:","why:"]},{"label":"Asks for weighted math","patterns":["weighted","45,?000","900 ?\\+ ?100"]},{"label":"Asks for the verdict","patterns":["right or wrong"]},{"label":"Asks for a plain why","patterns":["plain","non-analyst"]},{"label":"Limits the data","patterns":["only the numbers","use only"]}],"hint":"Averages of averages are a classic AI slip. Ask for the weighted calculation written out.","mission":"Is this average order value right?","musts":["Three labeled lines","Weighted math written out","Plain-language why"],"rubricLabels":["Verdict says Wrong, because $65 simply averages the two averages and ignores that Store A has many more orders","Correct math writes ($45,000 + $8,000) / (900 + 100) = $53,000 / 1,000 = $53.00","Why is one plain sentence saying the stores should be weighted by their number of orders (Store A has 9 times as many), and the reply is exactly three lines labeled Verdict:, Correct math: and Why:"]}