{"id":"res-sb-supplier-return-chatbot-errors","audience":"small-business","category":"fact-checking","technique":"verification","title":"Catch a chatbot's mistakes about a supplier's return policy","free":false,"task":"You asked a chatbot how to return a damaged shipment of mugs to your supplier, and you want to check its answer against the supplier's actual return policy. You want only the errors, as numbered lines in the form \"Chatbot says X; policy says Y\", under 60 words total, plain text. Use only the policy, skip lines that match it, and don't rewrite the answer.","materialLabel":"The supplier's return policy and the chatbot's answer (attached to your prompt automatically)","material":"CLAYWORKS WHOLESALE - RETURN POLICY (updated Aug 2026)\n1. Report damaged items within 7 days of delivery.\n2. Include photos of the damaged items and the shipping box.\n3. Damaged items are replaced or credited to your account; cash refunds are not offered.\n4. Clayworks pays return shipping for damaged items. Use the prepaid label we email you.\n5. Non-damaged returns have a 15% restocking fee.\n\n-----\nCHATBOT ANSWER\nTo return damaged mugs to Clayworks, report the damage within 30 days of delivery. Send photos of the damaged mugs and the shipping box. Clayworks will give you a full cash refund. You'll need to pay return shipping yourself. There's no restocking fee for damaged items.","rules":[{"type":"maxWords","max":60},{"type":"plainText"}],"mistakes":[{"title":"Asking if the answer is right","why":"The AI rewrites a corrected answer, and you can't see what the chatbot got wrong.","fix":"Say: \"List only the errors. Don't rewrite.\"","checks":["r2"]},{"title":"Not naming the source","why":"The AI may judge by typical return policies, not Clayworks' actual one.","fix":"Add: \"The policy is the only source of truth.\"","checks":["r0"]},{"title":"Flagging correct lines","why":"To look thorough, the AI lists matching steps too.","fix":"Say: \"Skip lines that match the policy.\"","checks":["r1"]},{"title":"Trusting the AI to count words","why":"AI is bad at counting. Ask for a total word limit and it often lands 10-20% over.","fix":"Cap each line: \"each line under 14 words.\"","checks":["maxWords"]},{"title":"Getting markdown back","why":"AI defaults to bold labels and headings, which show up as stray symbols in your notes.","fix":"Ask for \"plain text, no bold, no headings.\"","checks":["plainText"]}],"promptHints":[{"label":"Names the source of truth","patterns":["policy.{0,30}(only|source|truth)","only.{0,30}policy","against the policy"]},{"label":"Asks for errors only","patterns":["only.{0,20}(error|mistake|wrong)","skip.{0,30}match","contradict"]},{"label":"Gives the line format","patterns":["chatbot says","policy says","\\bX\\b.{0,20}\\bY\\b"]},{"label":"Sets a length limit","patterns":["\\d+\\s*words","one (short )?sentence","word (limit|count)"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting","no headings"]}],"hint":"Chatbots answer from what's typical, not from your supplier's rules. Make the policy the judge, ask only for mismatches, and give the line shape.","rubricLabels":["Flags all three errors: reporting window (chatbot: 30 days; policy: 7 days), refund (chatbot: cash refund; policy: replacement or account credit), and return shipping (chatbot: you pay; policy: Clayworks pays with a prepaid label)","Does not flag lines that match or aren't contradicted by the policy (photos of mugs and box; no restocking fee for damaged items)","Outputs only numbered lines in the form 'Chatbot says X; policy says Y': no intro, no rewritten steps, no closing advice"]}