{"id":"res-fin-chatbot-budget-variance-check","audience":"finance","category":"fact-checking","technique":"verification","title":"Check a chatbot's budget variance explanation against the report","free":false,"task":"You pasted your department's budget-vs-actual report into a chatbot and it explained the variances. You want one numbered line per claim, under 50 words total, labeled Matches, Wrong, or Not in report, with the report's figure after any Wrong label. Use only the report, plain text, nothing else.","materialLabel":"The budget-vs-actual report and the chatbot's claims (attached to your prompt automatically)","material":"BUDGET VS ACTUAL - Marketing Dept, Q3 2026 (in dollars)\nLine item | Budget | Actual | Variance\nPaid ads | 120,000 | 138,000 | +18,000 over\nEvents | 45,000 | 31,000 | -14,000 under\nContractors | 60,000 | 66,000 | +6,000 over\nSoftware | 24,000 | 24,000 | 0\nTotal | 249,000 | 259,000 | +10,000 over\n\nNotes from the department: Events came in under budget because the Denver trade show was canceled by the organizer.\n\n-----\nCHATBOT CLAIMS\n1. Paid ads were $18,000 over budget.\n2. Events were $14,000 over budget.\n3. Total spending was $10,000 over budget.\n4. Contractor costs ran over because of a new agency retainer.","rules":[{"type":"maxWords","max":50},{"type":"plainText"}],"mistakes":[{"title":"Trusting the explanation","why":"A plausible reason, like an agency retainer, sounds right even though nothing in the report says it.","fix":"Add: \"If the report doesn't say it, label it Not in report.\"","checks":["r2"]},{"title":"Missing the direction","why":"The dollar amount is right, so a quick check passes 'over' when it was 'under'.","fix":"Say: \"Check amount and direction (over or under).\"","checks":["r1"]},{"title":"Getting new analysis","why":"The AI explains the variances itself instead of checking the chatbot's claims.","fix":"Say: \"Label each claim only.\"","checks":["r0","r2"]},{"title":"Trusting the AI to count words","why":"AI is bad at counting. Ask for a total word limit and it often lands 10-20% over.","fix":"Cap each line: \"each line under 10 words.\"","checks":["maxWords"]},{"title":"Getting markdown back","why":"AI defaults to bold labels and headings, which show up as stray symbols in your review notes.","fix":"Ask for \"plain text, no bold, no headings.\"","checks":["plainText"]}],"promptHints":[{"label":"Gives exact labels","patterns":["matches","not in (the )?report","label"]},{"label":"Names the source","patterns":["only.{0,30}report","report.{0,20}(only|source|truth)"]},{"label":"Checks direction","patterns":["direction","over or under","under budget"]},{"label":"Sets a length limit","patterns":["\\d+\\s*words","one (short )?sentence","word (limit|count)"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting","no headings"]}],"hint":"Chatbots invent tidy explanations and flip signs. Make the report the judge, check direction as well as amount, and give a label for unsupported reasons.","rubricLabels":["Labels claims 1 and 3 Matches","Labels claim 2 Wrong (events were $14,000 under budget, not over)","Labels claim 4 (new agency retainer) Not in report, and outputs only the numbered labeled lines with no intro or analysis"]}