{"id":"eval-mkt-judge-json-prices-match-list","audience":"marketing","category":"evaluation","track":"evals","technique":"grounding","title":"Write a JSON judge that checks ad prices against the list","free":false,"task":"Your AI writes ad copy, and the costly failure is a price that doesn't match the price list. Write an LLM judge for that one failure mode, in exactly 1 code block of at most 20 lines, with the placeholders {copy} and {price_list}. It returns only JSON {\"pass\": bool, \"reason\": str}; a false pass quotes the wrong price. Copy with no price passes, and '$49' and '$49.00' count as the same price. Under 40 words outside the block.","materialLabel":"Sample ads and team notes (attached to your prompt automatically)","material":"Cobalt Bay Mobile - AI ad copy review, week 45\n\nPRICE LIST (example used in testing)\nUnlimited Plus: $49/month\nBasic 10GB: $25/month\nFamily (4 lines): $140/month\n\nSAMPLE ADS\nA: \"Unlimited Plus for $49.00 a month. No contract.\"\nB: \"Get Basic 10GB for just $20/month!\"  (reviewer: wrong price, it's $25)\nC: \"Switch your whole family for $140 a month.\"  (reviewer: fine)\nD: \"Fast 5G everywhere you go. Plans for everyone!\"\nE: \"Unlimited Plus, now $39/month.\"  (reviewer: wrong price)\n\nTeam notes\n- Brand team also complains about hype words, missing legal footnotes and banner length.\n- The judge runs in a pipeline that reads JSON only.\n- Last quarter a 1-10 \"ad quality\" score was useless.","rules":[{"type":"codeBlocks","count":1},{"type":"maxCodeLines","max":20},{"type":"maxProseWords","max":40}],"mistakes":[{"title":"One judge for everything","why":"The AI also checks hype and footnotes, so a fail doesn't tell you a price was wrong.","fix":"Say: \"Check only prices against the list.\"","checks":["r0"]},{"title":"Failing harmless copy","why":"A strict judge fails ads with no price or flags $49.00 as different from $49.","fix":"Add: \"No price passes. $49 and $49.00 are the same.\"","checks":["r1"]},{"title":"A score instead of a verdict","why":"AI judges default to scores that a pipeline can't read.","fix":"Say: \"Return only JSON with the keys pass and reason. A false pass quotes the wrong price.\"","checks":["r2"]},{"title":"An essay around the prompt","why":"The AI explains the judge and adds examples in more code blocks.","fix":"Say: \"Exactly 1 code block, at most 20 lines, one sentence after it.\"","checks":["codeBlocks","maxCodeLines","maxProseWords"]}],"promptHints":[{"label":"Limits it to one failure mode","patterns":["only","one failure","ignore","nothing else"]},{"label":"Handles the edge cases","patterns":["no price","\\$49\\.00","same price"]},{"label":"Asks for JSON pass/reason","patterns":["json","\"pass\"","pass.{0,20}reason"]},{"label":"Uses placeholders","patterns":["\\{copy\\}","\\{price_list\\}","placeholder"]},{"label":"Limits the size","patterns":["1 code block","one code block","\\d+\\s*lines"]}],"hint":"A price judge compares the copy to the list and nothing else. Say what counts as a match, what passes automatically, and show the JSON shape.","rubricLabels":["The judge checks only whether prices in the copy match the price list (it does not also grade hype words, legal footnotes, length, tone or quality)","Tells the judge that copy with no price passes and that $49 and $49.00 count as the same price","Uses the placeholders {copy} and {price_list} and tells the judge to return only JSON with exactly the keys pass and reason, where a false pass quotes the wrong price"]}