{"id":"eval-mkt-judge-json-banned-hype-words","audience":"marketing","category":"evaluation","track":"evals","technique":"constraints","title":"Write a JSON judge for banned hype words in ad copy","free":false,"task":"Your team's AI writes ad copy, and the brand guide bans certain hype words. Since the v4 update approved yesterday, \"best\" alone is allowed again because Best seller is a real badge. You want an LLM judge for that one failure mode only, not tone, length or accuracy. Write the judge prompt in exactly 1 code block of at most 25 lines, with the placeholder {copy}. It must list the banned words, return only JSON {\"pass\": bool, \"reason\": str}, and when pass is false the reason must quote the banned word found. Keep text outside the block under 30 words.","materialLabel":"Brand guide excerpt and sample copy (attached to your prompt automatically)","material":"Maple & Moss Home - brand voice guide (v4, Oct)\n\nWho we sound like: a friend who knows a lot about plants. Warm, specific, never pushy.\n\nBANNED WORDS (any form, any capitalization)\nultimate, revolutionary, game-changing, best, best-ever, miracle, guaranteed\n\nOther guidelines\n- Headlines under 8 words\n- Mention a care tip when possible\n- No claims about air purification unless legal approved it\n\nSAMPLE COPY FROM THE AI (last week)\nA: \"Meet the Fern Trio: three easy ferns for low-light corners. Best seller since spring.\"\nB: \"The ultimate snake plant for beginners. Practically unkillable.\"\nC: \"Our miracle potting mix keeps roots happy for months.\"\nD: \"Revolutionary self-watering pots, now in sage and clay.\"\nE: \"Pothos in a hanging basket. Water when the top inch is dry.\"\n\nNotes from the team\n- The CI step reads the judge output with a JSON parser, so any extra text fails the run.\n- Dee asked if the judge could also rate warmth 1-10. We said separate judge, later.\n- Copy runs through about 300 judge calls a week.","rules":[{"type":"codeBlocks","count":1},{"type":"maxCodeLines","max":25},{"type":"maxProseWords","max":30}],"mistakes":[{"title":"One judge for everything","why":"The AI turns the whole brand guide into one big checklist, so you can't tell which rule failed.","fix":"Say: \"Check only banned words. Ignore tone, length and claims.\"","checks":["r0"]},{"title":"Using the old word list","why":"The AI may still ban \"best\", which flags every Best seller badge as a failure.","fix":"Add: \"'best' and 'best seller' are allowed in v4.\"","checks":["r1"]},{"title":"A score instead of a verdict","why":"AI judges default to scores and paragraphs, which break a JSON parser.","fix":"Say: \"Return only JSON with the keys pass and reason, nothing else.\"","checks":["r2"]},{"title":"An essay around the prompt","why":"The AI explains how the judge works and adds test examples in extra code blocks.","fix":"Say: \"Exactly 1 code block, at most 25 lines, one short sentence outside it.\"","checks":["codeBlocks","maxCodeLines","maxProseWords"]}],"promptHints":[{"label":"Limits it to one failure mode","patterns":["only","one failure","nothing else","ignore"]},{"label":"Asks for JSON pass/reason","patterns":["json","\"pass\"","pass.{0,20}reason"]},{"label":"Handles the allowed word","patterns":["best seller","\"best\"","v4","allowed","\\bbest\\b"]},{"label":"Uses a placeholder","patterns":["\\{copy\\}","placeholder"]},{"label":"Limits the size","patterns":["1 code block","one code block","\\d+\\s*lines"]}],"hint":"A judge you can automate checks one thing and returns something a script can read. Name the single rule, give the exact word list, and show the JSON shape.","rubricLabels":["The judge checks only for the banned words (it does not also grade tone, warmth, headline length, care tips, air-purification claims or accuracy)","Lists the six banned words (ultimate, revolutionary, game-changing, best-ever, miracle, guaranteed) and does not ban \"best\" on its own or \"best seller\"","Uses the {copy} placeholder and tells the judge to return only JSON with exactly the keys pass and reason, where a false pass has a reason quoting the banned word found"]}