Library › AI Evals › Marketing and content

Write a JSON judge for banned hype words in ad copy

Practice: Setting limits and rules. About 5 minutes.

About this exercise

Your team's AI writes ad copy, and the brand guide bans certain hype words. Since the v4 update approved yesterday, "best" alone is allowed again because Best seller is a real badge. You want an LLM judge for that one failure mode only, not tone, length or accuracy. Write the judge prompt in exactly 1 code block of at most 25 lines, with the placeholder {copy}. It must list the banned words, return only JSON {"pass": bool, "reason": str}, and when pass is false the reason must quote the banned word found. Keep text outside the block under 30 words.

Common mistakes when you ask AI for this

  • One judge for everything. The AI turns the whole brand guide into one big checklist, so you can't tell which rule failed. Try: Say: "Check only banned words. Ignore tone, length and claims."
  • Using the old word list. The AI may still ban "best", which flags every Best seller badge as a failure. Try: Add: "'best' and 'best seller' are allowed in v4."
  • A score instead of a verdict. AI judges default to scores and paragraphs, which break a JSON parser. Try: Say: "Return only JSON with the keys pass and reason, nothing else."
  • An essay around the prompt. The AI explains how the judge works and adds test examples in extra code blocks. Try: Say: "Exactly 1 code block, at most 25 lines, one short sentence outside it."

What a strong prompt includes

  • Limits it to one failure mode
  • Asks for JSON pass/reason
  • Handles the allowed word
  • Uses a placeholder
  • Limits the size

Practice this now ↑

Related exercises