Write a JSON judge for replies in the wrong language

About this exercise

Your support bot sometimes replies in English to customers who wrote in another language. Write an LLM judge for that one failure mode, in exactly 1 code block of at most 20 lines, with the placeholders {user_message} and {reply}. It returns only JSON {"pass": bool, "reason": str}; a false pass names both languages. Product names, URLs and code snippets in English don't count as switching languages. Under 40 words outside the block.

Common mistakes when you ask AI for this

• One judge for everything. The AI also checks accuracy and length, so a fail doesn't tell you it was the language. Try: Say: "Check only whether the reply matches the user's language."
• Failing harmless English. A strict judge fails replies that keep English menu names or code. Try: Add: "Product names, URLs and code in English don't count."
• A score instead of a verdict. AI judges default to scores and paragraphs, which break a JSON file. Try: Say: "Return only JSON with the keys pass and reason. A false pass names both languages."
• An essay around the prompt. The AI explains the judge and adds examples in more code blocks. Try: Say: "Exactly 1 code block, at most 20 lines, one sentence after it."

What a strong prompt includes

• Limits it to one failure mode
• Excuses English names and code
• Asks for JSON pass/reason
• Uses placeholders
• Limits the size

Practice this now ↑

Related exercises

• Add dark mode by overriding your existing CSS variables (Coding AI · Software developers)
• Add input checks to your number guessing game (Coding AI · High school students)
• Add labels to a signup form without rewriting the page (Coding AI · High school students)
• Announce the employee assistance program in three steps (Writing AI · HR and recruiting)
• Answer a patient portal message about delayed lab results (Writing AI · Nurses and healthcare workers)