Library › AI Evals › Software developers

Label who should grade each of Pedalbot's 6 output fields

  • For Software developers
  • You practice Setting limits and rules
  • Time about 5 minutes
  • Level Beginner

About this exercise

Pedalbot's reply has 6 fields. For each one, pick the grader: [CODE] if a program can check it exactly against a known value or pattern, [AI JUDGE] if it is meaning or style that a written rubric can score, or [HUMAN] if a wrong answer risks legal or safety trouble so a person must review it. Write exactly 6 numbered lines in this format: "1. field_name [LABEL] - reason", with each reason 8 words or fewer. Plain text, under 110 words.

Common mistakes when you ask AI for this

  • Judging everything with AI. The AI follows the team's first plan and labels every field AI JUDGE. Try: Say: "CODE when a program can check it exactly."
  • Missing the risky field. The AI treats the waiver like any other text field. Try: Add: "HUMAN when a wrong answer risks legal or safety trouble."
  • Wrong label format. The AI writes Code or Human check without brackets. Try: Show it: "1. field_name [LABEL] - reason", labels in square brackets.
  • Long reasons. The AI explains each pick in a full sentence. Try: Say: "Each reason 8 words or fewer."
  • Tables and bold. The AI makes a bold table with extra commentary. Try: Say: "Exactly 6 numbered lines, plain text."

What a strong prompt includes

  • Defines the three graders
  • Shows the line format
  • Caps reason length
  • Says exactly 6 lines
  • Asks for plain text

Practice this now ↑

Related exercises