Library › AI Evals › HR and recruiting

Group 8 handbook-bot failures by case type and field

  • For HR and recruiting
  • You practice Keeping the AI to your sources
  • Time about 5 minutes
  • Level Intermediate

About this exercise

Northgate Labs's HR handbook bot, PolicyPal, failed 8 test cases. Group them by case type and field. Output a markdown table with one row per group and the columns Case type, Field, Ids and Count, then 2 lines: "Largest case type:" (its name with the math, like 3 of 8 = 38%) and "Shared cause:" (the one cause behind the largest case type, under 20 words). No other text. Under 100 words. HR reviews any fix before it ships.

Common mistakes when you ask AI for this

  • Grouping by field only. The AI makes one group per field and loses the case type that explains the failures. Try: Say: "Group by case type AND field, one row per pair."
  • Following the vendor's order. The AI starts with the date format because the notes suggest it. Try: Say: "Find the largest case type by count and name its cause."
  • Naming a symptom. The AI says 'the answers are wrong' or blames the temperature. Try: Say: "Name what the bot does when the handbook is silent."
  • Extra text and bold. AI adds recommendations, bold and an intro. Try: Ask for "the table plus 2 lines, plain text, nothing else."

What a strong prompt includes

  • Groups by case type and field
  • Asks for a table with counts
  • Asks for the largest group with math
  • Asks for the shared cause
  • Asks for plain text

Practice this now ↑

Related exercises