{"id":"eval-mgr-checks-for-ai-status-updates","audience":"managers","category":"evaluation","track":"evals","technique":"audience","title":"Write 4 pass/fail checks for AI-drafted status updates","free":false,"task":"Your team uses AI to draft weekly status updates for your director, and you want 4 checks to grade drafts before they're sent. Your director wants risks first, at most 5 bullets, an owner on every blocked item, and no 'on track' without a date. Write exactly 4 numbered lines, each 'N. Check: ... | Pass: ... | Fail: ...', binary only. Plain text, under 180 words.","materialLabel":"A sample AI draft and the director's comments (attached to your prompt automatically)","material":"Sample AI-drafted status update (Platform team, week 44)\n\n\"Hi Renata! Here's a quick update from the Platform team.\n\nGreat progress this week! The team shipped the new login page and closed 14 tickets. Morale is high.\n\n- Login page: shipped\n- Billing migration: on track\n- Search reindex: blocked waiting on vendor\n- Mobile push: in progress\n- Hiring: 2 interviews done\n- Docs: updated\n- Offsite planning: started\n\nRisks: the vendor delay on search might push the Q4 launch.\n\nLet me know if you have questions!\"\n\nDirector's comments on past updates (Renata, Slack, last month)\n- \"I read the first two lines. If a risk is at the bottom I miss it.\"\n- \"'On track' means nothing to me without a date.\"\n- \"Who's unblocking search? Nobody's named.\"\n- \"This is too long, I just need the handful of things that matter.\"\n\nYour notes\n- 6 people on the team, AI drafts from Jira and standup notes.\n- Renata once said she likes the friendly tone, keep that.","rules":[{"type":"numberedItems","count":4},{"type":"plainText"},{"type":"maxWords","max":180}],"mistakes":[{"title":"Getting a rating scale","why":"The AI suggests rating 'clarity' and 'conciseness' from 1 to 5, which two managers will score differently.","fix":"Say: \"Binary only: each line has a Pass and a Fail.\"","checks":["r1"]},{"title":"Generic quality checks","why":"The AI doesn't know what your director cares about, so it invents checks like 'is professional'.","fix":"List them: \"risks first, at most 5 bullets, owner on blocked items, dates with on track.\"","checks":["r0"]},{"title":"Checks you can't count","why":"A check like 'is it short?' gets a different answer from every reader.","fix":"Use numbers: \"Fail if more than 5 bullets.\"","checks":["r2"]},{"title":"A guide instead of checks","why":"The AI explains how to write good updates with headings and examples.","fix":"Be exact: \"Exactly 4 lines: Check | Pass | Fail.\"","checks":["numberedItems","maxWords","plainText"]}],"promptHints":[{"label":"Names the director's wants","patterns":["risk","5 bullets|five bullets","owner","on track"]},{"label":"Asks for pass/fail","patterns":["pass","fail","binary"]},{"label":"Shows the line shape","patterns":["check:","\\|","shape"]},{"label":"Says exactly how many","patterns":["\\b(4|four)\\b.{0,25}(check|line)","exactly \\d+"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting","no headings"]}],"hint":"Checks work when they test what the reader actually wants. Give the AI your director's four wants, ask for pass/fail lines, and use numbers wherever you can.","rubricLabels":["The 4 checks cover exactly the director's 4 wants: risks come first, at most 5 bullets, every blocked item names an owner, and no 'on track' without a date","Each line has all three parts in the shape 'N. Check: ... | Pass: ... | Fail: ...' and the Pass and Fail parts are concrete opposites (no scores, no 'somewhat', no 'mostly')","The bullet check uses the number 5 and the risk check says risks appear before any progress or wins, and no check penalizes the friendly tone"]}