Library › AI Evals › Product managers
Write the on-call alert when the handoff score drops hard
- For Product managers
- You practice Saying who it's for
- Time about 5 minutes
- Level Intermediate
About this exercise
Pedalbot's nightly handoff-accuracy score came in at 89 and the alert rule in the material applies. Write a 4-line plain-text Slack message to the on-call person. Line 1 is exactly 'ALERT: Pedalbot handoff score 89 tonight'. Line 2 is 'Math: avg A, SD S, alert line L, tonight N SD below' with real numbers. Line 3 is 'Check first: ' plus the deploy from the deploy log, as a thing to check and not a claimed cause. Line 4 is 'Owner: ' plus the on-call name. Under 55 words.
Common mistakes when you ask AI for this
- Excusing it with the bar. The AI says 89 is still above 85, so no worry. Try: Say: "The alert rule is standard deviations, not the merge bar."
- Wrong statistics. The AI uses the range or divides by 6 and gets the wrong threshold. Try: Add: "Average and SD of the previous 7 nights, divide by 7."
- Blaming v7 as fact. The AI states the deploy caused the drop. Try: Add: "Say 'check', do not claim the cause."
- Long Slack post. The AI writes greetings, bullets and bold. Try: Add: "Plain text, exactly 4 lines, under 55 words."
What a strong prompt includes
- Fixes the four line openers
- Includes the numbers
- Names who gets it
- Treats the cause as a lead
- Asks for plain text
Related exercises
- Announce a delay to three audiences with different limits (Leadership AI · Communications and PR)
- Announce one feature separately to admins and to end users (Customer Support & Success AI · Customer success managers)
- Answer "we'll build it ourselves" with their own numbers (Sales AI · Sales and customer success)
- Answer "what does a CSM do for us?" in plain spoken words (Customer Support & Success AI · Customer success managers)
- Answer a director's accrual question as a short Q&A (Finance AI · Accountants)