Library › AI Evals › Software developers
Grade 4 AI commit messages against your team's convention
Practice: Asking for a format you can use. About 5 minutes.
About this exercise
Your team's AI writes commit messages, and you're checking 4 against the convention. This sprint, ticket ids changed from 'PL-' to 'PLAT-', so 'PL-' now fails C3. Grade the 4 messages in exactly 4 numbered lines: '1. PASS' or '1. FAIL - C2 - "exact words from the message"', naming the first criterion it fails. Plain text, under 80 words.
Common mistakes when you ask AI for this
- Missing the id change. The convention still shows PL-, so the AI passes message 3. Try: Tell it: "Ticket ids are now PLAT-. PL- fails C3."
- No evidence behind a FAIL. The AI writes 'wrong mood' without showing the word. Try: Say: "Quote the exact words from the message that fail it."
- Rewrites instead of grades. The AI rewrites each commit message to fix it. Try: Show the shape: "2. FAIL - C2 - \"quote\"", no rewrites.
- Getting markdown back. AI defaults to code formatting and bold verdicts. Try: Ask for "plain text, no bold, no code blocks."
What a strong prompt includes
- Shows the line shape
- Asks for a quote as evidence
- Gives the id change
- Says exactly how many
- Asks for plain text
Related exercises
- Adapt one ad image to 3 sizes with Midjourney (Image & Video AI · Marketing and content)
- Announce a referral bonus with the steps to claim it (Marketing AI · HR and recruiting)
- Announce a team meeting change people will actually follow (Writing AI · Managers and team leads)
- Answer a 'found an error' question in STAR order (Writing AI · Finance and accounting)
- Answer a wholesale inquiry with clear terms (Writing AI · Small business owners)