{"id":"evals2-define-golden-before-output","audience":"teachers","category":"evaluation","track":"evals","stage":"define-success","level":"beginner","technique":"grounding","title":"Write the golden answer from the handbook before the bot answers","free":false,"task":"Your school's front-office bot has not answered this parent question yet, and you want the right answer fixed in writing first. Using only the handbook excerpt, fill in this template as exactly 5 numbered lines, in this order: Question, Golden answer (under 35 words, starting with the word Tardy), Must include (two facts separated by a semicolon), Must not say (the one wrong answer an out-of-date rule would give), Source (the subsection numbers, like Section 4.1), each line starting like \"1. Question: ...\". Plain text, under 130 words.","materialLabel":"Handbook excerpt and parent question (attached to your prompt automatically)","material":"Maple Ridge Middle School - Family Handbook (excerpt)\n\nSection 2: Attendance\n2.1 Students who arrive after the tardy bell are marked tardy. The tardy bell rings at 8:10 a.m. (changed on Sept 12; it was 8:15 a.m. in earlier handbooks).\n2.2 Students who arrive after 9:30 a.m. are marked absent for the morning.\n2.3 To excuse a tardy or an absence, a family must send a written note or email to the front office within 3 school days of the student's return.\n\nSection 4: Early pickup\n4.1 A parent or a listed contact must sign the student out at the front office.\n4.2 No pickups from 2:45 to 3:05 p.m. during dismissal.\n\nParent question received by the front-office bot (Nov 4):\n\"Our bus broke down and my son walked in at 8:12 today. Is he marked tardy, and do I need to send a note?\"\n\nAside from the office manager: last year's handbook PDF is still linked on the old school website.","rules":[{"type":"numberedItems","count":5},{"type":"plainText"},{"type":"maxWords","max":130}],"mistakes":[{"title":"Answering from memory","why":"The AI gives a typical school policy instead of this handbook.","fix":"Say: \"Use only the handbook excerpt.\"","checks":["r1"]},{"title":"Using the old bell time","why":"The AI picks up 8:15 from the earlier handbooks note.","fix":"Add: \"Use the current bell time; Must not say names the retired rule.\"","checks":["r2"]},{"title":"Skipping fields","why":"The AI writes one paragraph and no Must not say or Source.","fix":"Say: \"Fill in the 5 fields in order: Question, Golden answer, Must include, Must not say, Source.\"","checks":["r0"]},{"title":"Long golden answers","why":"The AI writes a friendly multi-paragraph reply.","fix":"Add: \"Golden answer under 35 words.\"","checks":["maxWords","r1"]},{"title":"Markdown template","why":"The AI bolds the field names and adds a heading.","fix":"Add: \"Plain text, no bold.\"","checks":["plainText","numberedItems"]}],"promptHints":[{"label":"Names the template fields","patterns":["golden answer","must include","must not say","source"]},{"label":"Grounds it in the handbook","patterns":["only","handbook","excerpt"]},{"label":"Handles the changed rule","patterns":["current|8:10|updated|retired|old"]},{"label":"Sets a length cap","patterns":["under \\d+ words"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting"]}],"hint":"A golden answer is written from the source before you ever see the bot's reply, so it can't be bent to fit. Give the AI the five fields and tell it to use only the handbook.","mission":"Write the golden answer before the bot ever answers.","musts":["Fill the 5-field template","Use current bell time","Golden answer under 35 words"],"rubricLabels":["Five numbered lines labeled in this order: 1. Question, 2. Golden answer, 3. Must include, 4. Must not say, 5. Source, with Source reading Section 2.1, 2.3 and the Question in quotation marks","Line 2 starts \"2. Golden answer: Tardy\", is under 35 words, and says a note is due within 3 school days (it never says he is absent or on time)","Line 3 (\"3. Must include:\") gives the current 8:10 bell, and line 4 (\"4. Must not say:\") names the retired 8:15 cutoff or saying he is on time"]}