{"id":"eval-tch-golden-answer-fraction-tutor","audience":"teachers","category":"evaluation","track":"evals","technique":"context","title":"Write the golden answer for a fractions tutor test","free":false,"task":"You're building test cases for the 5th-grade math tutor bot, and this question needs a golden answer. Your curriculum says 'common denominator' and never 'LCD', the Step line must name each multiplier (like '1/2 x 3/3'), and the bot always ends by asking the student to try a similar problem. Write exactly 3 numbered lines: '1. Step: ...' with the work shown, '2. Answer: ...', and '3. Try: ...' with one new problem and no answer. Plain text, under 60 words.","materialLabel":"The test question and the unit notes (attached to your prompt automatically)","material":"TEST CASE #22 for MathPal (Grade 5, Unit 6: adding fractions)\nStudent question: \"What is 1/4 + 2/3? I keep getting 3/7.\"\n\nUNIT 6 NOTES (Ms. Ferraro)\n- To add fractions with different denominators, rewrite both with a common denominator, then add the numerators.\n- Example from class: 1/2 + 1/3 = 3/6 + 2/6 = 5/6.\n- Common mistake: adding the tops and the bottoms (1/4 + 2/3 = 3/7). This is wrong.\n- Simplify the answer if you can.\n- Mixed numbers come in Unit 7. Don't use them yet.\n\nTest-writing notes\n- Last run, MathPal's answers were 120+ words and used words like \"least common denominator (LCD)\".\n- Golden answers are the shortest complete correct answer we compare the bot's answer to.\n- Parents see these answers in the weekly report.","rules":[{"type":"numberedItems","count":3},{"type":"plainText"},{"type":"maxWords","max":60}],"mistakes":[{"title":"Using the wrong words","why":"The AI says 'LCD' or 'least common denominator' because it doesn't know your curriculum's term.","fix":"Say: \"Use 'common denominator', never 'LCD'.\"","checks":["r1"]},{"title":"Skipping the multipliers","why":"The AI jumps straight to 3/12, but your curriculum shows what each fraction is multiplied by.","fix":"Add: \"Name each multiplier, like 1/4 x 3/3 = 3/12.\"","checks":["r1"]},{"title":"No practice problem","why":"The AI ends with the answer and doesn't know your bot always asks the student to try one.","fix":"Add: \"End with Try: one new problem, no answer.\"","checks":["r2"]},{"title":"A full lesson","why":"The AI explains fractions at length, which makes a poor answer to grade against.","fix":"Show the shape: \"1. Step: 2. Answer: 3. Try:\"","checks":["numberedItems","maxWords","r0"]},{"title":"Getting markdown back","why":"AI formats fractions with bold or LaTeX that won't show up in the test file.","fix":"Ask for \"plain text, no bold, no LaTeX.\"","checks":["plainText"]}],"promptHints":[{"label":"Gives the curriculum term","patterns":["common denominator","LCD","never"]},{"label":"Shows the three-line shape","patterns":["step:","answer:","try:"]},{"label":"Asks for a practice problem","patterns":["try","similar problem","new problem","practice"]},{"label":"Keeps it short","patterns":["\\d+\\s*words","short"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting","no headings"]}],"hint":"A golden answer is the shortest correct reply you'll grade the bot against. Tell the AI your curriculum's words, show the exact shape, and include the practice problem your bot always gives.","rubricLabels":["Shows the work with a common denominator of 12 (3/12 + 8/12) and gives the answer 11/12","The Step line includes \"1/4 x 3/3\" and \"2/3 x 4/4\" (or the same multipliers written another way), and the answer never uses 'LCD' or 'least common denominator'","Line 3 starts with Try: and gives one new fraction addition problem with different denominators, without giving its answer"]}