{"id":"eval-tch-sampling-plan-ai-feedback-comments","audience":"teachers","category":"evaluation","track":"evals","technique":"constraints","title":"Plan which AI feedback comments you read each week","free":false,"task":"You have 2 hours a week to read the AI writing coach's comments to your students, at 2 minutes each. Write exactly 4 numbered lines: Capacity (the math), Always review (the high-risk groups from the data, with their total), Random sample (how many of the rest, drawn across all 5 class periods), and Sample rate (the math as a percent). Plain text, under 100 words.","materialLabel":"This week's AI comment numbers (attached to your prompt automatically)","material":"Mr. Ortiz - English 9, AI Writing Coach comments, week of Nov 3\n\nTotal AI comments to students this week: 420\n\nHigh-risk groups (department guidance: read every one)\n- Comments a student marked \"confusing\": 8\n- Comments on final drafts (these affect revisions before grading): 12\n\nEverything else: 400\n\nBy class period (all 420)\nP1 90 | P2 85 | P3 80 | P4 95 | P6 70\n\nNotes\n- Last month I only read period 1's comments because I teach it first thing.\n- The AI gave a period 6 student feedback on the wrong essay in October.\n- Each comment takes about 2 minutes to read with the student's draft.\n- The department wants a short note on what I found by Friday.","rules":[{"type":"numberedItems","count":4},{"type":"plainText"},{"type":"maxWords","max":100}],"mistakes":[{"title":"Reading only one class","why":"The AI accepts last month's habit of reading only period 1.","fix":"Say: \"Random sample drawn across all 5 periods.\"","checks":["r2"]},{"title":"Forgetting the must-read comments","why":"The AI samples at random and misses confusing and final-draft comments.","fix":"Add: \"Always review the high-risk groups first, then sample the rest.\"","checks":["r1"]},{"title":"Hiding the math","why":"The AI says 'about 60' or '10%' without showing how.","fix":"Say: \"Show the math on each line, like 2 x 60 / 2 = 60.\"","checks":["r0","r2"]},{"title":"A long plan","why":"The AI writes a review routine with tips.","fix":"Be exact: \"Exactly 4 numbered lines: Capacity, Always review, Random sample, Sample rate.\"","checks":["numberedItems","maxWords","plainText"]}],"promptHints":[{"label":"Asks for the math","patterns":["math","show","x 60","calculat"]},{"label":"Reads high-risk comments first","patterns":["high.?risk","always review","confusing","final draft"]},{"label":"Samples every period","patterns":["random","all (5|five) (class )?periods","every period","across"]},{"label":"Says exactly how many","patterns":["\\b(4|four)\\b.{0,25}(line)","exactly \\d+"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting","no headings"]}],"hint":"Checking AI feedback has to fit your week and still see the risky comments. Start from your time, cover the must-read groups, and sample the rest at random across every class.","rubricLabels":["Capacity shows 2 hours x 60 = 120 minutes / 2 = 60 comments","Always review names comments marked confusing (8) and final-draft comments (12) for a total of 20, and Random sample gives 60 - 20 = 40 from the other 400","The random sample is drawn across all 5 class periods, not only period 1, and Sample rate shows 40 / 400 = 10%"]}