{"id":"eval-col-rubric-ai-midterm-study-guide","audience":"college","category":"evaluation","track":"evals","technique":"constraints","title":"Write a 4-line pass/fail rubric for AI study guides","free":false,"task":"Your study group uses AI to make midterm study guides from lecture notes, and you want a rubric to check them. Your professor said only slides 1-30 are on the midterm. Write exactly 4 numbered criteria, each 'N. Pass if ...; Fail if ...', binary only, covering: only slides 1-30, every term has a definition from the notes, at most 15 terms, and the 3 formulas she said to memorize. Plain text, under 160 words.","materialLabel":"Your lecture notes index and a sample AI guide (attached to your prompt automatically)","material":"ECON 201 (Prof. Adeyemi) - lecture notes index\nSlides 1-12: supply and demand, equilibrium\nSlides 13-22: elasticity (formula: % change in quantity / % change in price)\nSlides 23-30: consumer surplus (formula: area under demand curve above price), producer surplus\nSlides 31-45: market failures, externalities (after the midterm)\n\nProf's email (Oct 30): \"Know the elasticity formula, consumer surplus, and the equilibrium condition Qd = Qs by heart.\"\n\nSAMPLE AI STUDY GUIDE (made by Leo's prompt)\n\"Midterm Study Guide! 🎓\n1. Supply and demand - the basics of markets\n2. Elasticity - how much people respond to price\n3. Externalities - costs that affect third parties\n4. Public goods - non-rival, non-excludable\n...\n24. Opportunity cost - what you give up\nRate each topic 1-5 for how well you know it!\"\n\nGroup chat\n- Leo: \"it's 24 terms, too long\"\n- Sara: \"externalities isn't even on the midterm\"\n- Kim: \"half of these have no definition\"\n- Leo: \"the emoji is cute tho\"","rules":[{"type":"numberedItems","count":4},{"type":"plainText"},{"type":"maxWords","max":160}],"mistakes":[{"title":"Getting a rating scale","why":"Asked if a guide is good, the AI suggests rating 'clarity' 1-5.","fix":"Say: \"Binary only: each line is 'Pass if ...; Fail if ...'.\"","checks":["r0","r2"]},{"title":"Not giving the professor's limits","why":"The AI doesn't know only slides 1-30 count or which 3 formulas matter.","fix":"Add: \"Only slides 1-30. Must include elasticity, consumer surplus and Qd = Qs.\"","checks":["r1"]},{"title":"Fuzzy length check","why":"A check like 'not too long' gets different answers from everyone.","fix":"Use a number: \"Fail if more than 15 terms.\"","checks":["r1"]},{"title":"Too many criteria","why":"The AI writes 8 or more criteria, including formatting and emoji.","fix":"Be exact: \"Exactly 4 numbered criteria, one line each.\"","checks":["numberedItems","maxWords","plainText"]}],"promptHints":[{"label":"Asks for binary pass/fail","patterns":["pass if","pass/fail","binary","no (score|scale)"]},{"label":"Gives the professor's limits","patterns":["1-30|1 to 30|30 slides","formula","qd","elasticity"]},{"label":"Caps the terms","patterns":["15","at most \\d+ terms"]},{"label":"Says exactly how many","patterns":["\\b(4|four)\\b.{0,25}(criteri|line)","exactly \\d+"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting","no headings"]}],"hint":"A useful rubric checks what your professor actually said matters. Give the AI the slide range, the formulas and the term cap, and ask for binary lines.","rubricLabels":["Every criterion is binary with a concrete condition checkable by reading one guide (no 1-5 or 1-10 scales, no points, no 'partially', no test that is only a vague word like 'clear' or 'useful')","Covers all four required topics: only slides 1-30 (nothing like externalities or public goods), every term has a definition, at most 15 terms, and all 3 formulas (elasticity, consumer surplus, Qd = Qs)","Each of the 4 lines has both parts on the same line in the shape 'Pass if ...; Fail if ...', and no criterion is about emoji or style"]}