{"id":"eval-col-lit-review-claims-vs-abstracts","audience":"college","category":"evaluation","track":"evals","technique":"verification","title":"Check an AI lit-review paragraph against 3 abstracts","free":false,"task":"An AI wrote a literature-review paragraph from 3 abstracts for your psych seminar, and you want to check each claim before using it. Write exactly 5 numbered lines, one per claim, each 'N. Supported (A2)', 'N. Contradicted (A1) - abstract says \"...\"' or 'N. Not in abstracts', naming the abstract you checked. Judge only by the abstracts. Plain text, under 110 words.","materialLabel":"The 3 abstracts and the AI's paragraph (attached to your prompt automatically)","material":"ABSTRACTS (PSYC 410 seminar, week 9: sleep and memory)\n\nA1 - Varga & Holt (2021): 64 undergraduates learned word pairs, then either slept or stayed awake for 8 hours. The sleep group recalled 18% more pairs. The effect was smaller for pairs that were already well learned.\n\nA2 - Nakamura (2023): In 120 adults, a 20-minute afternoon nap did not improve recall of a story compared with quiet rest. Participants who napped reported feeling less tired.\n\nA3 - Oduya, Fenn & Ruiz (2022): Survey of 1,200 students. Students who slept under 6 hours on weeknights reported lower GPAs. The authors note the survey cannot show cause and effect.\n\nAI-WRITTEN PARAGRAPH (claims numbered by you)\n(1) Sleep after learning improves memory: in one study, students who slept recalled 18% more word pairs than those who stayed awake (Varga & Holt, 2021). (2) This benefit was strongest for material that was already well learned. (3) Even a short 20-minute nap boosts story recall (Nakamura, 2023). (4) Napping also reduced how tired people felt. (5) Sleeping less than 6 hours causes lower grades, according to a large survey (Oduya et al., 2022).","rules":[{"type":"numberedItems","count":5},{"type":"plainText"},{"type":"maxWords","max":110}],"mistakes":[{"title":"Asking 'is it accurate?'","why":"The AI says the paragraph is 'mostly accurate' and misses the flipped findings.","fix":"Say: \"Label each claim Supported, Contradicted or Not in abstracts.\"","checks":["r0"]},{"title":"Missing the causal claim","why":"'Causes lower grades' sounds close to the survey, so the AI lets it pass.","fix":"Add: \"Treat a cause claim from a survey as Contradicted if the abstract says it can't show cause.\"","checks":["r0"]},{"title":"Not naming the source","why":"The AI says a claim is supported without saying which abstract, so you can't check it.","fix":"Ask: \"Name the abstract you checked, like (A2).\"","checks":["r1"]},{"title":"No proof for corrections","why":"The AI says a claim is wrong but doesn't show the abstract's words.","fix":"Add: \"For Contradicted, quote the abstract.\"","checks":["r2"]},{"title":"A rewritten paragraph","why":"The AI 'helps' by rewriting the whole paragraph, which you didn't ask for.","fix":"Be exact: \"Exactly 5 numbered lines, nothing else.\"","checks":["numberedItems","maxWords","plainText"]}],"promptHints":[{"label":"Gives exact labels","patterns":["supported","contradicted","not in","label"]},{"label":"Asks which abstract","patterns":["which abstract","\\(A\\d\\)","name the abstract","A1"]},{"label":"Asks for a quote","patterns":["quote","what the abstract says"]},{"label":"Says exactly how many","patterns":["\\b(5|five)\\b.{0,25}(line|claim)","exactly \\d+"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting","no headings"]}],"hint":"AI literature reviews often flip a finding or turn a correlation into a cause. Give the AI three labels, make it name the abstract behind each one, and ask for the quote when it disagrees.","rubricLabels":["Labels claim 1 Supported, claim 2 Contradicted, claim 3 Contradicted, claim 4 Supported and claim 5 Contradicted (A3 says the survey cannot show cause and effect)","Each Supported or Contradicted line names the abstract it was checked against (claims 1 and 2: A1, claims 3 and 4: A2, claim 5: A3)","Each Contradicted line quotes what the abstract says in quotation marks, and there is no rewritten paragraph or writing advice"]}