{"id":"eval-tch-reading-summary-claim-check","audience":"teachers","category":"evaluation","track":"evals","technique":"verification","title":"Check an AI reading summary against the source text","free":false,"task":"An AI wrote a 6-point summary of a local history reading for your 7th graders, and you want to check it before printing. Write exactly 6 numbered lines, one per summary point, each labeled Supported, Contradicted or Not in source. For each Contradicted line, add what the source says in quotation marks. Judge only by the reading. Plain text, under 120 words.","materialLabel":"The reading and the AI's summary (attached to your prompt automatically)","material":"SOURCE READING: \"The Harlow Creek Mill\" (grade 7 local history packet, p. 3)\nThe Harlow Creek Mill was built in 1838 by brothers Amos and Eli Pruitt. It used the creek's current to turn a large wooden wheel, which powered stones that ground corn and wheat into flour. Farmers from as far as twelve miles away brought grain to the mill, often waiting overnight in busy autumn weeks. In 1871 a spring flood damaged the wheel, and the town paid to rebuild it within a year. The mill stopped grinding grain in 1924, when a larger steam mill opened in the county seat. Today the building is a small museum run by volunteers, open on Saturdays from May to October.\n\nAI SUMMARY (generated for students)\n1. The Harlow Creek Mill was built in 1838 by the Pruitt brothers.\n2. It used a wooden water wheel to power stones that ground grain into flour.\n3. Farmers came from up to twenty miles away to use the mill.\n4. A flood in 1871 damaged the wheel, and the town rebuilt it.\n5. The mill was the largest in the state during the 1800s.\n6. It closed in 1924 and is now a museum open every weekend all year.","rules":[{"type":"numberedItems","count":6},{"type":"plainText"},{"type":"maxWords","max":120}],"mistakes":[{"title":"Asking 'is it accurate?'","why":"The AI says it's 'mostly accurate' and moves on, missing the small number changes.","fix":"Say: \"Label each point Supported, Contradicted or Not in source.\"","checks":["r0","r2"]},{"title":"Mixing up wrong and missing","why":"A claim the reading never makes isn't the same as one it contradicts, and the AI blurs the two.","fix":"Define it: \"Not in source means the reading doesn't mention it at all.\"","checks":["r0"]},{"title":"No proof for corrections","why":"The AI says a point is wrong without showing what the reading says.","fix":"Add: \"For Contradicted, quote the source in quotation marks.\"","checks":["r1"]},{"title":"Getting a new summary","why":"The AI often 'helps' by rewriting the whole summary, which you didn't ask for.","fix":"Say: \"Exactly 6 numbered lines, one per point, nothing else.\"","checks":["numberedItems","r2"]},{"title":"Getting markdown back","why":"AI defaults to bold labels, headings and tables, which break when you paste into a doc or tracker.","fix":"Ask for \"plain text, no bold, no headings, no tables.\"","checks":["plainText"]},{"title":"Trusting the AI to count words","why":"AI is bad at counting. Ask for a total and it often lands 20% over.","fix":"Cap each line instead: \"each line under 20 words.\"","checks":["maxWords"]}],"promptHints":[{"label":"Gives exact labels","patterns":["supported","contradicted","not in source","label"]},{"label":"Makes the reading the judge","patterns":["only.{0,30}(reading|source|text|passage)","against the (reading|source)"]},{"label":"Asks for a quote","patterns":["quote","what the (source|reading) says"]},{"label":"Says how many lines","patterns":["\\b(6|six)\\b.{0,20}(line|point)","exactly \\d+","one per point"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting","no headings"]}],"hint":"AI summaries often get small numbers wrong or add a fact the text never said. Give the AI three exact labels and make the reading the only judge.","rubricLabels":["Labels points 1, 2 and 4 Supported, points 3 and 6 Contradicted, and point 5 Not in source","The Contradicted lines quote what the source says: point 3 gives twelve miles, and point 6 gives Saturdays from May to October","Uses only the three labels Supported, Contradicted and Not in source, with no rewritten summary, teaching tips or intro"]}