{"id":"col-check-ai-answer","audience":"college","category":"fact-checking","technique":"verification","title":"Check an AI answer against your textbook","free":true,"task":"You asked a chatbot about the causes of the Dust Bowl for your American History 120 reading quiz, and you want to check its answer against your assigned textbook pages before studying from it. You want each claim in the AI answer that the textbook contradicts, as a numbered line in the form \"AI said X; textbook says Y\", under 100 words in total, in plain text, with nothing else. Treat the textbook as the only source of truth, and don't flag claims it simply doesn't mention.","materialLabel":"Textbook excerpt and the AI answer (attached to your prompt automatically)","material":"TEXTBOOK - The American Journey, Ch. 22, p. 641 (assigned reading)\n\nIn the 1920s, high wheat prices and new tractors led farmers on the southern Plains to plow up millions of acres of native grassland. The grass roots had held the soil in place. When a severe drought began in 1931, crops failed and the exposed topsoil dried out. Strong winds lifted it into huge dust storms. The worst-hit area included the Oklahoma and Texas panhandles and western Kansas. The most famous storm, \"Black Sunday,\" struck on April 14, 1935. That same year, Congress created the Soil Conservation Service to teach farmers methods such as contour plowing and planting rows of trees as windbreaks. Many families lost their farms and migrated west, most often to California.\n\n-----\nAI ANSWER (what the chatbot told me)\n\nThe Dust Bowl was caused by a mix of human choices and nature. During the 1920s, farmers on the southern Plains plowed up millions of acres of grassland whose roots had held the soil. The drought began in 1929, right as the stock market crashed, and lasted most of the decade. Winds then carried the loose topsoil away in massive storms, especially in the Oklahoma and Texas panhandles. The worst storm, Black Sunday, hit in 1937. In response, the government created the Soil Conservation Service in 1933 as part of the first hundred days of the New Deal. Folk singer Woody Guthrie later wrote songs about the dust storms. Many displaced families moved to California looking for work.","rules":[{"type":"maxWords","max":100},{"type":"plainText"}],"mistakes":[{"title":"Asking \"is this right?\"","why":"The AI writes a long review of the whole answer, praising the good parts and mixing in its own knowledge.","fix":"Name the job: \"List only claims the textbook contradicts.\"","checks":["r2"]},{"title":"Letting the AI use its own knowledge","why":"The AI checks against what it remembers, not your textbook, and may flag things your class never covered.","fix":"Say: \"The textbook excerpt is the only source of truth.\"","checks":["r0"]},{"title":"Counting 'not mentioned' as wrong","why":"The AI treats anything missing from the textbook as an error, so true but extra details get flagged.","fix":"Add: \"Skip claims the textbook doesn't mention.\"","checks":["r1"]},{"title":"Trusting the AI to count words","why":"AI is bad at counting. Ask for a total word limit and it often lands 10-20% over.","fix":"Cap each part instead: \"each line under 15 words.\"","checks":["maxWords"]},{"title":"Getting a formatted report","why":"AI defaults to bold headings and a table, which is hard to paste into your notes.","fix":"Ask for \"plain text numbered lines, no bold.\"","checks":["plainText"]}],"promptHints":[{"label":"Names the source of truth","patterns":["textbook.{0,30}(only|source|truth)","only.{0,20}textbook"]},{"label":"Asks for contradictions only","patterns":["contradict","conflict","disagree"]},{"label":"Skips unmentioned claims","patterns":["(doesn't|does not|not) mention","not in the textbook","skip"]},{"label":"Gives the line format","patterns":["ai said","textbook says","\\bX\\b.{0,20}\\bY\\b"]},{"label":"Sets a length limit","patterns":["\\d+\\s*words","one line","word (limit|count)"]}],"hint":"A fact-check is only as good as what you check against. Tell the AI which source wins, what counts as a contradiction, and the exact line format you want back.","rubricLabels":["Flags all three contradictions: when the drought began (AI: 1929; textbook: 1931), the date of Black Sunday (AI: 1937; textbook: April 14, 1935), and when the Soil Conservation Service was created (AI: 1933; textbook: 1935)","Does not flag claims the textbook doesn't mention (such as the Woody Guthrie songs or the stock market crash on its own) or claims that match the textbook (plowed-up grassland, the panhandles, moving to California) as contradictions","Outputs only the numbered lines in the \"AI said X; textbook says Y\" form: no intro, no corrected rewrite of the answer, no closing advice"]}