{"id":"res-hs-volcano-chatbot-labels","audience":"high-school","category":"fact-checking","technique":"verification","title":"Label each chatbot claim against your earth science text","free":false,"task":"You asked a chatbot about the 1980 Mount St. Helens eruption for Earth Science, and it gave you 8 numbered claims. You want one numbered line per claim, under 90 words total, each labeled Confirmed, Wrong, or Not in textbook, and every Wrong line must give the textbook's version. Judge only by your textbook excerpt, not by what the AI already knows. Plain text, nothing else.","materialLabel":"Textbook excerpt and the chatbot's claims (attached to your prompt automatically)","material":"TEXTBOOK - Exploring Earth Science, Ch. 9, p. 212\n\nMount St. Helens, in Washington State, erupted on May 18, 1980. A magnitude 5.1 earthquake beneath the volcano triggered the collapse of its north face, the largest landslide in recorded history. The sideways blast that followed flattened about 230 square miles of forest. Fifty-seven people died. Ash from the eruption fell across 11 states. The eruption lowered the mountain's summit by about 1,300 feet, leaving a horseshoe-shaped crater open to the north.\n\n-----\nCHATBOT ANSWER (what it told me)\n\n1. Mount St. Helens is a volcano in Oregon.\n2. It erupted on May 18, 1980.\n3. A magnitude 5.1 earthquake set off a huge landslide on the north side.\n4. The blast flattened about 230 square miles of forest.\n5. 157 people were killed in the eruption.\n6. President Carter toured the damage a few days later.\n7. The eruption lowered the summit by about 300 feet.\n8. It is the most active volcano in the Cascade Range today.","rules":[{"type":"maxWords","max":90},{"type":"plainText"}],"mistakes":[{"title":"Letting the AI use its own memory","why":"The AI checks claims against what it remembers, so it may call the Carter visit true even though your textbook never mentions it.","fix":"Say: \"Judge only by the textbook excerpt.\"","checks":["r1"]},{"title":"No fixed labels","why":"Without labels the AI writes a paragraph per claim, and you can't tell at a glance which ones to fix.","fix":"Give them: \"Label each Confirmed, Wrong, or Not in textbook.\"","checks":["r2","r1"]},{"title":"Not asking for the right version","why":"The AI may just say \"incorrect\" without the textbook's number, so you still don't know what to study.","fix":"Add: \"For each Wrong line, give the textbook's version.\"","checks":["r0"]},{"title":"Trusting the AI to count words","why":"AI is bad at counting. Ask for a total word limit and it often lands 10-20% over.","fix":"Cap each line: \"each line under 10 words.\"","checks":["maxWords"]},{"title":"Getting markdown back","why":"AI defaults to bold labels and headings, which show up as stray symbols in your study notes.","fix":"Ask for \"plain text, no bold, no headings.\"","checks":["plainText"]}],"promptHints":[{"label":"Gives exact labels","patterns":["confirmed","not in (the )?textbook","label"]},{"label":"Names the source of truth","patterns":["only.{0,30}textbook","textbook.{0,30}(only|truth)","not (your|what you) (know|remember)"]},{"label":"Asks for the correct version","patterns":["textbook('s)? (version|says|figure)","correct (number|version|fact)","what the textbook says"]},{"label":"Sets a length limit","patterns":["\\d+\\s*words","one (short )?sentence","word (limit|count)"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting","no headings"]}],"hint":"A fact-check needs a referee. Tell the AI the textbook is the only judge, give it exact labels, and say what to write when a claim is wrong.","rubricLabels":["Labels claims 1, 5 and 7 as Wrong and gives the textbook's version for each (Washington State; 57 people died; about 1,300 feet)","Labels claims 2, 3 and 4 Confirmed and claims 6 and 8 Not in textbook (not Wrong and not Confirmed, even if they may be true)","Outputs only the 8 labeled lines using the labels Confirmed, Wrong or Not in textbook: no intro, no corrected rewrite, no extra facts"]}