{"id":"hs-fact-check-chatbot-answer","audience":"high-school","category":"fact-checking","technique":"verification","title":"Check a chatbot's answer against your textbook","task":"You asked a chatbot about the Constitutional Convention for your civics review, and you're not sure its answer is right. Have the AI check that answer against your textbook excerpt only. You want exactly 3 numbered lines, one per claim the textbook contradicts, each in the shape 'Chatbot said X; textbook says Y', under 25 words each, plain text, nothing else. Claims the textbook supports or doesn't mention should not be listed.","free":true,"materialLabel":"The chatbot's answer and your textbook (attached to your prompt automatically)","material":"CHATBOT ANSWER (copied from my review chat)\nThe Constitutional Convention met in Boston in the summer of 1787. About 70 delegates attended. James Madison, often called the Father of the Constitution, took detailed notes on the debates. George Washington was chosen to lead the meetings. On September 17, 1787, 39 delegates signed the Constitution. It went into effect after the ninth state ratified it in 1788. In 1791, the Bill of Rights added 12 amendments protecting freedoms like speech and religion.\n\nMY TEXTBOOK (Civics Today, ch. 3, p. 58)\nIn May 1787, 55 delegates gathered in Philadelphia to fix the weak Articles of Confederation. They soon decided to write a new plan of government instead. James Madison's ideas shaped much of the plan, and he is often called the Father of the Constitution. After months of debate and compromise, 39 delegates signed the Constitution on September 17, 1787. The Constitution needed approval from 9 of the 13 states. When New Hampshire became the ninth state to ratify in June 1788, it went into effect. Many Americans wanted a list of protected rights, so in 1789 Congress proposed 12 amendments. The states ratified 10 of them in 1791. These 10 amendments are known as the Bill of Rights.\n\n(civics unit test Thurs. study ch 3-4)","rules":[{"type":"numberedItems","count":3},{"type":"maxWords","max":90},{"type":"plainText"}],"mistakes":[{"title":"Asking \"is this right?\"","why":"The AI answers in long prose, mixing in what it knows, and mentions correct claims too.","fix":"Ask for a list: \"Exactly 3 numbered lines, one per claim the textbook contradicts.\"","checks":["numberedItems","maxWords"]},{"title":"Trusting the AI to count words","why":"AI is bad at counting. Ask for a total and it often lands 20% over.","fix":"Cap each line: \"each line under 20 words.\"","checks":["maxWords"]},{"title":"Not naming the referee","why":"The AI checks the answer against its own memory, which can be wrong too. Your test follows your textbook.","fix":"Say: \"Use only my textbook excerpt as the standard. Skip claims it doesn't mention.\"","checks":["r1"]},{"title":"No shape for the corrections","why":"The AI might just say 'the location is wrong' without the right answer, which you need to study.","fix":"Show it: \"Each line: Chatbot said X; textbook says Y.\"","checks":["r2","r0"]},{"title":"Forgetting it's going in your notes","why":"AI uses bold and headings, which make a messy paste into your study notes.","fix":"Add: \"Plain text, nothing before or after the list.\"","checks":["plainText"]}],"promptHints":[{"label":"Uses the textbook as the standard","patterns":["textbook","only.{0,20}(excerpt|source)","against"]},{"label":"Asks for only the contradictions","patterns":["contradict","disagree","wrong","doesn't match","does not match"]},{"label":"Shows the line shape","patterns":["chatbot said","textbook says","said x","shape","format"]},{"label":"Limits the lines","patterns":["\\d+\\s*words","\\b(3|three)\\b.{0,15}(line|item|claim)"]},{"label":"Skips what checks out","patterns":["skip","don't (list|mention|include)","do not (list|mention|include)","only (list|the)"]}],"hint":"An AI checking another AI will use its own memory unless you name the referee. Tell it to judge only by your textbook, list only what conflicts, and show each fix side by side.","rubricLabels":["Lists all 3 contradictions: Boston (textbook: Philadelphia), about 70 delegates (textbook: 55), and 12 amendments in the Bill of Rights (textbook: 12 proposed, 10 ratified)","Doesn't list claims the textbook supports or doesn't mention (Madison, 39 signers, Sept 17, 1788 ratification, Washington leading the meetings), and adds no outside facts","Each line uses the shape 'Chatbot said ...; textbook says ...' (or very close), stating both what the chatbot said and what the textbook says"]}