{"id":"sec-sb-chatbot-backup-advice-check","audience":"small-business","category":"security","track":"security","technique":"verification","title":"Check a chatbot's backup advice against a trusted guide","free":false,"task":"A chatbot gave you backup advice. Check each of its 4 claims against the state small business guide only, and write exactly 4 numbered lines starting 'Agrees with guide', 'Contradicts guide' or 'Not in guide', each with the guide's section number when there is one. Plain text, under 90 words.","materialLabel":"Chatbot answer and guide excerpt (attached to your prompt automatically)","material":"What the chatbot said (asked by Hal, owner of Hal's Hardware, 5 staff):\n\"Great question! Here's how to protect your business data:\n1. Keep one backup on an external drive that stays plugged into your main computer, so it's always up to date.\n2. Test restoring a few files from your backup at least every 3 months.\n3. Cloud storage that syncs your files counts as a backup, so if you use it you're covered.\n4. Use backup software that encrypts your files with military-grade AES-256 encryption.\"\n\nState Small Business Cyber Guide, 2026 edition, section 4 \"Backups\" (excerpt):\n4.1 Keep at least one backup copy disconnected from your network or computer, so ransomware can't reach it.\n4.2 Test restoring files at least once every quarter. A backup you haven't tested may not work.\n4.3 File sync services copy deletions and damaged files right away. Sync alone is not a backup; use a service with version history of at least 30 days.\n4.4 Write down who is responsible for checking backups each week.","rules":[{"type":"numberedItems","count":4},{"type":"plainText"},{"type":"maxWords","max":90}],"mistakes":[{"title":"Grading by what sounds right","why":"The AI judges with general knowledge and says encryption advice 'agrees' with best practice.","fix":"Say: \"Check only against the guide. If the guide doesn't cover it, say Not in guide.\"","checks":["r1","r2"]},{"title":"No fixed labels","why":"The AI writes 'mostly correct' or 'partly true'.","fix":"Give the labels: \"Agrees with guide, Contradicts guide, or Not in guide.\"","checks":["r0"]},{"title":"A long rewrite","why":"The AI writes its own backup plan.","fix":"Be exact: \"Exactly 4 numbered lines, one per claim.\"","checks":["numberedItems","maxWords"]},{"title":"Letting markdown in","why":"AI adds bold and headings by default, and in your notes they show up as stray symbols.","fix":"Add: \"Plain text only, no bold, no headings.\"","checks":["plainText"]}],"promptHints":[{"label":"Checks against the guide only","patterns":["guide","only","not general"]},{"label":"Gives the labels","patterns":["agrees","contradicts","not in guide"]},{"label":"Asks for section numbers","patterns":["section","4\\.\\d","cite"]},{"label":"Says exactly how many","patterns":["\\b4\\b.{0,25}(line|claim|item)","exactly (4|four)"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting","no headings"]}],"hint":"AI can't fact-check itself without a source. Give it the guide as the only judge, with three fixed labels, including one for 'the guide doesn't say'.","rubricLabels":["Each line starts with exactly 'Agrees with guide', 'Contradicts guide' or 'Not in guide'","The labels are: claim 1 Contradicts guide (4.1), claim 2 Agrees with guide (4.2), claim 3 Contradicts guide (4.3), claim 4 Not in guide","Claim 4's line does not invent a guide section or say the guide recommends AES-256"]}