{"id":"sec-it-chatbot-password-advice-check","audience":"it-security","category":"security","track":"security","technique":"verification","title":"Check a chatbot's password advice against your standard","free":false,"task":"A manager pasted a chatbot's password advice and asked if it's right. Check each of its 5 points against your company standard only, not general knowledge. Write exactly 5 numbered lines, each starting with exactly 'Matches standard' or 'Outdated'. Each Outdated line cites the section number (like 3.1) and what it says instead. End with the line 'Verdict for Ken: Don't use as is'. Plain text, under 90 words.","materialLabel":"Chatbot answer and your company standard (attached to your prompt automatically)","material":"Chatbot answer (pasted by Ken, Ops manager):\n\"Here's a strong password policy for your staff:\n1. Require passwords at least 8 characters long.\n2. Make everyone change their password every 90 days.\n3. Require at least one uppercase letter, one number and one special character.\n4. Set up security questions (like your first pet's name) as a backup way to reset passwords.\n5. Encourage a password manager so people can keep long, unique passwords.\"\n\nCompany IT Security Standard v4 (updated Aug 2026), section 3 Passwords:\n3.1 Minimum length is 14 characters. Longer passphrases are encouraged.\n3.2 No scheduled password changes. Change a password only when there is evidence it was exposed.\n3.3 No character-mix rules. Instead, every new password is checked against a list of breached and common passwords.\n3.4 Security questions are not allowed for resets. Use the self-service reset, which requires MFA.\n3.5 All staff get Keyvault Teams, our approved password manager.","rules":[{"type":"numberedItems","count":5},{"type":"plainText"},{"type":"maxWords","max":90}],"mistakes":[{"title":"Grading against general knowledge","why":"The AI thinks 8 characters is acceptable in general and calls point 1 fine.","fix":"Say: \"Check each point against our standard only, not general knowledge.\"","checks":["r0"]},{"title":"Saying wrong without saying right","why":"The AI marks points as bad but doesn't give the rule to use instead.","fix":"Add: \"For each Outdated line, say what the standard says instead.\"","checks":["r1"]},{"title":"Missing your specifics","why":"The AI recommends 'a password manager' instead of the one you provide, and ends without a clear answer for Ken.","fix":"Ask: \"Name our approved tools. End with: Verdict for Ken: Don't use as is.\"","checks":["r2"]},{"title":"A long rewrite","why":"The AI writes a whole new policy with headings.","fix":"Be exact: \"Exactly 5 numbered lines, one per point, each under 14 words.\"","checks":["numberedItems","maxWords"]},{"title":"No section numbers","why":"The AI explains in its own words, so Ken can't check the source.","fix":"Ask: \"Cite the section number, like 3.1, on each Outdated line.\"","checks":["r1"]},{"title":"Letting markdown in","why":"AI adds bold and headings by default, and in a reply to Ken they show up as stray symbols.","fix":"Add: \"Plain text only, no bold, no headings.\"","checks":["plainText"]}],"promptHints":[{"label":"Checks against your standard","patterns":["standard","our (policy|rules)","only","not general"]},{"label":"Gives exact labels","patterns":["matches","outdated","label"]},{"label":"Asks for the right answer","patterns":["instead","what the standard says","correct","replacement"]},{"label":"Says exactly how many","patterns":["\\b5\\b.{0,25}(line|point|item)","exactly (5|five)"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting","no headings"]}],"hint":"The AI will grade advice against what it already believes. Make it grade against your standard, with fixed labels and the right rule for each miss.","rubricLabels":["Each of the 5 lines starts with exactly 'Matches standard' or 'Outdated': points 1 to 4 Outdated and point 5 Matches standard","Each Outdated line cites its section number (3.1, 3.2, 3.3, 3.4) and the standard's rule (14 characters, change only when exposed, check against breached passwords, MFA self-service reset)","Point 5's line names Keyvault Teams as the approved password manager, and the last line is 'Verdict for Ken: Don't use as is'"]}