{"id":"res-hc-hand-hygiene-chatbot-check","audience":"healthcare","category":"fact-checking","technique":"verification","title":"Check a chatbot's hand hygiene summary before new aides see it","free":false,"task":"You asked a chatbot to summarize your hospital's hand hygiene policy for new patient care aides, and it gave 5 points. You want exactly 5 numbered lines, one per point, labeled Matches policy, Wrong, or Not in policy, with the policy's version after any Wrong label. Use only the policy text, plain text, nothing else.","materialLabel":"The hand hygiene policy and the chatbot's summary (attached to your prompt automatically)","material":"HAND HYGIENE POLICY - Brookfield Community Hospital (IC-104, rev. 2026)\n1. Clean hands before and after every patient contact, and after touching patient surroundings.\n2. Alcohol-based hand rub is the preferred method when hands are not visibly soiled.\n3. Use soap and water when hands are visibly soiled, after using the restroom, and when caring for patients on Contact Plus precautions (signage on the door).\n4. Rub hand sanitizer over all surfaces until dry, about 20 seconds.\n5. Gloves do not replace hand hygiene. Clean hands before putting on and after removing gloves.\n6. Artificial nails are not allowed for staff with direct patient contact.\n\n-----\nCHATBOT SUMMARY FOR NEW AIDES\n1. Clean your hands before and after every patient contact.\n2. Soap and water is always better than hand sanitizer, so use it whenever you can.\n3. If you're wearing gloves, you don't need to clean your hands.\n4. Rub sanitizer until your hands are dry, about 20 seconds.\n5. Wash with soap and water for at least 60 seconds.","rules":[{"type":"numberedItems","count":5},{"type":"plainText"}],"mistakes":[{"title":"Checking against general knowledge","why":"The AI compares the summary to what it knows about handwashing, not to your hospital's policy.","fix":"Say: \"Judge only by the policy text.\"","checks":["r1","r2"]},{"title":"Calling missing details wrong","why":"The policy gives no washing time, so the 60 seconds is unsupported rather than contradicted.","fix":"Add: \"If the policy doesn't say, label it Not in policy.\"","checks":["r2"]},{"title":"No fixed labels","why":"The AI writes a long review, which is hard to use when fixing a handout.","fix":"Give them: \"Matches policy, Wrong, or Not in policy.\"","checks":["r0","r2"]},{"title":"Not saying how many","why":"The AI decides the length itself and usually returns more lines, one per point than you can use.","fix":"Be exact: \"exactly 5 numbered lines, one per point.\"","checks":["numberedItems"]},{"title":"Getting markdown back","why":"AI defaults to bold labels and headings, which show up as stray symbols in the training handout.","fix":"Ask for \"plain text, no bold, no headings.\"","checks":["plainText"]}],"promptHints":[{"label":"Gives exact labels","patterns":["matches","not in (the )?policy","label"]},{"label":"Names the source","patterns":["only.{0,30}policy","policy.{0,20}(only|source|truth)","ic-104"]},{"label":"Asks for the policy's version","patterns":["policy('s)? (version|says)","correct version","what the policy says"]},{"label":"Says exactly how many","patterns":["\\b(5|five)\\b.{0,25}(line|point)","exactly \\d+"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting","no headings"]}],"hint":"A training handout has to match your hospital's policy, not general advice. Make the policy the only judge and give the AI three exact labels.","rubricLabels":["Labels points 1 and 4 Matches policy","Labels point 2 Wrong (policy prefers hand rub unless hands are visibly soiled, after restroom use, or Contact Plus) and point 3 Wrong (gloves do not replace hand hygiene; clean before and after)","Labels point 5 (60 seconds of washing) Not in policy, and outputs only the 5 labeled lines with no intro, no rewritten summary and no extra safety advice"]}