{"id":"eval-col-five-tests-citation-formatter-bot","audience":"college","category":"evaluation","track":"evals","technique":"context","title":"Write 5 edge-case tests for a campus citation bot","free":false,"task":"You're helping the writing center test its APA citation bot, and you need 5 test cases: an empty message, an adversarial request (asking it to write the paper), the wrong style (asking for MLA), a very long input (40 sources), and a refusal case (asking it to make up a source). Write exactly 5 numbered lines, each 'N. Case | Message: \"exact message\" | Expected: what the bot should do per its rules'. Plain text, under 200 words.","materialLabel":"The citation bot's rules (attached to your prompt automatically)","material":"Hollins State Writing Center - \"CiteRight\" bot, rules for the spring pilot\n\nWhat it does\n- Formats references in APA 7 from details the student pastes (author, year, title, source, URL or DOI).\n\nRules\n- APA 7 only. If a student asks for MLA, Chicago or another style, it says it only does APA 7 and links the library's style guides page.\n- Up to 25 sources per message. If a student pastes more, it formats the first 25 and asks them to send the rest in a new message.\n- It never writes or rewrites any part of a paper (essays, paragraphs, thesis statements). It says it only formats citations and suggests booking a writing center tutor.\n- It never invents a source, author, page number or DOI. If asked for \"a source that says X\", it says it can't create sources and suggests the library's research help chat.\n- Empty or blank message: it asks the student to paste the source details.\n\nPilot notes\n- A student asked it to \"just write my intro paragraph\" and it did. Fixed in v2.\n- Most students paste 5-10 sources.\n- The library also wants a Zotero export someday. Not in this pilot.","rules":[{"type":"numberedItems","count":5},{"type":"plainText"},{"type":"maxWords","max":200}],"mistakes":[{"title":"Generic test cases","why":"The AI writes 'test that it handles errors', which no one can run.","fix":"Say: \"Each Message is an exact student message in quotes.\"","checks":["r1"]},{"title":"Not naming the edge cases","why":"The AI tests normal citations and forgets the adversarial and refusal ones.","fix":"List them: \"empty, write my paper, MLA, 40 sources, make up a source.\"","checks":["r0"]},{"title":"Vague expected results","why":"The AI writes 'responds appropriately' instead of what the rules require.","fix":"Add: \"Expected says exactly what the bot's rules require.\"","checks":["r2"]},{"title":"Too many tests","why":"The AI writes a long test plan with 10 or more cases.","fix":"Be exact: \"Exactly 5 numbered lines.\"","checks":["numberedItems","maxWords","plainText"]}],"promptHints":[{"label":"Names the edge cases","patterns":["empty","write (my|the) paper|adversarial","mla|wrong style","40|long","make up|invent|refus"]},{"label":"Asks for exact messages","patterns":["exact","in quotes","quoted","word for word"]},{"label":"Ties expected to the rules","patterns":["rules","expected","should do"]},{"label":"Says exactly how many","patterns":["\\b(5|five)\\b.{0,25}(test|case|line)","exactly \\d+"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting","no headings"]}],"hint":"Edge-case tests show whether the bot keeps its rules when students push. Name each case, ask for the exact message, and tie each expected result to the bot's rules.","rubricLabels":["Has exactly the 5 named cases, one each: empty message, adversarial request to write the paper, wrong style (MLA), very long input (40 sources), and a request to make up a source","Every Message is an exact student message in quotation marks a tester could paste (the 40-source case may describe the pasted list), not a description like 'a student asks for help'","Every Expected matches the rules: asks for source details; refuses to write and suggests a writing center tutor; says APA 7 only and links the style guides page; formats the first 25 and asks for the rest in a new message; says it can't create sources and suggests the research help chat"]}