{"id":"eval-hr-error-analysis-resume-parser-fields","audience":"hr","category":"evaluation","track":"evals","technique":"format","title":"Sort 13 AI resume-parsing errors into 3 failure types","free":false,"task":"Your applicant system's AI parser fills in fields from resumes, and you logged 13 errors last week. Group them into exactly 3 numbered failure types, each line 'Type: #ids = count', then one last line 'Total: N'. Leave out any error caused by the candidate's own typo. Plain text, under 70 words.","materialLabel":"Your log of parsing errors (attached to your prompt automatically)","material":"Pell & Rowan Staffing - AI resume parser error log, Nov 3-7 (recruiting ops)\n\n#1 start date read as 2021, resume says 2012\n#2 job title \"Senior Analyst\" saved as \"Analyst\"\n#3 forklift certification on the resume, not captured\n#4 end date \"Present\" saved as the parse date\n#5 candidate wrote \"Mangaer\" on their resume; parser copied it\n#6 \"Team Lead, Warehouse\" saved as \"Warehouse\"\n#7 CPR certificate missing from the certifications field\n#8 graduation year 2019 saved as 2009\n#9 two jobs merged into one title \"Driver Dispatcher\"\n#10 CDL Class A license not captured\n#11 months swapped: 03/07 read as July instead of March\n#12 \"Assistant Store Manager\" saved as \"Store Manager\"\n#13 OSHA 10 card listed in the resume, field left blank\n\nContext: the vendor asked which field type to fix first.","rules":[{"type":"numberedItems","count":3},{"type":"plainText"},{"type":"maxWords","max":70}],"mistakes":[{"title":"Too many types","why":"The AI splits start dates, end dates and graduation years into separate groups.","fix":"Say: \"Exactly 3 failure types.\"","checks":["numberedItems","r0"]},{"title":"No ids or counts","why":"The AI describes the errors without showing which ones belong where.","fix":"Show the shape: \"Type: #1, #4 = 2\".","checks":["r2"]},{"title":"Counting the candidate's typo","why":"The AI counts #5, which the parser copied correctly from the resume.","fix":"Add: \"Leave out errors from the candidate's own typo, and end with Total: N.\"","checks":["r1"]},{"title":"Getting markdown back","why":"AI defaults to bold headings and long bullets.","fix":"Ask for \"plain text, no bold, no headings.\"","checks":["plainText","maxWords"]}],"promptHints":[{"label":"Says exactly how many","patterns":["\\b(3|three)\\b.{0,25}(type|group|categor)","exactly \\d+"]},{"label":"Asks for ids and counts","patterns":["#ids","\\bids\\b","count","how many"]},{"label":"Leaves out the candidate's typo","patterns":["#5","typo","candidate'?s","leave out|exclude"]},{"label":"Asks for a total","patterns":["total"]},{"label":"Asks for plain text","patterns":["plain","no bold","no markdown","no formatting","no headings"]}],"hint":"To tell the vendor what to fix first, you need failure types with counts. Fix the number of types, ask for the ids behind each count, and drop the error that wasn't the parser's fault.","rubricLabels":["Uses these 3 groups (names may differ): wrong dates #1, #4, #8, #11 = 4; wrong or changed job title #2, #6, #9, #12 = 4; missed certification or license #3, #7, #10, #13 = 4","Leaves out #5 and ends with a line saying Total: 12","Each type line lists its # ids followed by the count, and each count equals the number of ids on that line"]}