{"id":"evals3-prompt-patch-null-deposit","audience":"developers","category":"evaluation","track":"evals","stage":"fix-prompt","level":"intermediate","technique":"constraints","title":"Write a prompt patch for null answers and predict the result","free":false,"task":"Pedalbot also reads Spokewise rental agreements and fills a deposit field. It invents amounts when the agreement states none. Write a prompt patch of at most 2 lines in exactly 1 code block, with the first line starting \"Deposit rule:\", that states exactly when deposit must be null, what to output, and the only case where 0 is allowed. Then add one line starting \"Prediction:\" naming the one results column the patch should move and the columns that should stay flat. Keep text outside the code block under 35 words.","materialLabel":"Current prompt, results and notes (attached to your prompt automatically)","material":"SPOKEWISE - DEPOSIT EXTRACTOR (v4)\nCurrent prompt: \"Read the rental agreement and return JSON: {\"bike_type\": ..., \"daily_price\": ..., \"deposit\": ...}. Use dollars as numbers.\"\n\nRESULTS TABLE (50 test agreements, v4)\nColumn | passed | total\nDeposit correct (agreement states a deposit) | 36 | 40\nDeposit null (agreement states no deposit) | 2 | 10\nValid JSON, 3 keys | 50 | 50\nDaily price correct | 47 | 50\n\nEXAMPLES OF THE MISS\n- Agreement for a standard city bike, no deposit line -> bot returned \"deposit\": 50.\n- Agreement for a tandem, \"no deposit required\" printed in the text -> bot returned \"deposit\": 0.\n- Agreement that says \"deposit: see station\" -> bot returned \"deposit\": \"N/A\".\n\nNOTES\n- The downstream script treats null as \"ask the station\". It crashes on \"N/A\" and shows 0 as \"free\".\n- A deposit of 0 is only right if the agreement prints a deposit of $0.\n- The team will rerun the same 50 agreements after the patch.\n- Do not change the other fields' instructions; they pass today.","rules":[{"type":"codeBlocks","count":1},{"type":"maxCodeLines","max":2},{"type":"maxProseWords","max":35}],"mistakes":[{"title":"A vague rule","why":"The AI writes 'if unsure, use null', which leaves the model guessing when it feels sure.","fix":"Say: \"Deposit is null exactly when the agreement states no deposit amount.\"","checks":["r0"]},{"title":"Forgetting the output form","why":"The AI doesn't say what null looks like, so the bot keeps sending 0 or N/A.","fix":"Add: \"Output the JSON value null, not 0 or N/A.\"","checks":["r1"]},{"title":"Predicting everything improves","why":"The AI says every column will go up.","fix":"Say: \"Name the one column that should move and the ones that stay flat.\"","checks":["r2"]},{"title":"A long essay","why":"The AI rewrites the whole prompt in many code blocks.","fix":"Say: \"Exactly 1 code block of at most 2 lines, first line starting Deposit rule:.\"","checks":["codeBlocks","maxCodeLines","maxProseWords"]}],"promptHints":[{"label":"Says when to use null","patterns":["null","exactly when","only (when|if)","does not state"]},{"label":"Says what to output","patterns":["json (value )?null","not 0","n/a"]},{"label":"Asks for a prediction","patterns":["prediction","predict","column"]},{"label":"Names what stays flat","patterns":["flat","stay","unchanged"]},{"label":"Limits the size","patterns":["1 code block","one code block","\\d+\\s*lines","\\d+\\s*words"]}],"hint":"A good patch is narrow: it says exactly when a behavior happens and what the output looks like, so you can predict which column of your results it will move and which it won't.","mission":"Patch the prompt so missing deposits return null.","musts":["Patch states when null","Output JSON null","Predict which column moves"],"rubricLabels":["The patch starts with \"Deposit rule:\" and says deposit must be null whenever the agreement does not state a deposit amount, and not to guess or infer one from the bike type","The patch says to output the JSON value null (not \"N/A\" or an empty string), and that 0 is allowed only when the agreement prints a deposit of $0","The Prediction line says the Deposit null column should move and Deposit correct, Valid JSON and Daily price correct should stay flat"]}