{"id":"mkt2-signup-lift-refund-guardrail","audience":"product-managers","category":"data","track":"marketing","technique":"verification","level":"advanced","title":"Apply a ship rule to a test: signup lift with a refund guardrail","free":false,"task":"Your team's rule for the new pricing page test is to ship only if signups rise by at least 5% relative and the refund rate (refunds divided by signups) doesn't get worse by more than 1 percentage point. Write a prompt for exactly 3 plain-text lines: \"Lift:\" with the relative signup lift shown with its numbers, \"Guardrail:\" with both refund rates and the difference, and \"Decision:\" Ship or Don't ship plus a reason under 12 words. Under 80 words.","materialLabel":"Test results (attached to your prompt automatically)","material":"PRICING PAGE TEST - results (equal traffic, planned sample reached)\n\nVariant A (current page): 2,000 visitors, 100 signups, 8 refunds within 14 days\nVariant B (new annual-first layout): 2,000 visitors, 112 signups, 16 refunds within 14 days\n\nSignup rate = signups / visitors. Refund rate = refunds / signups.\nRule: ship only if relative signup lift is at least 5% AND the refund rate is no more than 1 percentage point higher than A's.\n\nNotes: the growth lead is excited about the 12% lift and wants to ship Monday. Revenue per signup is similar in both variants. Support tickets about billing were higher for B (21 vs 9).","rules":[{"type":"plainText"},{"type":"maxWords","max":80}],"mistakes":[{"title":"Shipping on the lift alone","why":"The AI follows the excited lead and ignores refunds.","fix":"Say: \"Ship only if both conditions pass; the refund rate is a guardrail.\"","checks":["r2"]},{"title":"Dividing by visitors for refunds","why":"Refund rate uses signups, not visitors.","fix":"Say: \"Refund rate = refunds / signups.\"","checks":["r1"]},{"title":"Skipping the numbers","why":"A verdict without math can't be checked.","fix":"Say: \"Show the numbers on each line.\"","checks":["r0"]},{"title":"Letting markdown through","why":"AI adds bold text and headings by default.","fix":"Add: \"Plain text only, no bold, no headings.\"","checks":["plainText"]},{"title":"Trusting the AI to count words","why":"AI is bad at counting and often runs past a word limit.","fix":"Cap each part (for example \"each line under 15 words\") so the total lands well under the limit.","checks":["maxWords"]}],"promptHints":[{"label":"Names the three labels","patterns":["lift:.*guardrail:.*decision:"]},{"label":"States the refund formula","patterns":["refunds (divided by|/) signups"]},{"label":"Requires both conditions","patterns":["both","and the refund","guardrail"]},{"label":"Shows numbers","patterns":["show","numbers","math"]},{"label":"Caps reason and length","patterns":["under 12 words","80 words"]}],"hint":"A ship rule has two conditions, and the guardrail can veto an exciting lift. State the formulas, both conditions and the three lines.","mission":"Decide whether to ship the new pricing page.","musts":["Lift must be 5%+","Refund rise under 1 point","Show the numbers"],"rubricLabels":["The Lift line shows A at 5.0% and B at 5.6% of visitors, a 12% relative lift, which meets the 5% threshold","The Guardrail line shows A at 8.0% and B at about 14.3% refund rate, a difference of about 6.3 points, which fails the 1-point limit","The Decision line says Don't ship with a reason under 12 words, and the output has exactly 3 lines"]}