Library › Marketing AI › Product managers

Write an A/B test plan with sample size and run time worked out

  • For Product managers
  • You practice Setting limits and rules
  • Time about 5 minutes
  • Level Advanced

About this exercise

You're planning a test on the pricing page headline. Write a prompt for exactly six plain-text lines labeled Hypothesis:, Change:, Primary metric:, Sample per variant:, Days to run: and Stop rule:. Days to run shows the math from the daily traffic split between two variants, then rounds up to whole weeks. Under 100 words, and no test of anything beyond the one change.

Common mistakes when you ask AI for this

  • Stopping when ahead. The AI follows the VP's wish. Try: Say: "Stop rule: run the full sample and whole weeks."
  • Bundling extra changes. The AI adds the button color and testimonial. Try: Say: "One change only: the headline."
  • Forgetting to round to weeks. The AI says 16 days. Try: Say: "Show the math, then round up to whole weeks."
  • Missing labels. The AI writes a narrative plan. Try: List the six labels.
  • Letting markdown through. AI adds bold text and headings by default. Try: Add: "Plain text only, no bold, no headings."
  • Trusting the AI to count words. AI is bad at counting and often runs past a word limit. Try: Cap each part (for example "each line under 15 words") so the total lands well under the limit.

What a strong prompt includes

  • Lists the six labels
  • Shows the days math
  • Rounds to weeks
  • Limits to one change
  • Sets the stop rule

Practice this now ↑

Related exercises