Library › Data AI › Product managers

Work out when an A/B test will reach its sample size

  • For Product managers
  • You practice Catching mistakes and made-up facts
  • Time about 5 minutes
  • Level Intermediate

About this exercise

Your VP asks when the checkout test can be called. Write exactly 4 labeled lines in plain text, under 80 words, with the math shown: Needed, So far, Remaining and Finish date. Count the start day as day 1, assume traffic stays the same every day, and give the finish date as the last day of the test.

Common mistakes when you ask AI for this

  • Dividing the wrong numbers. The AI divides 8,000 by 400 and forgets the 2,000 already collected. Try: Add: "Remaining: 8,000 - 2,000 = 6,000, then 6,000 / 400 = 15 days."
  • Off-by-one date. The AI counts from Friday or double counts the start day. Try: Add: "Count the 15 days from Saturday Nov 8; the last day is Saturday Nov 22."
  • Adding the sale. The AI shortens the test because of the Nov 14 sale. Try: Add: "Ignore the sale; assume 400 visitors per day."
  • Leaving the format to chance. The AI picks its own length and layout unless told the exact shape. Try: Add: "Plain text, no bold or headings."
  • Trusting the AI to count words. AI is bad at counting. Ask for a total word limit and it often lands over it. Try: Cap each part instead, for example "each line under 15 words."

What a strong prompt includes

  • Asks for the labels
  • Shows the subtraction
  • Gives the day count
  • Says where to start counting
  • Rules out the sale

Practice this now ↑

Related exercises