Library › Coding AI › Product managers

Compare A/B conversion, dropping users who saw both

Practice: Giving the right context. About 5 minutes.

About this exercise

Write one SQL query for the checkout_v2 experiment that returns one row per variant with users, converted and conversion_pct (rounded to 1 decimal, no integer division). Leave out users who were exposed to both variants. A user converts only with a purchase within 7 days after their first exposure, measured from first_seen. One code block, at most 32 lines, then under 25 words outside the code.

Common mistakes when you ask AI for this

  • Not mentioning double exposure. The AI counts users who saw both versions in both groups, which muddies the result. Try: Add: "Drop users exposed to both variants."
  • Leaving "converted" vague. The AI counts any purchase ever, even before the user saw the test. Try: Add: "A purchase within 7 days after first exposure."
  • Trusting integer division. Dividing two counts can round down to 0 in SQL. Try: Add: "Multiply by 100.0, round to 1 decimal, one row per variant."
  • Getting a menu of options. The AI likes to offer two or three versions of the query, so you're left to pick and test them yourself. Try: Add: "Give one query only, in one code block, no alternatives."
  • Getting a tutorial. By default the AI explains every part of the code at length, which you then have to scroll past. Try: Add: "After the code, one line under 12 words. Nothing else."

What a strong prompt includes

  • Drops double-bucketed users
  • Defines conversion
  • Names the experiment
  • Handles the percent
  • Asks for one answer only

Practice this now ↑

Related exercises