In context
You want to test hook A ('POV: iPhone hits concrete') vs hook B ('$1,200 in your pocket — protect it'). Set up a Meta A/B test, 50/50 budget split, same audience (Advantage+ Audience, US). Run 7 days, collect 150 purchases per arm, measure CPA difference. Hook A wins at $14 CPA vs hook B at $17 CPA. Ship hook A.
Why it matters
- True A/B tests require audience separation, not just running two ads in one ad set.
- Sample size matters — under 100 conversions per arm = noise, not signal.
- Run A/B on creative far more than on targeting; creative has 10x the effect size.
Related terms
Frequently asked questions
How long to run an A/B test?
Minimum 7 days to span weekday/weekend; 14 days for more variance.
Can I A/B test more than 2 variants?
Technically yes (A/B/C), but each arm needs full sample. 2-arm tests reach significance faster.
What's a valid winner threshold?
95% statistical confidence and 15%+ CPA delta is the working standard. Under 10% delta = keep both running.