Is A/B Testing REALLY Valid?

Share This Post

Share on facebook
Share on linkedin
Share on twitter
Share on email

Based on your post it sounds like you were not conductiing A/B Split tests, that you were instead running serial tests. A/B Split tests will run the control and treatment over the same time period, randomly splitting the traffic between the control and the treatment. Serial testing does not “split” the traffic, but instead runs a control test for a period of time, then runs a second serial test of the treatment over an approximately equal amount of time.

Serial testing is never going to be as accurate as split tests because the two separate test periods will always have uncontrolled variables. Split tests will typically yield statistically valid test results in about 1/8th the time it takes to get close to the same level of validity with a serial test.

The bottom line is to always use split testing, where possible, for the most valid test results.

Subscribe To Our Newsletter

Get updates and learn from the best

More To Explore


Calling All WordPress/SEO Experts

The problem is a lot of folks think that PageRank has far more influence on SERP ranking than it actually does. You would not see

Do You Want To Boost Your Business?

drop us a line and keep in touch

We Would Be Happy To Meet You And Learn All About Your Business