Too little traffic for A/B tests? Do this instead

The standard advice for improving a page is to A/B test it: show half your visitors one version and half another, then keep the winner. It works if you have the traffic. Most small sites don't, and running a test anyway mostly teaches you to trust noise.

The math, quickly

Say 2% of your visitors buy or sign up, and you think a new headline could lift that to 2.4%. That's a 20% improvement, which would be a great result. To tell it apart from luck with the usual level of confidence, you need roughly 21,000 visitors on each version, about 42,000 in total.

At 5,000 visitors a month, that's more than eight months for a single test. At 500 a month, it's about seven years.

Bigger effects need fewer visitors. Going from 2% to 3% needs about 3,800 visitors per version, which is still around 15 months at 500 visitors a month, and very few headline changes are that good.

So the rule our Growth Plans follow is simple: under about 5,000 visitors a month, skip split tests and use the methods below. They're faster, cheaper, and they tell you why, which a split test never does.

Run a five-second test with five people

Show your homepage to someone in your audience for five seconds, then hide it and ask: What does this business do? Who is it for? What would you click first? Do that with five people. A video call with screen sharing works fine, and so does your phone across a table.

If two or more of them can't say what you do, your headline is the problem. You've learned that in an afternoon instead of a year. (The headline guide covers the fix.)

Watch three people try to use it

Ask three people who fit your audience to do the main thing on your page while sharing their screen and thinking out loud: "find out what it costs and sign up," or "book a call." Don't help and don't explain. Write down every pause, every scroll back up, every "hmm."

Three sessions usually surface the same two or three problems. That's your to-do list.

Ask the people who almost didn't buy

If you have even a handful of customers or sign-ups, send them one question: "What almost stopped you?" The answers are the doubts your page needs to answer, in your customers' own words. Use those words on the page.

No customers yet? Ask five people in your audience what would stop them from buying. It's not the same, but it's far better than guessing.

Change one thing, then compare before and after

You can still measure, just more roughly. Change one thing (the headline, the button text, how the price is shown), note the date, and compare the four weeks after with the four weeks before.

Be honest about what that can tell you. A holiday, a newsletter mention or one big referral can swamp the difference. Treat a before and after as a hint, not a verdict, and only trust big swings.

Watch recordings, not heatmaps

Session recording tools let you watch real visits one at a time. At low traffic, ten recordings tell you more than a heatmap built from a few hundred clicks. Look for the same things as in the live sessions: where people stop, what they tap that isn't a link, and where they leave.

When you do have the traffic

Past roughly 5,000 visitors a month, with a decent number of sign-ups or sales, split tests start to pay off. Test big, clear differences rather than shades of button color, run each one for at least two full weeks, and decide before you start what result would make you switch.

Until then, five people and a stopwatch will teach you more.