Key takeaways
- Most small-site tests cannot reach significance, and acting on them is worse than not testing
- Fix obvious friction first; watch ten session recordings before hypothesising
- A test is only worth running if its result changes a decision
The most common CRO mistake on a small site is not a bad hypothesis. It is running a properly designed test on traffic that can never resolve it, then acting on the result anyway.
The arithmetic nobody does first
To reliably detect a 10% relative improvement on a 3% conversion rate, you need tens of thousands of visitors per variant. Most B2B sites with $5,000 a month of ad spend see a few hundred. After two weeks the test shows +14%, everyone celebrates, and the "winner" is noise that will vanish in the next month.
This is not a statistical edge case. It is the normal outcome, and it is how sites end up with a landing page optimised for a random Tuesday.
What to do instead, in order
- Fix the broken things. Forms that fail silently, pricing pages with no prices, mobile layouts where the CTA is under the fold. These need doing, not testing.
- Watch ten sessions. Screen recordings of real visitors will show you three problems no test would have found, because you would never have hypothesised them.
- Run sequential changes. Change one thing, measure for two weeks against the previous period, accept that seasonality is a confound and say so in the report.
- Test only what has volume. Headline and hero changes on your highest-traffic page are testable. The pricing page of a niche product usually is not.
The questions worth asking before any test
- What decision does this result change? If the answer is "none", do not run it.
- What is the minimum detectable effect, and is that effect big enough to matter commercially?
- Who will implement the winner, and when? A test with no implementation path is research theatre.
The honest version of CRO. On a small site, most of the gain comes from fixing obvious friction, not from testing. A $900 audit that finds four broken things is worth more than a year of underpowered experiments.
When testing does make sense
High traffic, a clear funnel, and a team that can ship changes weekly. At that point, a proper programme — one hypothesis at a time, powered sample sizes, decisions written down — compounds. Below that threshold, spend the money on the form.
Start with the audit, not the test.
CRO audit — $900
