Write it as one sentence with three parts: if a specific change is made, then a named metric moves in a stated direction, because of a reason your research has shown. The change must be concrete, the metric must be measurable and the reason must come from evidence about your shoppers. A hypothesis that a test cannot prove wrong is an opinion with a template around it.
What makes a hypothesis testable?
- One change, or a set of changes that serve a single idea, so the result can be traced back to it.
- One primary metric that the change can plausibly move within the test period.
- A direction: up, down, or no worse than today.
- A defined audience, such as mobile visitors on product pages.
- An outcome that could prove it wrong. If every possible result would count as a success, the hypothesis says nothing.
Where should the reason come from?
From what shoppers actually do and say. Funnel data shows where they leave. Session recordings show what they were doing just before. Post-purchase surveys, support tickets and product reviews tell you what almost stopped a buyer. Usability tests show where people get confused, and a structured review of the page against established usability principles shows where they are likely to.
The reason is the part that makes a result useful after the test ends. If the test wins, you know which shopper problem was worth solving and can look for it elsewhere on the store. If it loses, you know the explanation was wrong, or that the change did not address it.
What does a weak hypothesis look like?
Here is a typical one: "A new product page design will increase conversions." It names no specific change, no metric beyond a vague word and no reason. Whatever the test shows, nobody learns why.
A stronger version, written as an illustration: "If the size guide opens as an overlay on the product page instead of a separate page, then more mobile visitors add the product to their cart, because survey answers and recordings show shoppers leaving the product page to check sizing and not coming back." Each part can now be checked, and the evidence is named.
Which metric should a hypothesis name?
The metric closest to the change that still connects to revenue. A change to the product page usually aims at add to cart. A change to shipping information in checkout aims at completed orders. Metrics far from the change need much more traffic, because many other things influence them along the way.
Add guardrail metrics next to the primary one. Revenue per visitor, average order value and page speed are common choices. A variant that lifts add to cart while cutting order value is not a winner, and naming the guardrails in advance keeps that discussion short.
Can one hypothesis cover several changes at once?
Yes, if the changes serve one idea. Moving the delivery date, the returns policy and the payment options closer to the add to cart button can all test the same reason: shoppers leave to find out what happens after they buy. If that version wins, you know the idea was right, though not which of the three changes did the work.
To measure each change separately, you need either several tests in a row or a multivariate test, which splits traffic across every combination. Each extra combination raises the traffic a clear answer needs, so most stores get further with one idea per test.
What belongs in the test plan next to the hypothesis?
- The pages and the audience included.
- The primary metric and the guardrail metrics.
- The minimum detectable effect, the significance level and the power.
- The sample size and the planned run time in full weeks, as explained in How long should an A/B test run?
- The decision for each outcome: a clear win, a clear loss and an unclear result.
Writing the decisions down before launch matters most. Once a result is on the screen, it is easy to find a reason to keep a change you already liked.
How do you decide which hypothesis to test first?
Score each one on the strength of its evidence, the size of the expected effect and the effort to build and test it. Published scoring models such as ICE and PIE follow this idea with different labels. The model matters less than applying one consistently, and less than being honest about evidence: an idea backed by recordings and survey answers ranks above an idea backed by a competitor doing it.
Traffic also sets limits. A strong hypothesis on a page with little traffic may be better handled without a test.
What happens to a hypothesis after the test?
It goes into a learning log with its result, its numbers and the decision taken, whatever the outcome. Over months, that log becomes a record of what your customers respond to. Volition keeps one for every brand it works with, and the log stays with the brand.