What is a test hypothesis?
A test hypothesis is a focused statement of conjecture. In actuality, the statement may be true or untrue. Through testing and analysis, the objective of hypothesis testing is to prove or disprove the statement and be in a position to indicate the reliability of conclusion.
How to form a test hypothesis
A good test hypothesis will be written as a statement or question that specifies:
- The dependent variable(s): who or what you expect to be affected
- The independent variable(s): who or what you predict will affect the dependent variable
- What you predict the effect will be
Developing a test hypothesis statement
A website owner thinks that by making changes to the images of a product range (hats in this example) they can increase revenue. Before they go ahead and conduct that change, they decide to conduct AB testing and test the hypotheses that:
- H0: “Showing a model shot rather than an item shot on the hats category, will result
with an increase in online revenue”
- H1: “Showing a model shot rather than an item shot on the hats category, will result
in a reduction in online revenue”.
- H0: Null hypothesis: A null hypothesis is a hypothesis that says there is no
association or statistical significance between the two variables in the statement.
- H1: Alternative hypothesis: It is simply the opposite of the null hypothesis; the
alternative hypothesis shows that observations are the result of a real effect.
In the example above, the dependent variable are the visitors to the hats category webpage,
the independent variables are the images shown to the visitors, and finally, the effect
prediction is the increase in online hat revenue.
Following a test, the expectation is that either:
H0 is accepted & H1 is rejected
or
H1 is rejected & H0 is rejected.
Test hypothesis concepts and key terms
Below is the main terminology to bear in mind when executing a test hypothesis:
- Type 1 error: False Negative. Is the rejection of the H0 null hypothesis,
when in actuality the hypothesis was true.
- Type 2 error: False Positive. Is the acceptance of H0, when in actuality the
hypothesis is false.
Controlling the risks of these errors, we would usually test to a Level of statistical
significance: This is a calculation that determines the level of confidence that can be made
about test obersvation. 95% statistical significance level is the recommendation and means
there is only a 5% probability that the result is due to chance and the risk of type 1 or 2
errors mitigated.
- One-tailed test: When the given statistical hypothesis is either greater than
or less than a certain value, but not both. That is, looks for an increase or
decrease in the parameter.
- Two-tailed test: When the given statistics hypothesis assumes a less than or
greater than value. That is, looks for any change in the parameter (which can be
any change- increase or decrease).
In conclusion...
Not every test hypothesis is always possible to prove as true, as many times, a test
hypothesis can reject the alternative hypothesis and it’s OK. In the end, conducting a test
hypothesis is a very important part of your conversion rate optimisation strategy as it
removes the guessing element of your decision-making and allows you to make statistical
decisions based on scientific evidence..
