Hypothesis Testing I: Overview
High frequency and low variance can be accidental. We want to determine whether the co-occurrence is random or whether it occurs more often than chance.
This is a classical problem in Statistics called Hypothesis Testing.
We formulate a null hypothesis H0 (no association beyond chance) and calculate the probability that a collocation would occur if H0 were true, and then reject H0 if p is too low and retain H0 as possible, otherwise.