There are two conditions where a headline wins and an experiment ends -- hard convergence and soft convergence.
Hard convergence is the usual method where there is 95% confidence that one headline is more successful at attracting clicks and attention.
Soft convergence handles the case of two roughly equal headlines. If twenty minutes have passed and two headlines are performing about the same, but the one that’s behind is unlikely to overtake the one in first place and then beat it by more than 25%, we end the experiment. We have this alternate method in place to deliver test results more quickly, choosing the winner by a small margin rather than letting the test drag on.