Articles in this section

Troubleshoot: Issues with Headline Tests

  • Updated
When a new visitor arrives to your homepage with an in-progress test, our Headline Testing javascript code attempts to match the headline the visitor sees to the headline that your editor initiated the test on. These headlines are matched on three data points in your site's HTML: the URL, the headline text, and the headline HTML sub-location. If these three points do not match between the version of the page your visitors load and the version of the page where the test was initiated, this counts as a failed trial and a test variant will not be served. 
 
When a Chartbeat user at your Org goes to initiate a new headline test on your site, our code collects these three data points from the link element where the user initiates their test: 
  • article URL:  the precise value of the <a> tag href attribute, which is the target article URL in your homepage code
  • headline text: the precise headline text from your CMS, the value which will become variant A in your test
  • headline sub-location: the headline's sub-location, or truncated XPath which demonstrates the relationship of the headline text element to the parent <a> tag in your HTML

Example: Below, you can see the href (URL) and headline text values our code will collect if a test is initiated on this headline. The headline sub-location (aka truncated XPath), would be span[1] since the headline text is nested in the first <span> tag within the parent <a> tag:

15m2.png

 

Issue #1: Headline Tests ending after 15 minutes with "No winner found"

In some cases, headline tests may complete after 15 minutes with no winner found:

15m.png

For tests that stop at the 15-minute mark with no winner found, we can assume there is a mismatch between the source code where your test was initiated and the source that code 95% of new visitors loaded. This difference lies in one or more of the three data points mentioned above (URL, headline text, headline sub-location).  Headline Tests will end at the 15-minute mark if over 95% of new visitor sessions result in failed trials.

 

Solution:

When your editors are initiating new tests, they should ensure that they are viewing a version of the homepage where the headline HTML will match precisely the HTML that 95% or more of your visitors will load. They may be viewing a slightly modified version of the page, for example, an editor-specific version that requires a login.   

 

Issue #2: Headline Tests initiated on my desktop homepage do not display for my mobile audience

Similar to the first issue mentioned above, this indicates that your mobile homepage HTML does not match your desktop homepage HTML related to the three relevant data points for Headline Test matching: headline URL, text, and sub-location.

 

Solution:

Have your developer team review differences between headline URLs, text, and sub-location between your mobile and desktop homepage templates to ensure precise matching for users on both device types.

Related articles