Report Types
Summary Report (CSV): For comparisons across tests.
Variant Breakdown Report (CSV): For comparisons across variants, regardless of test.
Report Metrics
Status
- cancelled: The test was stopped manually before a winner was found.
- winner_found: A winning headline has been found and Chartbeat is automatically displaying that winning headline for all visits on your homepage.
- stopped_with_winner: A winning headline has been found but a user has stopped Chartbeat from automatically displaying the winning headline to all users on your homepage. If you've stopped a headline that Chartbeat was auto-playing and want that headline to be permanently played to all readers, you must change the headline directly in your CMS.
- no_winner_found: A winning headline has not been found. This most often occurs because the headline options are too similar to reach 95% statistical confidence that one is meaningfully better than the others.
Views: Only those trials where the reader saw, but did not click, the headline variant.
Clicks: Only those trials where the reader clicked the headline variants but did not engage with the destination page for more than 15 seconds.
Engaged Clicks: Only those trials where the reader clicks the headline variant, then engages with the destination page for least 15 seconds.
Summary Report (CSV) fields:
| test_id |
Unique ID assigned to each headline test. |
| host |
The Chartbeat site where the test was initiated. |
| domain |
The domain name where the test was initiated. |
| path |
The Chartbeat page path where the test was initiated. |
| created_at |
Time when the test was started. |
| winner_found_at |
Time when a winning headline was found. This field will not contain values for tests where no winner was found. |
| stopped_at |
Time when the test was ended, manually or automatically. |
| test_duration_sec |
Test duration in seconds. This field will not contain values for tests that resulted in 'no winner found' and 'canceled' statuses. |
| test_duration_min |
Test duration in minutes. |
| status |
Final status of the completed test. |
| num_variants |
The number of headline variants used in the test. The original headline (variant A) counts as one variant. |
| winning_variant |
Indicates the variant ID of the winning headline (A, B, C...). |
| ctr_increase |
The increase in Click Through Rate comparing the winning headline variant to the original headline (variant A), expressed as a decimal. This field will not contain a value for tests where no winner was found, or where the original headline was found the winner. |
| winning_headline |
Contains the headline text of the winning headline variant. |
| destination_url |
The target URL of the headline where the test was initiated. |
| headlines |
Lists each headline variant, including both the variant ID and the variant headline text. |
| sections |
Chartbeat section tags associated with the destination page URL. |
| created_by |
The email of the Chartbeat user who created the test. |
| ai_assisted_variants |
Variant(s) created with our AI-Assisted Headline tool, if using (A, B, C...). |
Variant Breakdown Report (CSV) fields:
| headline_id |
Unique ID assigned to each headline variant (A, B, C...). |
| is_winner |
TRUE for winning headlines, FALSE for non-winning headlines, empty for tests where no winner was found. |
| headline |
Headline text for each variant. |
| views* |
Pageviews where the variant was displayed, but not clicked. |
| clicks* |
Pageviews where the variant was clicked and the user spent less that 15 seconds of engaged time on the subsequent page (destination URL). |
| engaged_clicks* |
Pageviews where the variant was clicked and the user spent more that 15 seconds of engaged time on the subsequent page (destination URL). |
| ctr_mean_percent |
The Click Through Rate for the headline, expressed as a percentage. |
| type |
If using our AI-Assisted Headline tool, the type of test - "User Generated" or "AI Assisted". |