Association Between Media Dose, Ad Tagging, and Changes in Web Traffic for a National Tobacco Education Campaign: A Market-Level Longitudinal Study

Review of Evaluation Metrics Used in Digital and Traditional Tobacco Control Campaigns

BACKGROUND

Mass media campaigns for public health are increasingly using digital media platforms, such as web-based advertising and social media; however, there is a lack of evidence on how to best use these digital platforms for public health campaigns. To generate this evidence, appropriate campaign evaluations are needed, but with the proliferation of digital media-related metrics, there is no clear consensus on which evaluation metrics should be used. Public health campaigns are diverse in nature, so to facilitate analysis, this review has selected tobacco control campaigns as the scope of the study.

OBJECTIVE

This literature review aimed to examine how tobacco control campaigns that use traditional and digital media platforms have been evaluated.

METHODS

Medicine and science databases (Medical Literature Analysis and Retrieval System Online [MEDLINE], EMBASE, PsycINFO, Cumulative Index to Nursing and Allied Health Literature [CINAHL], and Scopus), and a marketing case study database (World Advertising Research Center) were searched for articles published between 2013 and 2018. Two authors established the eligibility criteria and reviewed articles for inclusion. Individual campaigns were identified from the articles, and information on campaigns and their evaluations were supplemented with searches on Google, Google Scholar, and social media platforms. Data about campaign evaluations were tabulated and mapped to a conceptual framework.

RESULTS

In total, 17 campaigns were included in this review, with evaluations reported on by 51 articles, 17 marketing reports, and 4 grey literature reports. Most campaigns were from English-speaking countries, with behavioral change as the primary objective. In the process evaluations, a wide range of metrics were used to assess the reach of digital campaign activities, making comparison between campaigns difficult. Every campaign in the review, except one, reported some type of engagement impact measure, with website visits being the most commonly reported metric (11 of the 17 campaigns). Other commonly reported evaluation measures identified in this review include engagement on social media, changes in attitudes, and number of people contacting smoking cessation services. Of note, only 7 of the 17 campaigns attempted to measure media platform attribution, for example, by asking participants where they recalled seeing the campaign or using unique website tracking codes for ads on different media platforms.

CONCLUSIONS

One of the key findings of this review is the numerous and diverse range of measures and metrics used in tobacco control campaign evaluations. To address this issue, we propose principles to guide the selection of digital media-related metrics for campaign evaluations, and also outline a conceptual framework to provide a coherent organization to the diverse range of metrics. Future research is needed to specifically investigate whether engagement metrics are associated with desired campaign outcomes, to determine whether reporting of engagement metrics is meaningful in campaign evaluations.

Generating Engagement on the Make Healthy Normal Campaign Facebook Page: Analysis of Facebook Analytics

Background

Facebook is increasingly being used as part of mass media campaigns in public health, including the Make Healthy Normal (MHN) campaign in New South Wales, Australia. Therefore, it is important to understand what role Facebook can play in mass media campaigns and how best to use it to augment or amplify campaign effects. However, few studies have explored this.

Objective

This study aimed to investigate usage of and engagement with the MHN Facebook page and to identify influential factors in driving engagement with the page.

Methods

We examined both post-level and page-level analytic data from Facebook from the campaign’s launch in June 2015 to September 2017. For post-level data, we conducted a series of negative binomial regressions with four different outcome measures (likes, shares, comments, post consumers), including some characteristics of Facebook posts as predictors. We also conducted time series analyses to examine associations between page-level outcomes (new page likes or “fans” and number of engaged users) and different measures of exposure to the page (number of unique users reached and total count of impressions) and to television advertising.

Results

Of the 392 posts reviewed, 20.7% (n=81) received a paid boost and 58.9% (n=231) were photo posts. We found that posts that received a paid boost reached significantly more users and subsequently received significantly more engagement than organic (unpaid) posts (P<.001). After adjusting for reach, we found the effect of being paid was incremental for all outcome measures for photos and links, but not videos. There were also associations between day of the week and time of post and engagement, with Mondays generally receiving less engagement and posts on a Friday and those made between 8 AM and 5 PM receiving more. At the page level, our time series analyses found that organic impressions predicted a higher number of new fans and engaged users, compared to paid impressions, especially for women. We also found no association between television advertising and engagement with the Facebook page.

Conclusions

Our study shows that paying for posts is important for increasing their reach, but that page administrators should look to maximize organic reach because it is associated with significantly higher engagement. Once reach is accounted for, video posts do not benefit from being paid, unlike the other post types. This suggests that page administrators should carefully consider how they use videos as part of a Facebook campaign. Additionally, the lack of association between television advertising and engagement suggests that future campaigns consider how best to link different channels to amplify effects. These results highlight the need for ongoing evaluation of Facebook pages if administrators are to maximize engagement.

Television Advertising and Health Insurance Marketplace Consumer Engagement in Kentucky: A Natural Experiment

Background

Reductions in health insurance enrollment outreach could have negative effects on the individual health insurance market. Specifically, consumers may not be informed about the availability of coverage, and if some healthier consumers fail to enroll, there could be a worse risk pool for insurers. Kentucky created its own Marketplace, known as kynect, and adopted Medicaid expansion under the Affordable Care Act, which yielded the largest decline in adult uninsured rate in the United States from 2013 to 2016. The state sponsored an award-winning media campaign, yet after the election of a new governor in 2015, it declined to renew the television advertising contract for kynect and canceled all pending television ads with over a month remaining in the 2016 open enrollment period.

Objective

The objective of this study is to examine the stark variation in television advertising across multiple open enrollment periods in Kentucky and use this variation to estimate the dose-response effect of state-sponsored television advertising on consumer engagement with the Marketplace. In addition, we assess to what extent private insurers can potentially help fill the void when governments reduce or eliminate television advertising.

Methods

We obtained television advertising (Kantar Media/Campaign Media Analysis Group) and Marketplace data (Kentucky Health Benefit Exchange) for the period of October 1, 2013, through January 31, 2016, for Kentucky. Advertising data at the spot level were collapsed to state-week counts by sponsor type. Similarly, a state-week series of Marketplace engagement and enrollment measures were derived from state reports to Centers for Medicare and Medicaid Services. We used linear regression models to estimate associations between health insurance television advertising volume and measures of information-seeking (calls to call center; page views, visits, and unique visitors to the website) and enrollment (Web-based and total applications, Marketplace enrollment).

Results

We found significant dose-response effects of weekly state-sponsored television advertising volume during open enrollment on information-seeking behavior (marginal effects of an additional ad airing per week for website page views: 7973, visits: 390, and unique visitors: 388) and enrollment activity (applications, Web-based: 61 and total: 56).

Conclusions

State-sponsored television advertising was associated with nearly 40% of unique visitors and Web-based applications. Insurance company television advertising was not a significant driver of engagement, an important consideration if cuts to government-sponsored advertising persist.

Impact of the Tips From Former Smokers Campaign on Population-Level Smoking Cessation, 2012-2015

This study provides estimates of the long-term cumulative impact of the Centers for Disease Control and Prevention’s national tobacco education campaign, Tips From Former Smokers (Tips), on population-level smoking cessation. We used recently published estimates of the association between increased Tips campaign media doses and quit attempts to calculate campaign-attributable population sustained (6-month) quits during 2012–2015. Tips led to approximately 522,000 sustained quits during 2012–2015. These findings indicate that the Tips campaign’s comprehensive approach to combining evidence-based messages with the promotion of cessation resources was successful in achieving substantial long-term cigarette cessation at the population level over multiple years.

Evaluation of a federally funded mass media campaign and smoking cessation in pregnant women: a population-based study in three states

OBJECTIVES

In 2012, theCenters for Disease Control and Prevention initiated a national anti-smoking campaign, Tips from Former Smokers (Tips). As a result of the campaign, quit attempts among smokers increased in the general population by 3.7 percentage points. In the current study, we assessed the effects of Tips on smoking cessation in pregnant women.

METHODS

We used 2009-2013 certificates of live births in three US states: Indiana, Kentucky and Ohio. Smoking cessation by the third trimester of pregnancy was examined among women who smoked in the 3 months prepregnancy. Campaign exposure was defined as overlap between the airing of Tips 2012 (March 19-June 10) and the prepregnancy and pregnancy periods. Women who delivered before Tips 2012 were not exposed. Adjusted logistic regression was used to determine whether exposure to Tips was independently associated with smoking cessation.

RESULTS

Cessation rates were stable during 2009-2011 but increased at the time Tips 2012 aired and remained elevated. Overall, 32.9% of unexposed and 34.7% of exposed smokers quit by the third trimester (p<0.001). Exposure to Tips 2012 was associated with increased cessation (adjusted OR: 1.07, 95% CI 1.05 to 1.10).

CONCLUSIONS

Exposure to a national anti-smoking campaign for a general audience was associated with smoking cessation in pregnant women.

Does Digital Video Advertising Increase Population-Level Reach of Multimedia Campaigns? Evidence From the 2013 Tips From Former Smokers Campaign

Background

Federal and state public health agencies in the United States are increasingly using digital advertising and social media to promote messages from broader multimedia campaigns. However, little evidence exists on population-level campaign awareness and relative cost efficiencies of digital advertising in the context of a comprehensive public health education campaign.

Objective

Our objective was to compare the impact of increased doses of digital video and television advertising from the 2013 Tips From Former Smokers (Tips) campaign on overall campaign awareness at the population level. We also compared the relative cost efficiencies across these media platforms.

Methods

We used data from a large national online survey of approximately 15,000 US smokers conducted in 2013 immediately after the conclusion of the 2013 Tips campaign. These data were used to compare the effects of variation in media dose of digital video and television advertising on population-level awareness of the Tips campaign. We implemented higher doses of digital video among selected media markets and randomly selected other markets to receive similar higher doses of television ads. Multivariate logistic regressions estimated the odds of overall campaign awareness via digital or television format as a function of higher-dose media in each market area. All statistical tests used the .05 threshold for statistical significance and the .10 level for marginal nonsignificance. We used adjusted advertising costs for the additional doses of digital and television advertising to compare the cost efficiencies of digital and television advertising on the basis of costs per percentage point of population awareness generated.

Results

Higher-dose digital video advertising was associated with 94% increased odds of awareness of any ad online relative to standard-dose markets (P<.001). Higher-dose digital advertising was associated with a marginally nonsignificant increase (46%) in overall campaign awareness regardless of media format (P=.09). Higher-dose television advertising was associated with 81% increased odds of overall ad awareness regardless of media format (P<.001). Increased doses of television advertising were also associated with significantly higher odds of awareness of any ad on television (P<.001) and online (P=.04). The adjusted cost of each additional percentage point of population-level reach generated by higher doses of advertising was approximately US $440,000 for digital advertising and US $1 million for television advertising.

Conclusions

Television advertising generated relatively higher levels of overall campaign awareness. However, digital video was relatively more cost efficient for generating awareness. These results suggest that digital video may be used as a cost-efficient complement to traditional advertising modes (eg, television), but digital video should not replace television given the relatively smaller audience size of digital video viewers.