Introduction

Throughout the COVID-19 pandemic, the Hispanic community in the United States has seen a higher proportion of cases compared to the Non-Hispanic White community [1]. This discrepancy in cases may be explained by rates of members of the Hispanic community working in essential services and in-person jobs, relying on public transportation and living in multigenerational households, all of which increase individuals’ risk of exposure to the virus [2]. In addition, members of the Hispanic community were more likely to experience worse outcomes from COVID-19 due to a higher prevalence of pre-existing health conditions and smaller likelihood of having health insurance and access to healthcare [2, 3]. This is reflected in data from the CDC that found that, as of April 2022, Hispanic people were 2.3 times as likely to be hospitalized and 1.8 times as likely to die from COVID-19 than white people [4].

Vaccine hesitancy among Hispanic adults was influenced by concerns of short- and long-term side effects, beliefs that the Hispanic population should have had greater representation in vaccine trials, distrust in the medical community and misinformation [5,6,7]. Meanwhile, studies found that Hispanic people who were keen to get vaccinated faced several barriers, such as not being able to take time off of work, a lack of transportation, language barriers, fears of disclosing citizenship status and fears of having to pay out-of-pocket for the vaccine [8]. They also dealt with vaccine registration or informational websites that were not translated into Spanish or were poorly translated. [9, 10]

Vaccine hesitancy among the U.S. population, and the Hispanic population specifically, is partially attributable to the “infodemic” of misinformation that resulted alongside the COVID-19 pandemic and response [11]. During the COVID-19 pandemic, misinformation and disinformation soared online, most prolifically on social media channels. This coincided with a time when many people were turning towards social media for information about health and the pandemic [12]. Social media plays an important role in the public perceptions of risk. [13] As such, the prevalence of misinformation subsequently increased negative attitudes towards the COVID-19 pandemic and, later, the COVID-19 vaccines [14, 15]. Efforts online to counter misinformation were smaller in scale and outweighed by the deluge of misinformation. [16] This is especially true within the Hispanic communities’ and Spanish language conversations online. A March 2021 study found that 49% of Hispanic respondents thought that COVID-19 misinformation was a very serious problem, and 20% said they had wrong or harmful information about the COVID-19 vaccine shared directly with them, most commonly through Facebook [17]. The impact of this misinformation is exacerbated by social media companies’ inability to effectively identify and flag or remove Spanish misinformation, allowing it to spread for longer within online conversations. [18, 19]

Over time, vaccine hesitancy among the Hispanic population decreased, as efforts were made to address vaccine misinformation, provide culturally appropriate vaccine education and messaging and provide more accessible vaccine clinic locations [20]. A poll from May 2021 found that Hispanic people were the group that was most eager to get vaccinated at this time [21, 22]. As of July 2022, 88.2% of Hispanic adults have received at least one dose of a COVID-19 vaccine, and 85.2% are fully vaccinated, meaning they have received all doses of the primary series of the COVID-19 vaccine [23]. For the upward trend in vaccinations to continue, efforts to promote COVID-19 booster vaccinations and generally positive sentiment towards vaccinations must continue.

In late 2020, the Centers for Disease Control and Prevention (CDC) developed Partnering for Vaccine Equity (P4VE), a “program to support national, state, local and community-level partners, who prioritized equity in vaccination access and uptake for those groups that experience disparities in immunization.” [24] The CDC Foundation was awarded funding to develop a dynamic program designed to address health disparities related to COVID-19 faced by black, indigenous, people of color (BIPOC). CDC Foundation identified the need for a comprehensive approach to counter the COVID-19 infodemic and build vaccine confidence in priority populations by countering misinformation online. As one part of P4VE, CDC Foundation partnered with the Public Good Projects (PGP), Hispanic Communications Network (HCN), and World Voices Media (WVM) to specifically address COVID-19 misinformation and vaccine hesitancy within the Hispanic community. These three organizations worked collaboratively to leverage their expertise in mass media and social media, media monitoring, working with community-based organizations (CBOs) and working with the Hispanic community. PGP is a public health communications nonprofit, HCN is a for-profit communications service that serves Hispanic people in the United States and WVM is a social impact communications nonprofit.

The purpose of this paper is to present the results of a multi-faceted campaign to increase COVID-19 vaccination rates among Hispanic adults that was grounded in and responsive to trending and emerging COVID-19 misinformation. While this work was part of the broader P4VE work spearheaded by CDC and CDC Foundation, this paper focuses on the work of the three partners, PGP, HCN and WVM, within the larger P4VE context. This paper fills a gap in the literature about the design, implementation and reach of such a campaign geared towards Hispanic adults for any topic, as well as more specifically about COVID-19. The results of this paper could be used by practitioners creating their programs for COVID-19 or other health issues for which misinformation proliferates and creates a barrier to care.

Methods

Theory of Change

The theory of change model for this project is shown in Fig. 1. The theory posits:

(Step 1) If we identify misinformation, disinformation and negative attitudes about COVID-19 and COVID-19 vaccinations using large-scale media monitoring; then,

(Step 2a) We can use these findings to create assets, talking points, newsletters and trainings to share with community-based organizations (CBOs) as they fight to address misinformation, and CBOs can relay back to us the misinformation they are hearing; and,

(Step 2b) We can use these findings to create mass media messaging including radio public service announcements (PSAs), op-eds, a radio/satellite media tour and website copy; and,

(Step 2c) We can use these findings to update the prompts we give to influencers, celebrities and volunteers so they can modify and incorporate misinformation counter-messaging into their posts, and we can monitor posts’ comments to identify sentiment and misinformation among followers; then,

(Step 3) The public will be exposed to science-backed prebunk (preemptive debunking) and debunk messaging addressing misinformation, disinformation and negative attitudes related to COVID-19 and COVID-19, which will decrease the public’s belief in COVID-19 misinformation, and lead to increased confidence in and uptake of the COVID-19 vaccine.

Fig. 1
figure 1

Theory of change

Full size image

This theory of change model is rooted in the hypothesis that to increase vaccine confidence and uptake, we needed to be constantly responding to the trending and emerging misinformation, disinformation and negative attitudes towards COVID-19, rather than approaching the campaign with a predetermined set of messages we wanted to communicate. We hypothesized that such a campaign model would deter the continued spread of misinformation and simultaneously spread correct and positive information about vaccination.

Campaign Creation

Formative work for this project began in August 2021, with key program components launching in September 2021 and running through April 29, 2022.

Step 1: Creating and Using Our misInformation Media Monitoring Systems

PGP has monitored publicly available English-language vaccine misinformation since 2019 [25]. For this project, PGP and HCN adapted these methods to monitor COVID-19 and vaccine misinformation in English and Spanish. Data was collected into a media monitoring software platform, which aggregates information from various sources, including social media platforms, news sites and video sites, among others. To collect data, we created two complex Boolean search queries in English and Spanish, each consisting of hundreds of keywords, hashtags and phrases. Any mention of these terms was collected into the media monitoring system. PGP regularly revisited queries for refinements and updates throughout the project period. Each week, PGP and HCN collaboratively reviewed top posts in the two queries to identify misinformation and disinformation that was trending or emerging in English and Spanish-language queries. Mis- and disinformation were reviewed to identify the extent of their spread and whether they were trending in both English and Spanish queries.

Step 2a: Supporting Community-Based Organizations

The misinformation identified by our media monitoring (Step 1) was used to support community-based organizations in multiple ways as they worked to prebunk and debunk COVID-19 misinformation and increase COVID-19 vaccination rates in Hispanic communities.

PGP created and sent weekly newsletter emails in both Spanish and English to roughly 380 Hispanic-serving CBOs from across the United States. All Spanish newsletters and insights were transcreated by HCN, meaning they were both translated from English to Spanish and adapted for cultural relevance. Each email contained the misinformation insights identified in our media monitoring, as well as counter-messaging talking points they could use if they heard community members repeat misinformation. Talking points were also uploaded as a stand-alone resource on the Vaccine Resource Hub, a website created for this project to specifically host resources for CBOs [26]. On the Vaccine Resource Hub, misinformation talking points were available for download, including as a single message bank that was updated throughout the year.

Many of our CBOs communicate with their constituents through social media. As such, PGP created social media assets every week and videos and/or carousel graphics every month, that included counter-messaging against the week’s trending or emerging misinformation. These assets were disseminated to CBOs through our weekly newsletter and the Vaccine Resource Hub. On the Vaccine Resource Hub, assets on similar themes were grouped into content toolkits.

HCN also conducted digital media activities, including creating website features for La Red Hispana, HCN’s public-facing, Spanish language, media outlet and distribution house [27]. They also created social media posts on the HCN social media pages.

PGP conducted virtual trainings for CBOs on COVID-19 and how to respond to misinformation. Trainings were promoted through the weekly newsletter to CBOs, the P4VE Learning Community, and other newsletters. Training materials were made accessible on the Vaccine Resource Hub. Training topics focused primarily on strategies and best practices in countering misinformation on social media specifically targeting Hispanic communities, with names including How to Combat and Respond to Anti-Vaccine Attacks on Social Media; Monitoring Spanish Language Misinformation; and An Overview of the White House COVID-19 Action Plan Vaccine Mandates. During these trainings, CBOs had the opportunity to communicate to PGP any misinformation they were hearing from their community. Debunk messaging about this misinformation was then incorporated into future content and resources.

Additionally, WVM developed and hosted Spanish-language virtual panel discussions. WVM partnered with 20 CBOs and four consulates (Mexico, Guatemala, Colombia and Honduras) to combat misinformation and disinformation surrounding COVID-19 and vaccine confidence. At each virtual event, participants received up-to-date vaccination information, learned outreach strategies for promotores de salud (Community Health Workers) and received advice on leveraging trusted messengers. WVM provided event copy, event resources and virtual events in Spanish and English. Virtual events were promoted through WVM’s social media channels, newsletters and the P4VE Learning Community. Event recordings were shared through the WVM’s Spanish and English monthly newsletters, which was sent to a network of CBOs and made accessible on the Vaccine Resource Hub. During these trainings, CBOs had the opportunity to communicate to WVM any misinformation they were hearing from their community. Debunk messaging about this misinformation was then incorporated into future content and resources.

Step 2b: Mass Media

HCN created mass media messages that both prebunked and debunked misinformation based on the findings of our media monitoring (Step 1). These included radio public service announcements (PSAs), op-eds and a radio/satellite media tour.

Step 2c: Influencers, Hispanic Celebrities, and Digital Volunteers

The misinformation identified by our media monitoring was also used in the creation of prompts sent to social media influencers recruited by PGP to our campaign. As part of this project, PGP created a fully branded campaign called El Beacon, a nationwide network of Hispanic nano-, micro-, and mid-tier social media influencers. PGP sourced and vetted influencers to meet our inclusion criteria: resided in the U.S., identified as Hispanic, and vaccinated for COVID-19. Outreach to influencers started in September 2021 and continued throughout the campaign, onboarding new influencers on a rolling basis. Influencers who joined the campaign were provided with a series of prompts on what to include in their posts’ captions. These prompts were directly informed by the findings of our media monitoring (Step 1) and included debunking of common misinformation circulating online and positive messages promoting reasons to get vaccinated. These prompts were updated throughout the year to ensure they were responsive to the present, trending or emerging misinformation. Influencers were also encouraged to share their own vaccination story in their post’s caption. PGP also monitored influencers’ posts for misinformation in the comments section and provided counter-messaging or resources to the influencer to address the misinformation. PGP also conducted sentiment analysis on post comments to further assess how followers were reacting to posts. [28]

Volunteers were recruited through paid digital advertisements using the El Beacon social media channels (Instagram and Facebook) and an El Beacon website in Spanish (elbeacon.org/es) and English (elbeacon.org). Volunteers received facts about the COVID-19 vaccines and alerts on misinformation, which were informed by our media monitoring. Volunteers used this information to post pro-vaccine messages to their social media profiles, including information that directly addressed or debunked the misinformation currently trending. Unlike paid influencers, we do not track metrics on our volunteers’ posts, as there is no requirement that they report their metrics back to us.

Additionally, HCN recruited Hispanic celebrities- a television host and an award-winning author and motivational speaker- to post about COVID-19 vaccination on social media and create a public service announcement that was broadcast by HCN media affiliates. Celebrities were provided a brief with information about El Beacon, guidelines for content development, and inoculation prebunk and debunk messages. Similar to other paid influencers, HCN asked them to share their personal story explaining why it was important for them to get vaccinated and their experience getting the vaccine. Celebrities also joined El Beacon and invited their communities to do the same.

Metrics and Analysis Methods

Media Monitoring Metrics

Media monitoring metrics include the total number of posts and the number and types of themes into which these posts were categorized. Many posts, but not all, are automatically categorized into themes. Themes are auto-created by the media monitoring system through machine learning. Media monitoring metrics cover the Year 1 of the campaign, from September 1, 2021 to April 29, 2022.

Digital Metrics

Digital metrics about our newsletters, resource downloads, digital volunteers, El Beacon influencers, Hispanic celebrities, mass media and webinars were pulled from a variety of tracking platforms. These platforms included Google Analytics, Hubspot, Grin and Zoom. Social media metrics include total impressions (number of people who saw a post), potential reach (total followers of an account), likes, comments and engagement rate (ratio of engagements to potential reach). Newsletter metrics include the number of emails sent, the open rate, the click-through rate (number of people who clicked on a link within the newsletter) and average subscribers. All digital metrics were analyzed across Year 1 of the campaign, from September 1, 2021 to April 29, 2022.

Sentiment Analysis

For qualitative analysis of social media comments, we took a simple random sample of 10% of influencer posts (n = 50). This sample was limited to posts that were Instagram feed posts, Facebook posts and Twitter posts. Instagram Stories were excluded from the analysis as they disappear from a profile after 24 h, and comments cannot later be pulled. Comments on the sampled posts were pulled from Instagram using a proprietary Instagram comment scraper. Comments from influencers responding to their followers were removed from the sample; all comments from the El Beacon profile, which PGP runs, were also removed from the analysis. Comments were then coded using an inductive coding scheme: positive- related to COVID-19, positive- unrelated to COVID-19, positive- emojis only, negative- related to COVID19, negative- unrelated to COVID-19, and negative- emojis only.

While this methodology has its limitations due to the inability to track digital volunteer metrics or conduct sentiment analysis on Instagram Stories, we believe it is an important addition to the current literature as it fills a gap about a COVID-19 vaccination campaign for Hispanic adults, and insights into a new public health campaign model.

Results

Step 1: Media Monitoring Metrics

Our English profile captured 212,700,000 mentions of COVID-19 vaccination during Year 1. Our Spanish profile captured 35,000,000 mentions of COVID-19 vaccination during Year 1. Across both profiles, we saw relatively consistent mentions captured from September 2021- January 2022. However, we saw a substantial decrease in the number of mentions between February 2022–April 2022. See Table 1 for complete media monitoring metrics.

Table 1 Media Monitoring Metrics, September 1, 2021–April 2, 2022
Full size table

We saw an overlap in the top themes across the English and Spanish profiles. For both profiles, COVID-19, negative health impacts, the pharmaceutical industry and health authorities were the top four themes.

Digital Metrics (STEPS 2a-2c)

Step 2a: Supporting Community-Based Organizations

PGP sent 70 newsletters (35 in English, 35 in Spanish) to CBOs. They were sent to an average of 1539 people. The email open rate was 36.9% and the click-through rate to the Vaccine Resource Hub was 4.7%. These are both over the industry average of 25.0% and 2.8%, respectively. [29] These emails contained a total of 206 misinformation insights (103 in English, 103 in Spanish) and 206 talking points (103 in English, 103 in Spanish) on how to rebut the identified misinformation. Documents containing the weekly talking points were downloaded a total of 127 times from our Vaccine Resource Hub.

PGP developed a total of 344 social media assets which received 568 downloads from the Vaccine Resource Hub. Assets on similar themes were grouped into 13 toolkits which were downloaded 127 times from the Vaccine Resource Hub.

HCN developed one feature page on the La Red Hispana website and 10 digital display banners to direct people to the feature page. These had an estimated total reach of 248,500 people. HCN developed 24 social media posts, including 6 videos, 18 static posts, and 5 animated graphics. These had an estimated reach of 647,987 people, 658,421 total impressions, and 82,100 engagements. The videos received a total of 37,410 views.

PGP provided 15 trainings to CBOs (See Campaign Creation for details) with a total attendance of 1553 people. WVM provided 5 virtual events with a total attendance of 659 people.

Step 2b: Mass Media

HCN developed six Spanish-language radio PSAs which had 91 placements across 50 markets. The radio PSAs had an estimated reach of 26,874,100 people. The two op-eds had 35 placements in 20 markets and reached an estimated 2,911,315 people [30, 31]. The radio/satellite media tour had 8 placements in 40 markets and reached an estimated 22,428 people.

Step 2c: Influencers, Hispanic Celebrities, and Digital Volunteers

PGP recruited 496 social media influencers for El Beacon. Influencers included 141 nano-influencers (1,000–10,000 followers), 268 micro-influencers (10,001–50,000 followers) and 87 mid-tier influencers (50,001–500,000 followers) [32]. Collectively, these influencers posted a total of 1,000 posts. These included 524 Instagram feed posts, 468 Instagram story posts, 4 Facebook posts, 2 TikTok videos and 2 Twitter posts. The feed posts collectively garnered 16,893 comments and 387,332 likes. They had a total potential reach of 37,193,294 people and 3,601,995 total potential impressions. Posts had an average engagement rate of 0.016. Of these, the posts had an actual reach of 432,771 Hispanic adults and garnered 1,498,429 impressions from Hispanic adults. Each Hispanic adult who saw an El Beacon influencer’s post saw it approximately 3.46 times. PGP recruited 1,034 digital volunteers.

HCN recruited two Hispanic celebrities who posted a total of six posts. These posts reached a total of 4,049,788 Hispanic adults and garnered 96,377 engagements.

We conducted sentiment analysis on 10% (n = 50) of El Beacon influencers’ feed posts from Facebook, Instagram, and Twitter. These posts collectively had 1,090 comments. Sentiment analysis found that 97.52% of comments (n = 1,063) on posts were positive. Less than 1% of comments (n = 9) expressed negative sentiment. The remaining 1.7% of comments (n = 18) were either neutral in sentiment or spam.

Discussion

What We Did

For this project, PGP, HCN and WVM, in partnership with CDC Foundation, created a multifaceted, dynamic campaign that is based in, informed by and responsive to online COVID-19 misinformation. Our project worked to address the health disparities faced by the Hispanic community throughout the COVID-19 pandemic. We did this by using fact-based messaging, delivered by trusted voices such as Hispanic social media influencers and CBOs, to counter COVID-19 misinformation and increase the community’s attitudes towards, and uptake of, the COVID-19 vaccine. Critically, we based all of our activities and messaging in English and Spanish-language misinformation that was trending or emerging within the Hispanic community’s online conversations and incorporated fact-based counter-messaging into every aspect of the project. This included providing counter-messaging talking points for: newsletters, webinars/ trainings and premade social media assets for CBOs; radio PSAs; and in the prompts for over 1500 social media influencers, celebrities and digital volunteers. While we cannot directly attribute our work to closing the vaccination disparities, we have seen disparities in COVID-19 cases, hospitalizations and vaccinations in the Hispanic community closing.

Why It Matters

Throughout the COVID-19 pandemic, governments, nonprofits and healthcare providers have launched innovative campaigns aimed at reducing transmission and/or increasing vaccination rates for many racial, ethnic and age groups [33,34,35,36]. These campaigns have used social media, CBOs and mass media to address health disparities and increase health equity. However, our project fills a gap in the literature and conversation in several ways. First, our project was a cross-sector collaboration between public health, media monitoring and media experts who worked in tandem to create and execute each aspect of our campaign. By engaging partners with different expertise, both in topics and in cultural expertise of the Hispanic community, we were able to ensure our project was not only comprehensive, but also culturally relevant and linguistically tailored to the Hispanic community. Future projects of this kind should identify ways to establish cross-sector collaborations or break down silos to ensure experts from different fields can share in messaging and resource creation.

Second, our project created and followed a new theory of change model for integrating misinformation into a public health campaign. The model we created posited that we needed to ground our campaign in debunking and prebunking misinformation to increase public support for, and uptake of, COVID-19 vaccinations. Instead of mentioning misinformation periodically or as it was feasible, every aspect of our campaign was based in addressing the misinformation trending and emerging within Hispanic conversations online by using large-scale media monitoring. We found that our new model was feasible. Our media monitoring identified nearly 250,000,000 mentions of COVID-19 vaccination in a seven-month period, including 35,000,000 mentions in just our Spanish dashboard. We then parsed these mentions for misinformation and disinformation, which were disseminated through Steps 2a–c of our model (Fig. 1). This increased our likelihood of reaching people where they are- whether it be online, listening to the radio, or at in-person community events with a CBO- with fact-based misinformation counter-messaging. There is a gap in the literature about how to structure a public health campaign that must address persistent and evolving misinformation. Our project highlights effective techniques for rapidly identifying and incorporating misinformation counter-messaging into messaging and content, which could be replicated for health topics rife with misinformation, like childhood vaccinations.

We have argued for the need to track misinformation the way that we track all diseases.37 The COVID-19 pandemic has highlighted this call as being more urgent and more pertinent than ever. As we track the continued impact of misinformation, we see a greater need to ensure that communities are equipped to be proactive before a piece of misinformation becomes pervasive and harm is caused. Instead, communities must have the tools- including science-backed prebunk and debunk messaging in the form of talking points, social media assets and mass communications tools- to address pieces of misinformation as they emerge. This is possible through diligent, large-scale media monitoring that is conveyed and translated into practice by communications experts and trusted community voices.

Where We Go From Here

Based on digital metrics and comment sentiment analysis, we feel that our campaign was effective at reaching our target audience with fact-based COVID-19 misinformation prebunk and debunk messaging. Future organizations and practitioners can use this as a guide for future public health campaigns with the Hispanic community. Applicability of this model may be explored for projects with other BIPOC communities and on other health topics with accompanying infodemics. Future research can also explore the evaluation of outcome measures after implementing this model, which was outside of the scope of our project.

Additionally, other public health practitioners should consider the need for “infodemiology” capacity building, meaning those who can track misinformation the way we track disease. Through our webinars and newsletters, we worked to train CBOs on how to identify the misinformation they read and heard and how to effectively respond or counter it. While the volume of misinformation on a given topic may ebb and flow with current events, public health should work urgently to build infodemiology skills within its workforce and partnerships. By training the next generation of infodemiologists, we can work collaboratively to identify and address misinformation when it does increase, and ultimately address future infodemics together.