1 Introduction

The scientific consensus on the anthropogenic-caused nature of climate changeFootnote 1 and its dangerous impacts is close to 100% (Lynas et al. 2021). Nevertheless, most people are not aware of this consensus (Maibach et al. 2014), and as a result, beliefs in the existence and human-caused nature of climate change remain low among a significant portion of societies. For instance, according to the Yale Climate Opinion Maps 2021, 14% of Americans say global warming is not happening, and 34% do not worry about it (Howe et al. 2015). Recent evidence from the European Social Survey (Round 10) shows that one in five Europeans does not worry about climate change.Footnote 2 Although some studies suggest that climate change worry increased during the COVID-19 pandemic at least in Europe (Stefkovics and Hortay 2022; Gregersen et al. 2022), climate skepticism remains a problem. Since skepticism may jeopardize climate policy support or climate actions (Hornsey et al. 2016; Bouman et al. 2020), understanding the formulation of climate change beliefs is important.

One of the most crucial factors that shape an individual’s climate change-related support and actions is climate change communication. People draw most of their knowledge about scientific issues from the media (Wilson 1995; Funk et al. 2017); thus, media outlets have a crucial responsibility to provide accurate and comprehensive information about anthropogenic climate change (Anderson 2011; Bayes et al. 2023; Goldberg et al. 2022).

Issues in media outlets are presented and contextualized in frames (Goffman 1974). Framing in communication refers to prioritizing and highlighting certain aspects of a given reality over others to make sense of the information being presented (Entman 1993; Chong and Druckman 2007). Scholars have found the framing theory constructive to discovering media representations of climate change and assessing their effectiveness with experimental methods (Badullovich et al. 2020). For instance, a recent content analysis of Vu et al. (2019) investigated the news coverage of climate change in 45 different countries and territories using topic modeling. The most popular frames were international relations and the economic impact. The results, however, varied by country characteristics. In countries with higher GDP per capita, media outlets were more inclined to present climate change as matters concerning science and domestic politics and less likely to interpret climate change from the natural impact and international relations angles. Other studies pointed to differences in the reporting of climate change between the USA and Europe (Boykoff and Boykoff 2004; Boykoff and Rajan 2007). The early communications differed in a way that controversy and uncertainty were given more importance in the USA, whereas European media outlets were more likely to adopt an alarmist tone (Boykoff and Boykoff 2004; Boykoff and Rajan 2007), but this difference may have decreased during the 2010s (Stecula and Merkley 2019).

This paper focuses on two key parallel frames that have emerged regarding environmental issues: global warming and climate change (Schuldt et al. 2011; Whitmarsh 2009; Leiserowitz et al. 2014). While the two terminologies are often cited as synonyms, in the scientific literature they are considered two distinct phenomena: global warming refers to surface temperature increases, while climate change refers to “the long-term change of the Earth’s climate including changes in temperature, precipitation, and wind patterns over a period of several decades or longer” (Leiserowitz et al. 2014, 6). Frank Luntz in his 2002 memorandum to the Bush administration was the first to advise that “it’s time for us [conservatives] to start talking about “climate change” instead of global warming” (p. 142). He argued that climate change is “less frightening,” while global warming “has catastrophic connotations attached to it.” The global warming framing indeed highlights the increase in temperature and may invite different associations as opposed to climate change which allows for a more general interpretation of how the world’s temperature is changing.

Whitmarsh (2009) in a qualitative study found that people more often believed that global warming is human-caused, whereas climate change was interpreted as a natural process; therefore, global warming evoked more concerns. Furthermore, associations of different impacts and solutions differed between the two frames. The author suggests that the global warming frame is more emotive, while climate change is more neutral. In another study by Leiserowitz et al. (2014) both affective reactions and spontaneous associations were different in the two frames. Global warming elicited more negative feelings and evoked more associations to “extreme weather” compared to the more neutral associations attached to climate change framing (“weather”). Differences between the two frames are also reflected in social media. An analysis of Twitter data collected in the USA, UK, Canada, and Australia showed that users in all four countries were more likely to use the term “global warming” instead of “climate change” when tweets were related to hoax frames (Jang and Hart 2015).

A short line of quantitative wording experiments investigated whether the use of the two frames shifts climate change beliefs. In survey research, it has been long studied how minor changes in question wording may affect responses (e.g., Cantril 1944). That may occur precisely because certain terminologies provide frames for interpreting survey questions.

Schuldt et al. (2011) were the first to examine the effect of “global warming” vs. “climate change” framing on existence belief in a sample of 2267 Americans. They found that existence belief was 6.3 percentage points higher in the “climate change” framing condition, although this difference was strongly moderated by political self-identification. The difference among Republicans was large (16.2 percentage points), whereas the treatment did not influence respondents with other political beliefs. Two later studies (Schuldt et al. 2017; Schuldt et al. 2015) replicated these results in the American context. The 2015 study extended the original by showing that the framing similarly affects perceptions of scientific consensus regarding climate change and support for limiting greenhouse gas emissions. The observed differences between Republicans, Democrats, and Independents were also fairly replicated. Morin-Chassé and Lachapelle (2020) re-analyzed the data of Schuldt et al. (2015). Their analysis nuanced earlier findings and showed that the strength of partisan attachment plays a role. The impact of issue labeling varied for Democrats with stronger partisan attachment compared to Democratic leaners, whereas issue labeling affects all Republicans, particularly those who are strong Republicans. In another experiment conducted in the USA, Villar and Krosnick (2011) found that climate change was perceived to be more serious than global warming among Republicans, whereas the reverse was true among Democrats.

Other studies, however, found no meaningful framing effects. Dunlap (2014), in an American survey, reported no differences in responses to worry or perceived seriousness, regardless of political beliefs. In contrast to other studies conducted in the USA, Benjamin et al. (2017) found stronger framing effects for political Independents and those who are disengaged from climate change on different aspects of climate change beliefs. Furthermore, evidence outside the USA is also mixed. In the Eurobarometer survey conducted in 31 European countries, Villar and Krosnick (2011) found no meaningful differences in the ratings of climate change’s seriousness when framed differently, and political identification did not moderate the findings. Another replication of the study of Schuldt et al. (2011) conducted in the USA, the UK, and Australia (Soutter and Mõttus 2020) reported no effect of question framing on climate change existence beliefs. Although conservatives believed less in climate change than liberals in all three countries, treatment effects were not moderated by political identification. Lastly, Neumann et al. (2022), in a sample of 5104 Australians, reported insignificant main and interaction effects regarding climate change importance, worry, and perceived personal and generational harm.

The results presented above show that while findings in the USA context are, to some extent, consistent, they may not be universally valid. This study provides additional evidence to this issue. In a Europe-wide nationally representative survey, we experimentally manipulated the wording of four climate change-related questions (“global warming” vs. “climate change”). We investigated treatment effects both in the full sample and separately among leftist, rightist, green, and right-wing party voters.

There are several reasons why further research is needed to investigate this problem. First, the substance of frames such as global warming or climate change are dynamically changing (Leiserowitz et al. 2014; Soutter and Mõttus 2020); therefore, the relevance of earlier findings can be questioned. For instance, the COVID-19 pandemic may have impacted how and to what extent the two frames were used. A simple Google Trends analysis shows a strong increase in “climate change” searches during the pandemic but not for “global warming” (see Figure S1). Second, the impact of frames can also vary between cultures. Considerable evidence shows that the effect of climate change framing is attached to political orientation in the USA, but it is unclear to what extent American findings generalize to other cultures. Although earlier studies used cross-national data, the most recent international study (Soutter and Mõttus 2020) was conducted in three countries only (Australia, the UK, and the USA). Third, we are not aware of any study conducting similar wording experiments in telephone surveys. Telephone surveys differ from self-administered surveys in a key aspect: respondents do not have a visual sight of the questions but rely solely on aural clues (for a review of such mode differences, see, e.g., De Leeuw 2005 or Bowling 2005). Framing effects may be weaker in phone surveys compared to online surveys due to constraints in pace and the inability to review questions and responses. Given that individuals are frequently exposed to aural frames through various mediums, such as radio broadcasts, framing experiments conducted by phone may have high external validity. Lastly, by using standard questions of the European Social Survey (ESS), we provide direct evidence to the ESS community, which may help researchers formulate better survey questions.

2 The current study and hypotheses

The aim of this study was to assess whether the use of two different terminologies (“global warming” vs. “climate change”) shifts responses to survey questions. We embedded a split-ballot experiment in a large-scale, cross-national telephone survey. Respondents were randomly assigned to one of the two wording groups and answered four questions regarding their climate change beliefs (see the exact questions in Table 1). We used the standard questions of the ESS. The use of ESS questions was necessary for other research purposes, although by doing thus, we somewhat deviated from earlier studies. The ESS question measuring existence belief, however, is worded very similar to the one used in the Schuldt et al. (2011) and the Soutter and Mõttus (2020) studies.

Table 1 Question-wording in the two experimental groups
Full size table

The ESS wording was equivalent to the “climate change” group’s wording. In the “global warming” group “climate change” was simply replaced by “global warming” in three questions. However, further modifications were required regarding the question measuring existence belief. Since this question already focused on rising temperatures, we excluded the phrase “climate is changing” in the “global warming” group. We note that both versions included clues on rising temperature, which means that the framing of these two groups was not as distinct as it was for the other three questions. However, making additional modifications would have led to an undesirable deviation from the original ESS version. Response options were tailored to the question accordingly (see Table S1).

Based on earlier findings, we expected that the global warming framing would elicit higher skepticism and lower worry than the climate change wording.

  • H1: Beliefs in the existence and human-caused nature of climate change, worry, and belief in the negative impact will be lower in the global warming group.

We further suspected that political ideology would moderate the effect of framing. Specifically, the results of Schuldt et al. (2011), Schuldt et al. (2017), Schuldt et al. (2015), and Villar and Krosnick (2011) suggested that Republicans are more likely to be affected by issue framing. In this study, we use the identification on the left-right scale and green or right-wing party support as potential moderators. The left-right scale has been identified as a decisive predictor of climate change beliefs in Europe (Poortinga et al. 2019; Lee et al. 2015; Czarnek et al. 2021; Hornsey et al. 2016), and particularly in Anglophone contexts (Smith and Mayer 2019), with leftists being more likely to believe in climate change and rightist being more skeptical. Therefore, we expected a stronger framing effect among rightists. Accordingly, climate change attitudes sort along party lines in Europe (Birch 2020; Lewis et al. 2019; Chen et al. 2021). Green parties put the issue of climate change at the top of their agenda, whereas populist right-wing parties often explicitly deny the existence of climate change (Forchtner and Kølvraa 2015; Lockwood 2018).

  • H2a: The effect stated in H1 will be stronger for rightists than for leftists

  • H2b: The effect stated in H1 will be stronger for right-wing party voters than for green party voters

3 Data and methods

3.1 Data

Data is drawn from a nationally representative telephone survey conducted in 27 member states of the European Union, Norway, Switzerland, and the UK from 1 August to 15 September 2021 by Századvég Foundation, a Hungarian research institute. The total sample size is approximately 30,000, with 1000 per country. Data was collected via computer-assisted telephone interviews (CATI), using a dual-frame (landline and cell phone) sampling frame. We followed a geographically stratified random sampling procedure to select households. The last birthday method was used for individual selection (Salmon and Nichols 1983). The survey asked respondents about their political attitudes, economic perceptions, their opinion about the EU, the COVID-19 pandemic, family, and energy issues. The experiment was placed in the middle of the questionnaire. In our weighting procedures, we followed a similar approach to the ESS (Lynn and Anghelescu 2018). Post-stratification weights were applied to correct for over-or underrepresentation, nonresponse, and sampling error, and were calculated by age group, gender, education, and region.

3.2 Methods

The items measuring climate change beliefs were recoded so that higher values mean lower skepticism or higher worry/negative impact. For the beliefs in human-caused climate change, we recoded “I don’t think climate change is happening” responses to 0.

To measure the respondents’ left-right identification, we used the following ESS question “In politics people sometimes talk of ‘left’ and ‘right’. Where would you place yourself on this scale?” The scale ranged from 0 “left” to 10 “right.” We recoded the responses so that respondents answering between 0 and 3 were considered leftist, and respondents answering between 7 and 10 were considered rightist.

We measured party support by asking respondents “If national elections were held today, for which party would you vote?” We updated the list of green and right-wing parties developed by Pearson and Rüdig (2020) and Down and Han (2021), respectively (see Table S2).

To test H1, multilevel mixed-effect models with random intercepts for countries were fitted. This method allowed us to account for the unobserved country-level heterogeneity. Additionally, we report the results of t- and chi-squared tests. To test H2, we ran the same models and added interaction terms between the treatment (0 – global warming group, 1– climate change group), left-right identification, and green or right-wing voting. Those with a moderate ideological alignment and other parties’ voters were excluded from these models. Due to the absence of green or right-wing parties in certain countries, this analysis was limited to 21 and 27 countries, respectively. We further provide the results of per-country t-tests to highlight country differences.

4 Results

Existence belief was significantly higher, and the perceived negative impact of climate change was significantly lower when the question was framed as “climate change.” In contrast, our treatment did not affect beliefs in human-caused climate change or climate change worry. The results of the t- and chi-squared tests support these findings with one exception. The negative impact of climate change was not significantly different between the two groups based on these tests (see the plotted averages in Figure S2, and test results in Tables S3–S7). These results provide little support for H1 (Table 2).

Table 2 Results of the multilevel mixed effect models with a random intercept for countries
Full size table

Turning to H2, leftists and green party voters were consistently less skeptical and more worried about climate change than rightists and right-wing party voters (see Tables S3–S6 for the distributions). The effect of the treatment, however, did not differ meaningfully between these groups. Interaction effects between the treatment and both left-right identification and green- and right-wing support were insignificant (see Tables 3 and 4). Thus, existence belief was higher in the climate change frame group regardless of political ideology or vote choice. Worry and the perceived negative impact were identical between the two experimental groups across the political spectrum. One exception was the belief in human-caused climate change, where we found that beliefs were higher among green voters when the issue was framed as global warming, whereas among right-wing voters, these beliefs were, in turn, higher when the issue was framed as climate change. The results of the t- and chi-squared show the same pattern (see Tables S3–S6). Thus, we found nearly no support for H2a and b.

Table 3 Results of the multilevel mixed effect models with a random intercept for countries and an interaction term with the left–right identification
Full size table
Table 4 Results of the multilevel mixed effect models with a random intercept for countries and an interaction term with the green- and right-wing support
Full size table

Lastly, we found large variations both in the size and direction of the framing effect between countries (see Fig. S5). The results about existence belief were the most consistent. The treatment shifted responses in the same direction in the majority of the countries, with nine countries yielding statistically significant differences. Nevertheless, the results do not offer a clear regional pattern. Treatment effects for the three other questions were inconsistent and weak, with around 3–4 countries showing significant framing effects in either direction.

5 Discussion

This paper provided new evidence on whether the use of different frames in environmental issues (“global warming” vs. “climate change” framing) influences self-reports to survey questions. We contributed to the literature by conducting a large-scale Europe-wide split ballot survey experiment in which we manipulated the framing of questions about different aspects of climate change beliefs. Our results regarding climate change existence belief replicated earlier findings (Schuldt et al. 2011; Schuldt et al. 2017; Schuldt et al. 2015), given that existence belief, were significantly higher in the climate change group. We also found that respondents in the “climate change” group perceived the impact of the issue slightly more negative. Nevertheless, in contrast to some of the earlier findings in the USA (Schuldt et al. 2011; Schuldt et al. 2017; Schuldt et al. 2015; Villar and Krosnick 2011), we did not find much evidence of people being more skeptical or less worried when asked about “global warming” compared to “climate change” in Europe. Furthermore, our findings did not support previous observations on the differential framing effects based on people’s political ideology or partisanship.

One explanation of these null results is that since the studies conducted during the 2000s and 2010s the public’s climate change concerns have become widespread (Pew Research Center 2021, 2020). When concerns are high, and people become less skeptical framing may play a less important role at least on the level of general attitudes toward the issue. People in several European countries are more concerned about climate change than Americans (Pew Research Center 2020); thus, the abovementioned reduction of framing effects due to higher concerns may be stronger in Europe.

This leads us to the findings that leftists, rightists, green, and right-wing voters reacted rather similarly to the two frames, contrary to previous US studies where Democrats and Republicans differed strongly in this regard. This is somewhat surprising given that the issue of climate change is strongly polarized along ideological and partisan lines both in the European public (Poortinga et al. 2019; Lee et al. 2015) and the political arena. However, recent analyses found that the effects of populist parties in Europe on the European Union’s energy and climate policy are mixed (Huber et al. 2021), climate change denial and populist antiestablishment attitudes are only weakly correlated (Jylhä and Hellmer 2020). Moreover, the study of Smith and Mayer (2019) showed that the effect of party affiliation on the perception of climate change’s danger and importance is strongest in Anglophone states, moderate in Western European nations and constrained in post-Communist states, which may as well explain the lack of a more general moderating effect. The findings suggest that compared to the strong partisan gap observed in the USA and other Anglophone countries, leftists, rightists, and green and right-wing voters use more similar cognitive climate change frames in European societies.

One of the key strengths of this paper was that it showed how framing may affect certain beliefs but not others. In line with Villar and Krosnick (2011), climate change worry is unlikely to be affected by the framing of the issue; however, people were more likely to be skeptical whether climate change/global warming is happening when it was framed as global warming. This may be attributed to the increased prevalence of the climate change frame in recent times (as shown in Figure S1), and people tend to believe in something that they hear/read more often (although, of course, the warming of the Earth is part of climate change). Our piece of evidence supports earlier assumptions that different psychological mechanisms may be responsible for climate change existence beliefs and attitudes (Krosnick et al. 2006). Nevertheless, it is possible that we found the strongest effect for existence belief because we made the most substantial changes to the wording of that specific question.

Lastly, we found that the effects varied a lot between countries indicating that the European society itself is not unified in this regard, and certain frames may play important roles in some cultures but not in others. This finding resonates well with other studies in the field of cross-cultural survey methodology (e.g., Stark et al. 2020).

One implication of our findings is that increasing the use of “climate change” framing in communication strategies promoting climate action itself will do little to lower skepticism and perhaps will not decrease worry at all, given that both terms are politically polarized (Dunlap 2014). Skepticism was high in our survey, especially among rightists and right-wing voters; thus, future research should focus on exploring how divisions can be reduced with other frames. Earlier studies already explored the effect of economic, scientific, moral, intergenerational, national security, and other frames on public support for climate policies (Diederich and Goeschl 2014; Gainous and Merry 2022; Severson and Coleman 2015; Shrum 2021). Wording experiments that manipulate other elements of survey questions are also encouraged given that earlier research has already pointed out the importance of some of these considerations (Chen et al. 2021).

One limitation of our study is that the survey did not include the measure of liberal-conservative identification. Future studies could explore whether differences exist between liberals and conservatives in these countries. Additionally, our vote choice question captured strong partisans (by asking respondents how they would vote today). Based on the results of Morin-Chassé and Lachapelle (2020), future research could explore the moderating effect of partisanship strength or affect toward green or right-wing parties.

Effective climate change communication is key to stimulating public action and support for climate policies (Bruine de Bruin et al. 2021). While our study has shown that choices between “global warming” vs. “climate change” framings might not make much difference, tracking and understanding climate change framings with the help of similar evidence-based research remains essential to support mitigating climate change. Survey methodological research and the ESS, in this particular case, should also keep investigating wording effects as a long line of research has proved already that minor changes in questionnaire design can shift responses.