Introduction

Civic education serves as the bedrock of effective democratic citizenship, laying the groundwork for informed and active participation in society (Muleya, 2017). Helvoort (2018) highlights the significance of civic knowledge within the broader framework of civic literacy. It encompasses knowledge about the rights and responsibilities of citizens, the structure of government, and the history of civic struggles and achievements, and equips individuals with the requisite understanding for meaningful participation in democratic processes. Accordingly, civic knowledge plays a fundamental role in fostering civic engagement and participatory democracy and resonates deeply within both the educational sphere and broader society (McCabe et al., 2017).

The International Civic and Citizenship Education Study (ICCS), conducted by the International Association for the Evaluation of Educational Achievement (IEA), assesses the civic knowledge of Grade 8 students on a large scale. The last iteration of this program was conducted in 2016, which involved 94,000 students from 24 countries (Schulz et al., 2018). Considering the recent conclusion of this program (in 2022), investigation of the factors affecting civic knowledge based on existing data becomes crucial.

Research has shown that students’ gender and school location influence students’ civic knowledge (Cicognani et al., 2012; Metzger & Ferris, 2013). In addition, internet access can also be essential for learning civic knowledge. Past research has highlighted the vital role of the Internet in hel** students improve their civic knowledge by searching for information or news on the Internet (Gottfried et al., 2017; Partheymüller & Faas, 2015). Given the increased prevalence of online teaching trends that require internet access (Engin & Donanci, 2015) and online learning tools (Fu & Hwang, 2018), students without internet access at home may face challenges in civic knowledge acquisition through online sources. More importantly, the availability of online learning methods can dominate students’ attitudes toward civic education (Virgiawan & Sundawa, 2022). Considering the fact that equal access to the internet remains a challenge even within developed countries, internet access should be considered a critical variable when measuring civic knowledge. The previous literature has focused on the correlation between internet accessibility and the fields of science, mathematics, and reading (Courtney et al., 2022; Gubbels et al., 2020; Lorah, 2018). The presented study aims to fill the research gap by delving into the impact of internet access on students’ civic knowledge in Taiwan using data from ICCS 2016.

The investigation of the influence of background variables (internet access in this study) embraces a common approach of comparing the mean difference between group test scores. Notably, such comparisons are meaningful only if the assessment tool is fair (i.e., unbiased) to examinees in different groups. Thus, testing the measurement equivalence is essential. The ICCS 2016 surveyed internet access availability for students at home (i.e., yes or no) and measured their civic knowledge through eight booklets. For each booklet, Taiwanese students with internet access at home exhibit typically higher mean scores in civic knowledge items in comparison to those without internet access. Such comparisons are however not exhaustive and have at least two major limitations. First, we need to discover whether the test items were equivalent for the two groups. Second, although the eight booklets were linked with common items, the (raw) test scores were not comparable across booklets because they were not calibrated on the same scale.

To investigate the equivalence of items, a differential item functioning (DIF) assessment has been conducted in the test. In large-scale assessments such as the Programme for International Student Assessment (PISA), the Trends in International Mathematics and Science Study (TIMSS), or the ICCS, DIF assessment has become a standard procedure (e.g., Schulz et al., 2011; Schulz et al., 2018; Martin et al., 2020). Recently, each country has employed standard techniques to avoid DIF among genders and ethnicities (Cuellar et al., 2021). Hence, although DIF items can be easily identified using existing methods, in-depth exploration of the underlying causes of the occurrence of DIF becomes critical. In other words, a post hoc explanation of what causes the item(s) to display DIF is more valuable (Zumbo, 2007). However, only gender was assessed for DIF in the ICCS 2016 technical report without further investigating the root cause of the gender DIF displayed by the item. Therefore, an additional analysis (i.e., DIF source investigation) should be conducted to address the limitations mentioned above in the methodology when conducting a DIF-related study.

Accordingly, the current study aims to determine the existence of DIF among students with and without internet access. Moreover, multiple analyses will be executed to provide further evidence regarding the influence of internet access on civic knowledge measurement.

Literature review

Internet access and students’ civic knowledge

Civic knowledge learning involves the study of all kinds of information related to society (Greenberg, 1970). Students need to have a solid foundation of civic knowledge and comprehension to allow them to construct well-informed ideas and perspectives, understand the institutions and operations of government, and engage in civic and political life actively (Lo & Adams, 2018). They can create this foundation by reading significant civic publications, gaining an understanding of ideas that stimulate in-depth thoughts about social concerns, and develo** the ability to establish connections between significant historical events and their contemporary relevance (Zorwick & Wade, 2016). Students can improve their knowledge of relevant topics throughout the curriculum by using civic engagement as an effective educational technique for addressing real-world situations (Kelley & Knowles, 2016).

The phenomenal growth of the Internet and telecommunications has made global information resources readily accessible in homes and schools (Diamond, 2020). The internet has heralded the rise of a new form of knowledge production and distribution and a virtually unlimited wealth of information resources with simultaneous availability worldwide (Gruchel et al., 2022). This phenomenon has significantly changed students’ knowledge acquisition patterns globally (Diamond, 2020), particularly empowering them to access all kinds of information and learning materials regardless of the limitations of space and time (Chen & Tsai, 2021). The Internet has been confirmed as one of the primary sources for students to gain knowledge (Diacopoulos & Crompton, 2020). Previous studies have examined the relationship between online news and individuals’ political knowledge acquisition (Gottfried et al., 2017; Partheymüller & Faas, 2015) as well as the differential effects of different media formats on political information acquisition (Dimitrova et al., 2014; Weaver & Drew, 1993). It has also been demonstrated that using social networking services (SNS) to meet new people positively correlates with bonding and bridging social capital in the virtual world, which fosters online civic involvement (Zhong, 2014). Moreover, in a globalized era, the study also indicated that multicultural exposure, multicultural interaction, and social media usage are positively correlated with global civic engagement (Tarman & Kilinc, 2022). These can be easily achieved when students have better internet access.

The internet and media are more than a source of civic knowledge learning; they also serve as a stimulus that propels students' interest in learning civic education (Pang, 2018). Moreover, the Internet is also closely related to online teaching and learning tools that have recently been widely employed and are readily accessible on mobile phones and tablets (Engin & Donanci, 2015; Fu & Hwang, 2018). McGrew et al. (2019) observed that American students’ media literacy and civic online reasoning levels were low. They further emphasized that more comprehensive research-based approaches to digital education were necessary to equip students with the IT abilities required for their active participation in civic life. In other words, the internet gap or students’ lack of IT ability or media literacy could impede the development of their civic knowledge.

In contrast, Virgiawan and Sundawa (2022) analyzed the obstacles to studying civic education and rationalized the relevance of building students' character in online learning environments. The results showed that innovation in online learning methods and models, coupled with teachers’ commitment to establishing discipline in learning civic education, can overcome the challenges that could hinder students' sense of responsibility and democratic attitude development in civic education. Specifically, students with no internet access at home will be excluded from all these civic knowledge acquisition and online learning opportunities (Kewajiban et al., 2018). The internet gap threatens students’ learning outcomes, which could be further intensified during situations like the COVID-19 pandemic or other similar scenarios (Nischal, 2022).

Students’ family backgrounds and their internet access, utility, and civic knowledge

Students’ socioeconomic background, internet access, and civic knowledge

Students’ family backgrounds can influence digital divides, internet access, and civic education. Graves et al. (2021) indicated that rural students face more trouble accessing the technology and connectivity needed for remote learning than their urban peers. Inadequate internet infrastructure in rural areas hinders remote learning. Therefore, initiatives for policy improvement and technology advancement must consider regional differences to achieve educational equity. Furthermore, Metzger et al. (2020) examined the differences in civic beliefs and efficacy among youth living in rural or city areas and from higher or lower socioeconomic households. Youth from more educated households viewed social movement engagement as more important and morally worthy, whereas rural youth and youth from less-educated households had lower political efficacy and rated standard political activity as less important and obligatory. These findings highlight the importance of community and socioeconomic context for the civic development of the youth.

The influence of students’ socioeconomic backgrounds has emerged in several other ways. Zhong (2014) found that the direct effect of using SNS to meet new social ties on civic engagement was partially mediated by social capital, while the effect of using SNS to connect with friends on civic participation was completely mediated by social capital. Since family capital, including social capital, is considered an important factor for the educational achievements of students from different social backgrounds (Liu & Gao, 2011), it is argued that students’ socioeconomic background can also affect the relationship between SNS use and civic engagement, as mentioned above.

Parenting, internet utility, and civic knowledge

The influence of students’ socioeconomic background can be traced back to their internet usage patterns determined by parenting styles. As summarized previously, internet access provides more opportunities for students to retrieve additional learning materials and information, but the kind of content students browse can be another concern. Internet use is significant in children’s everyday lives and can be distinguished into different uses (e.g., entertainment-related, informative, or school-related) (Gruchel et al., 2022). Several studies have supported the idea that internet utilization is most prevalent among younger and more educated individuals (Hoffman et al., 2000). Alshahrani et al. (2017) concluded that the use of the internet profoundly impacted students’ academic self-confidence, self-reliance, and student–lecturer connectedness. This is because using the internet on their mobile phones allows students to search and access academic information instantly (Mamudu & Oyewo, 2015).

Moreover, students believe that the use of the internet enables them to perform research ahead of time and tackle multiple home assignments along with widening the scope of reading and learning, promoting self-learning, encouraging and fostering peer learning, and enhancing student examination preparation (Apuke & Iyendo, 2018). They also believe that the internet helps them to discover new knowledge related to their courses, individually or as a group. They feel that the availability of online video tutorials on platforms like YouTube broadens their academic research and helps them to be conversant with certain practices and technical aspects of their various courses. These results support Hamid et al.'s (2015) findings, which established that internet utilization helps Australian and Malaysian students share materials for a group assignment, which leads to their improved self-directed learning. In other words, if students use the Internet for civic knowledge-related learning, they can broaden the scope of content they receive from school teachers and increase their active, spontaneous learning, which can further enhance their learning outcomes in civic education.

When reviewing the literature on Internet inquiries, the family holds particular importance as an informal place of learning. Parents' motivational beliefs are associated with parental instruction in children's practical and school-or-learning-related internet use (Gruchel et al., 2022). Specifically, parents’ perceived parental instruction predicts entertainment-related internet use negatively and informative or school-related use positively. Moreover, parents can guide their children's internet use for educational-related purposes. However, Zhong (2014) indicated that using SNS to meet new people was positively correlated with bonding and bridging social capital in the virtual world, which was positively related to online civic involvement but not significantly related to offline civic behaviors. In other words, when students have internet access at home and they browse civic knowledge content under their parents’ guidance, their civic literacy can be potentially enhanced further by engaging in real-life civic activities and learning materials.

The role of parents in promoting students’ civic engagement and learning is also evident in patterns of their discussions on civic events with their children. In their study, Chan and Mak (2020) concluded that adolescents’ political engagement and participation were influenced by parental participation, and parents with higher socioeconomic status tended to have higher political engagement. Oosterhoff and Metzger (2016) further examined the associations between mother–adolescent communications and teens’ own judgments about civic duty and behavior. They argued that mothers’ involvement in community service and messages concerning respect and hel** others were positively associated with teens’ community service behavior.

In summary, students from a higher socioeconomic family tend to have better access to the internet at home, which enables them to browse all online resources and experience more multicultural exposure, multicultural interaction, and social media usage that enhances their local and global civic engagement. Moreover, parents could be more aware of guiding students toward more productive use of the internet for school- or learning-related tasks and spend more time discussing and participating in civic activities with them. All these factors may equip students with better civic literacy and engagement.

Differential item functioning and its implementation

DIF assessment methods can be roughly classified into parametric and nonparametric approaches. In parametric methods such as the logistic regression method (LR; Swaminathan & Rogers, 1990), the multiple indicators and multiple causes (MIMIC) model (Shih & Wang, 2009; Woods, 2009), and the likelihood-ratio test method (LRT; Thissen et al., 1988), the parameter associated with DIF is modeled for direct estimation of the size and significance of DIF for every studied item. However, such methods are relatively complex for users to operate. In contrast, nonparametric methods, such as the Mantel–Haenszel (MH) method (Holland & Thayer, 1988) and the odds ratio (OR) method (** et al., 2018), do not have a statistical model for observed responses; hence, users can easily calculate the statistic to judge whether the studied item exhibits DIF or not. In particular, the OR method can accommodate sparse data matrices in large-scale tests (e.g., ICCS) and is user-friendly (** et al., 2018). Hence, this study implemented the OR method to assess DIF.

Investigating the influence of internet access on civic knowledge measurement

To summarize, the preceding sections have highlighted past research regarding internet access, civic knowledge, and the methodology for investigating the background variables. Our examination of the existing literature reveals several key trends in the field. Building upon these insights, this study seeks to address the following research questions:

  1. a)

    Were the items measuring civic knowledge in the ICCS 2016 fair to students regardless of whether they have internet access at home or not?

  2. b)

    What factor(s) makes the items unfair for students who do not have the internet at home?

  3. c)

    To what extent did students who used the internet at home have a higher level of civic knowledge than those who did not?

Note that in this paper, "fair (unfair)" refers to the equal (unequal) probability of students from households with and without internet access answering specific civic knowledge items correctly, assuming their civic knowledge abilities are equal.

Material and methods

Instrument

The civic knowledge test for ICCS 2016 comprised 79 multiple-choice items and 9 constructed-response items. All items were assigned to eight blocks, and eight booklets were created with three blocks each. Since the descriptions of construct-response items are not available publicly, only the multiple-choice items with complete descriptions of each option in the codebook were emphasized in this study.

Students’ background variables were collected through international student questionnaires. Notably, students were asked to provide a binary response (yes or no) to IS3G13 (Do you have an internet connection at home?). Thus, the binary response was used as the basis variable for grou** students to assess the DIF items.

To provide a meaningful explanation for the possible DIF items, several background variables were selected from both student and school questionnaires (see Table 1). These variables focused on devices, students’ socioeconomic status, and their interactions with parents. Most background variables were rated directly by participants on a four-point Likert scale, with the sole exception of the SES index (S_NISB), which is an integrated variable. The SES index was derived from the highest occupational status of parents, the highest educational level of parents, and the number of books at home, and the algorithm is presented in the technical report (Schulz et al., 2018) as follows:

  1. 1.

    Recode each indicator and conduct an imputation with a regression method.

  2. 2.

    Convert three indicators into z-standardized variables.

  3. 3.

    Conduct a principal component analysis of these indicator variables.

  4. 4.

    The SES index consists of factor scores for the first principal component.

Table 1 The description of background variables that are investigated to the sources of DIF items
Full size table

Sample

A total of 3953 students in Taiwan participated in the ICCS 2016 study. After excluding 15 cases with missing or invalid responses to the grou** variable, responses of 3605 students who answered ‘yes’ and 333 students who answered ‘no’ to IS3G13 were retained to investigate the effect of "has/does not have the internet at home" on civic knowledge measurement. Notably, the disparity ratio between groups is not uncommon in the education field. For example, a test item may be interpreted differently by aboriginal (minor group) and non-aboriginal (major group) groups (Guhn et al., 2007; Marotta et al., 2015).

Procedure

To reach the study goals, we conducted multiple statistical analyses sequentially, including DIF assessment, latent regression analysis in the Rasch model (Wilson et al., 2008), logistic regression, and correlation.

DIF Assessment

The OR method (** et al., 2018) was implemented to detect DIF items. As mentioned earlier, the eight booklets of the civic knowledge test were linked with common items, suggesting that the combined data matrix was sparse and had several missing responses. Such data properties make common DIF assessment methods cumbersome and less effective. The OR method is especially suitable for a huge dataset with missing responses (** et al., 2018).

In the OR method, the ratio of the odds of success for two groups (e.g., with vs. without internet access) is computed for each item as follows:

$$\widehat{\lambda }_{i} = {\mkern 1mu} \log {\mkern 1mu} \left( {\frac{{n_{{R1i}} /n_{{R0i}} }}{{n_{{F1i}} /n_{{F0i}} }}} \right)$$
(1)

where \(\widehat{\lambda }_{i}\) is the statistic of item i for DIF assessment; \({n}_{R1i}\) and \({n}_{R0i}\) are the numbers of students in the reference group who respond correctly and incorrectly to item i; \({n}_{F1i}\) and \({n}_{F0i}\) are the numbers of students in the focal group who answer correctly and incorrectly to item i, respectively. Furthermore, a string of \(\widehat{\lambda }\) values is assumed to follow a normal distribution with a mean of \(\uplambda\) asymptotically. When data follow the Rasch model without DIF items, \(\widehat{\lambda }_{i}\) is an unbiased estimate of \(\uplambda\) (the mean ability difference between the reference and focal groups, usually called impact in DIF literature), with a standard deviation of \(\left( {\widehat{\lambda }_{i} } \right)\):

$$\sigma \left( {\widehat{\lambda }_{i} } \right){\mkern 1mu} = {\mkern 1mu} \sqrt {n_{{R1i}}^{{ - 1}} + n_{{R0i}}^{{ - 1}} + n_{{F1i}}^{{ - 1}} + n_{{F0i}}^{{ - 1}} }$$
(2)

To have a reference to assess DIF items, the median of \(\widehat{\uplambda }\) values (\(\widetilde{\widehat{\uplambda }}\)) can be used as a robust estimator (** et al., 2018). Thus, item i will be deemed DIF when \({\widehat{\uplambda }}_{i}\) is far away from \(\widetilde{\widehat{\uplambda }}\). Specifically, the item is identified as DIF when the distance between the \({\widehat{\uplambda }}_{i}\) and \(\widetilde{\widehat{\uplambda }}\) is significantly different from 0, and the absolute value of the distance (effect size) is larger than 0.3 (Schulz et al., 2018). In this study, the reference group denoted those students who had internet at home, and the focal group denoted students who lacked internet. When \({\widehat{\uplambda }}_{i}\) is largely higher than \(\widetilde{\widehat{\uplambda }}\), item i favors the reference group. Likewise, when \({\widehat{\uplambda }}_{i}\) is lower than \(\widetilde{\widehat{\uplambda }}\), item i favors the focal group. The scale-purification procedure (Navas-Ara & Gomez-Benito, 2002; ** et al., 2018) was also implemented to control the Type I error rate.

Considering that a large group mean difference can bias the estimate of \(\widetilde{\widehat{\uplambda }}\) (DeMars, 2021), we also employed the LR and LRT methods as well as the Rasch DIF modeling approachFootnote 1 (Wang, 2008) in R (version 4.3.2) to triangulate the findings. Because students were assigned to different booklets, raw scores cannot be used to present students’ abilities. Thus, to perform the LR analysis, the person estimates from the Rasch analysis (assuming all items are DIF-free) were entered as the matching variable using the difLogistic function in the difR package (Magis et al., 2010). Also, the difLRT function in the difR was used to conduct the LRT analysis. The Rasch DIF modeling approach was realized using the tam.mml.mfr function in the TAM package (Robitzsch et al., 2022).

Investigating the possible sources of DIF

To avoid collinearity, a two-stage strategy was adopted to gain a deeper understanding of why students performed differently on the DIF items. The background variable might be the possible source of the DIF item when the background variable is predictable for the DIF item and related to the grou** variable. In the first stage, the test score of a suspected DIF item was regressed by the selected background variables (Table 1). In the second stage, the relationship between IS3G13 and significant background variables was examined. As S_NISB is a continuous variable, the point-serial correlation between IS3G13 and S_NISB was calculated. However, for the other (categorical) background variables, a chi-square test of independence was performed. The target background variable(s) are likely to be the source(s) of DIF when a correlation exists.

Latent regression analysis in the rasch model

The comparison of test scores between groups is meaningful only when all the test items are DIF-free. Thus, after removing the suspected DIF items, we applied the Rasch model to the DIF-free items and performed a latent regression analysis (with the binary response to IS3G13 as the independent variable) using the R package TAM (Robitzsch et al., 2022), so that the group difference (in logit) can be directly estimated in one go.

Results

DIF Assessment

Unfortunately, the LRT method failed to converge within 72 h, possibly due to the large number of missing values in the data matrix. Hence, only the DIF detection results of the OR, LR, and Rasch modeling methods were compared.

The \(\widehat{\uplambda }\)s for the 79 multiple-choice items were between –0.42 and 0.79 (Mean = 0.24, Median = 0.19, SD = 0.27). Figure 1 further shows a negatively skewed distribution of \(\widehat{\uplambda }\)s, implying that using the median to express central tendency is more appropriate than the mean. Therefore, \(\widetilde{\widehat{\uplambda }}\) (= 0.19) was used as the reference in the initial stage of the OR method. Six items were initially screened out as DIF, and the results did not change after scale purification. The \(\widetilde{\widehat{\uplambda }}\) for the remaining 73 DIF-free items was 0.17.

Fig. 1
figure 1

Histogram of odds rations \(\widehat{\lambda}_{s}\) in OR mathod

Full size image

As listed in Table 2, five items had a \(\widehat{\uplambda }\) value significantly higher than 0.17, whereas one item had a \(\widehat{\uplambda }\) value significantly lower than 0.17. Moreover, the DIF size for the six items exceeded 0.3 logits uniformly. To sum up, six items were suspected of displaying DIF in the civic knowledge test.

Table 2 Description of DIF assessment of suspected DIF items
Full size table

Five out of the six DIF items favoring students accessing the internet at home are related to social responsibility, government action, environment, and election. The only item (CI3CAM1) favoring students who could not access the internet at home was related to “art as a political expression.”

Common DIF items were also identified in other analyses. In the LR analysis,Footnote 2 three of them (CI2ECM1, item CI3CAM1, and CI3CRM2) were found to exhibit DIF. In the Rasch DIF modeling approach,Footnote 3 all six items were flagged. Therefore, these six DIF items were carried forward to the next analysis.

Investigating the possible source of DIF

A series of logistic regressions were completed to investigate how the background variables influenced the probability of answering the DIF item correctly, in which the binary scores on DIF items were regressed by one background variable at a time. As shown in Table 3, some DIF items’ scores could be explained by selected background variables. Scores on CI2CCM1 were associated with S_NISB. Scores on CI2ECM1 and CI312MI were associated with IS3G14E, and scores on CI3CRM2 were associated with S_NISB and IS3G12B. These six background variables could not explain the other two DIF items.

Table 3 Model comparison results of containing “background variable” or not
Full size table

The correlation results for background variables are shown in Table 4. The highest correlation was 0.27 (p < 0.001), which occurred between "Number of tablets or e-readers (IS3G12B)" and "SES (N_SIB)." Logically, families with high SES are more likely to afford tablet devices and e-readers than those with lower SES. Hence, the potential sources of DIF items focused on SES.

Table 4 Correlation, variance, and sample size of each background variable
Full size table

In conclusion, differences in SES might cause DIF in CI2CCM1 and CI3CRM2, and the frequency of “talking about what is happening in other countries with parents” might be the source of DIF for CI2ECM1 and CI312M1. Detailed elaborations of the four DIF items are provided in the discussion section.

Rasch analysis and latent regression on internet access

For the remaining 73 DIF-free items, the difficulties ranged from − 4.01 to 0.46 (M = − 1.82, SD = 1.01), indicating that the items were relatively easier for the students. The item’s unweighted mean square (i.e., outfit) statistic was between 0.60 and 1.27 (M = 0.95, SD = 0.18), while the weighted mean square statistic (i.e., infit) was between 0.84 and 1.20 (M = 0.99, SD = 0.08). Based on the reasonable range for outfit and infit (from 0.6 to 1.4) suggested by Bond et al. (2021), all items fell within an acceptable fit. In the latent regression model, the estimate of the group difference in civic knowledge was 0.22Footnote 4 logits (p < 0.001), indicating that the students who accessed the internet at home outperformed their counterparts in the civic knowledge test.

Discussion and conclusions

Discussion

This study complemented ICCS research on DIF assessment involving internet access to civic knowledge and identified six DIF items for the Taiwan sample. Specifically, five of the six items that signify social issues, such as corporate citizenship, asking for advice, government action, environment, and elections, exhibited DIFs and favored students in internet-using families. The other DIF item related to art and political expression favored students in non-internet-using families. These findings confirm that internet access can have both positive and negative impacts on civic knowledge. As an international, large-scale assessment, the ICCS may necessitate additional scrutiny regarding potential cultural and social biases inherent in the DIF items. Given the ICCS's global scope in assessing students' civic literacy, disregarding these DIF items could result in inaccurate conclusions regarding student performance, leading to either overestimation or underestimation.

On the other hand, schools can harness this positive potential by integrating online platforms, interactive simulations, and multimedia resources to enhance students’ learning experiences and deepen their understanding of civic concepts (Apuke & Iyendo, 2018). Furthermore, the government should take proactive measures to ensure equitable access to high-quality educational resources, including the internet. This could involve initiatives to bridge the digital divide by providing internet access and devices to underserved communities (Graves et al., 2021). By addressing disparities in access, policymakers can help ensure that all students have the opportunity to benefit from online resources and participate fully in formal and informal civic learning inside and outside the school. Therefore, parents should be attentive to signs of Internet Addiction Disorder in their children, ensuring that excessive internet usage does not impede their engagement with the real world and meaningful experiences.

Contrastingly, the only DIF item related to art and political expression favored students in non-internet-using families. Political expression encompasses a broad spectrum of activities, including participating in elections, attending protests and rallies, writing letters to elected officials, or engaging in discussions about public policy and governance. In this case, students from families with internet access might spend more of their leisure time on online activities, potentially preventing them from participating in person in the aforementioned activities, thereby losing the opportunity to immerse themselves in political expression through artwork. In other words, internet access can serve as an alternative source to retrieve civic education content and a vehicle for civic knowledge learning (Alshahrani et al., 2017). However, it can also facilitate entertainment-related use (Gruchel et al., 2022), which becomes a negative factor hindering student learning. In this case, parents’ guidance to their children on how to use the internet would be a key area for improvement. For example, parents should encourage their children to manage their time to allow different types of activities, including visits to galleries and museums. Additionally, parents might also direct their children to browse art and political expression online, which can also benefit their learning in this regard. Moreover, the government can support initiatives promoting digital literacy and responsible online behavior through public awareness campaigns, teacher training programs, and curriculum development endeavors. By investing in digital literacy education, policymakers can help equip citizens with the skills they need to navigate the digital world effectively and contribute to civic life positively.

To trace some potential factors for the six DIF items, this study also found that some students’ background variables and internet access could concurrently explain the DIF items, including students’ family socioeconomic status (DIF items “affection by damage to the environment” and “corporate citizenship”), the number of tablet devices or e-readers in students’ homes (DIF item “corporate citizenship”), and frequency of talking with parents about the events occurring in other countries (DIF items “reason to ask for advice” and “democratic government action”). This reinforces the argument that family background can influence students’ learning in many direct or indirect ways.

It has been argued that students’ family socioeconomic status is related to their internet access at home (Graves et al., 2021) and civic engagement (Metzger et al., 2020). With higher SES, students are born with better social capital and all kinds of resources, including internet access and digital technologies (Graves et al., 2021), which ensures their accessibility to more diversified vehicles and sources of knowledge acquisition and learning and thus improves their civic knowledge and educational quantity and quality. This is furthered by their parents’ parenting style (Chan & Mak, 2020). A previous study has shown that parents from the upper class engage more in civic and political conversations with their kids (Oosterhoff & Metzger, 2016). Therefore, the digital divide and the inequality among students are the governments’ and schools’ concerns. As suggested by the literature reviewed in the current study and its results, SES might influence internet access and lead to students obtaining a higher pass rate on the DIF items “affection by damage to the environment” and “corporate citizenship.”

Unfortunately, this study failed to determine potential factors for the DIF item related to “informed voters,” although it has been difficult in the past to trace the sources of DIF for all DIF items (e.g., Allalouf et al., 1999; Ercikan et al., 2004). In short, this study identifies six items as DIF and finds possible DIF sources of the five DIF items (83%).

The difference in civic knowledge between the two groups was investigated after the DIF assessment. The remaining 73 DIF-free items were used in the Rasch analysis with latent regression. We found that Taiwanese students who could access the internet at home showed higher civic knowledge than those who could not, echoing McGrew et al.'s (2019) argument that the internet gap could impede the development of students’ civic literacy.

On the other hand, as an internet-based platform facilitating interaction among people, social media is a collective term for websites focusing on communication, interaction, content sharing, and collaboration that might potentially influence the impact of internet access on civic knowledge. The relationship between social media and civic engagement is apparent (Christensen et al., 2021; Schulz & Friedman, 2019). As reflected in the literature review above, the relationship between civic engagement and civic knowledge is exciting (McCabe et al., 2017). As students rarely use other people’s electronic devices to access social media, internet access might affect their civic knowledge through the use of social media to engage in political or social issues. Due to the complexity of social media issues, which is not the focus of the present study, the effect of social media will be discussed in further study.

Conclusions and future study

This study investigated the influence of internet access on civic knowledge measurement using the ICCS’ global survey data. The findings of this study can be a starting point for further investigation of these items by ICSS to ensure better reliability and validity of the test. However, this study also poses certain limitations. First, only civic knowledge is investigated in this study, rendering the need for examining the skills, attitudes toward, and engagement with internet access in the future. Second, the generalizability of the current findings to other international settings may be limited due to the study data being restricted to Taiwan. Following this, the effect of social media is highly related to internet access, and it is valuable to be investigated in future studies. Its generalizability should be furthered by future research in other countries’ data. Finally, the “informed voters” DIF item will be reviewed and discussed with experts or linked to other background variables to find evidence on the source of inequivalence in civic knowledge measurement in further research.