Develo** a digital game for excel skills learning in higher education - a comparative study analyzing differences in learning between digital games and textbook learning

Abstract

In higher education, many universities in Taiwan let college students learn excel in a self-directed way. The current axle of the Excel curriculum mainly relies on self-directed learning. In the study, we designed the digital game “Legendary Wizard Excel” and took a certified Excel textbook as the research tool. The game we designed integrated the role-play with cognitive scaffolding to help learners learn Excel skills, whereas the textbook we used was “Excel Expert” in the Microsoft Office Specialist. We compared the Learning Effectiveness, Flow Status, and Technology Acceptance Model with 187 college students between two tools, and found that: (1) The game reached a high Technology Acceptance Model; (2) Both groups of learners had significant improvements in learning effectiveness and were engaged in the activity; (3) On learning effectiveness, learners in game-based learning groups achieved higher than learners in textbook groups; (4) Learners in game-based learning groups engaged better in the activity than learners in textbook groups. Therefore, in the future, we looked forward to bringing our results to higher education levels and workplace training to enhance the Excel skills.

1 Introduction

Excel skills are not a major in Taiwan’s schools nowadays, but Excel operation ability is a prerequisite for most advanced job vacancies (1111 Job Bank, 2021). Taking the salary of the information technology assistant in Taiwan as an example, with Excel professional skills, 56.86% of the jobs can have a higher salary than the market salary (1111 Job Bank, 2021). As for the labor market in the United States, a total of 10,103 jobs required an Excel certificate, and 5,205 of their annual salary were more than $100,000 (Credly, 2021).

One of our designers in the research group is a lecturer on the Office software application. While teaching Excel courses in schools, we found some hidden problems. In Taiwan’s higher education, Excel skills are not valued like other professional skills, such as programming and algorithms (National Institute of Education, 2022). Different schools will have different attitudes towards Excel teaching due to their teaching philosophy. Some treated Excel skills or certifications as a graduation condition, while some had no relevant curriculum in the study plan. Otherwise, some seemed to have scheduled, but the actual teaching hours were few, and students had to get the certification within semesters as the final assessment of Excel learning. In addition, because students have not worked yet, it is hard for them to reflect on the situation of using Excel in the workplace. Students cannot understand why they should learn it, and took Excel as unhelpful to their entrance exam. As a result of the above, it is difficult for students to engage in learning Excel, which leads to absence and inattentiveness in class, poor performance in semester evaluation, and even quitting the exam.

In summary, we found that Excel teaching is difficult for students to value (Bangquan, et al. 2012). School students are hard to acquire Excel expertise in a short time, while graduated students who are applying for jobs will have Excel learning needs (Lee et al., 2019). However, there is not much Excel teaching and learning research; among which, researchers suggested applying Game-based Learning to Excel learning, such as task design in Role Play Game mode, and visualization skills can improve students’ effectiveness and interest in excel learning. (Bangquan, et al. 2012; Lee et al., 2019).

Our study hoped to refer to the literature on Game-based Learning, self-directed learning, etc. to design a game for Excel learning. We compare the differences between traditional excel book learning and game-based learning. Therefore, the Technology Acceptance Model research purpose and questions are as follows:

1.1 Research purpose

• 1. Based on the core skills of Microsoft Office Specialist as our learning content, we aimed to develop a digital game that integrates cognitive scaffolding and examination mechanisms.

• 2. Comparing learners in the textbook group, we aimed to explore digital-game learners’ flow state, Technology Acceptance Model, and learning effectiveness.

• 3. We aimed to explore whether there are similarities or differences in students’ flow state, Technology Acceptance Model, and learning effectiveness when learning Excel through digital games and textbooks.

1.2 Research questions

Based on the above research purpose, our research questions are as follows:

• 1. What are the similarities and differences between game-based learning students and textbook-based learning students in their Flow State, Technology Acceptance Model, and learning effectiveness?

• 2. What are the correlations and pathways between each dimension of Flow State, Technology Acceptance Model, and learning effectiveness for students in game-based learning?

2 Theoretical Background

Our study is to develop a digital game for learning Excel that puts game elements and scaffolding together to cultivate learners’ competency in applying Excel. We analyzed and explored whether learners using game-based learning and traditional textbook have the same or different learning effectiveness, Technology Acceptance Model, and flow state. Therefore, we discuss our theoretical backgrounds in the following section, including Excel software application competency, game-based learning, self-directed learning, scaffolding theory, flow state, and Technology Acceptance Model.

2.1 Microsoft excel spreadsheet and education certification

Microsoft Excel is the spreadsheet software in the Microsoft Office suite, which is currently available on Windows, macOS, Android, and iOS platforms. Microsoft Excel has all the basic functionalities that spreadsheets should include. It labeled rows as numbers and columns as letters to organize data content, and it also has basic computing operations (Harvey, 2006). In addition, Excel has a range of functions to meet the needs of statistics, engineering, and finance fields. It can also draw simple charts, Pivot Table analysis which can view data from multiple angles, and Visual Basic Applications, macro set editing, etc. (Harvey, 2007).

Excel is arguably the most widely used spreadsheet software in the world (Cocking, 2017). Satya Nadella, Microsoft’s current CEO, revealed in the technology column that Excel was the most important consumer product in Microsoft, and an estimated about 750 million people worldwide were using Microsoft Excel (Cocking, 2017). According to Cocking (2017), there are seven reasons of Excel become one of the world’s most popular software:

• 1. With Excel software, you can quickly get the data without processing it by IT.

• 2. Users can find Excel is a very flexible tool once they get familiar with it.

• 3. Excel skills have highly marketable values so most organizations use Excel.

• 4. Excel’s problems are usually due to misuse.

• 5. Excel can analyze data from other large systems.

• 6. Excel users can get help from many channels while meeting problems.

• 7. Excel has gradually enhanced the functions of sharing collaborative work.

Workers and students learn Excel skills mainly by reading textbooks and applying for online courses. Self-learning is a process in which a person can self-analyze his or her own learning needs, find relevant resources to develop appropriate learning strategies, and finally assess learning outcomes by his or herself without any others’ help (Knowles, 1975). For the learning resources, every school had different ideologies and emphasis on teaching Excel skills. Some took Excel competence as a general education course; some took it as a required course or professional elective course. According to the different school policies, many universities did not provide enough class hours for students to be familiar with the Excel application. Therefore, self-directed learning became important for Excel learners.

Microsoft Office Specialist (MOS) is an international certification that Microsoft developed for Office suite software. The exam objective for certification of Microsoft Office Specialist 2016 Excel Expert includes four core skills and embedded 15 learning points, for a total of 57 suggested learning items. The Certification Examination is a hands-on, project-based training test, which includes five projects and 26 assignments with a total score of 1000 points. The passing score is 700 points, which entitles the person to a certificate and a digital badge issued by Acclaim, an international notary unit (Microsoft, 2018; Certiport, 2018). Our study was based on Excel Expert items in the Microsoft Office Specialist 2016 edition. The learning contents in this study are as follows (Table 1).

Table 1 Microsoft office specialist 2016 excel expert exam objective

To sum up, the reasons we used Excel learning as a curriculum are as follows: (1) Excel is the competence of tool-operation skills that requires learners to comprehensively integrate what they learned to apply it. (2) Most Excel learners in Taiwan learned by themselves. Self-directed learning needs appropriate guidance and clear study objectives. (3) Excel is a situational and task-oriented tool that is appropriate for game-based learning. The three reasons shaped the importance of Excel learning and explained our difficulties in taking Excel as a curriculum. Hence, our study believed that game-based learning can improve the problems of Excel teaching.

2.2 Game-based learning

Game-based Learning (GBL) refers to a learning way that learners study a specific subject and obtain knowledge by playing games, where learners make decisions, operate, and solve problems while facing kinds of situations (Hogle, 1996; Prensky, 2007; Kiili, 2007) During the gaming process, they can feel a sense of achievement; therefore, they will learn from repetitiveness and integration it with their knowledge (Hogle, 1996). In the game, along with the plot and progress, learners trigger events such as game levels, challenges, decisions, abrupt events, and so on (Kiili, 2007). Learners have to level up their own ability to face events, attempt repeatedly, and ponder the next operation. Through reflections, they can fix strategies, and use the cognitive scaffolding designed in the game to obtain the related knowledge and the contents of the subject (Kiili, 2007; Prensky, 2007).

With the development of technology, Digital Game-based Learning (DGBL) has become a new model of digital learning and has become popular in information technology education (Aldrich, 2004; Squire, 2005; Becker, 2007). Prensky (2007) views digital game-based learning in two directions: (1) The development of digital games changed human thinking so that traditional learning methods met limits. (2) Digital game-based learning can apply not only in the teaching field but also in the enterprise training fields for employees and organizations for suggestions or feedback. Prensky (2001) mentioned a good digital game can attract learners to engage. A good game guides players through the game process and scaffolding hints to learn subject-related knowledge and examines whether learners achieve learning purpose through levels, special events, etc. Prensky (2007) summarized the influence of digital games on learning into five characteristics: Form of Fun, Form of Play, Motivation, Feedback, and Ego Gratification. Digital games allow learners to adjust their learning strategies through repeated operations in the game. In the games, learners can cultivate problem-solving and a higher level of thinking through the process of autonomous learning (Kim & Chang, 2010; Hwang et al., 2012; Hsieh et al., 2015).

Here are some of the examples that apply digital games to logic instructions and skill training. We listed in Table 2.

Table 2 Related studies on integration of game-based learning into curriculum

However, as mentioned in the beginning, there are relatively few studies on the integration of game-based learning into Excel learning. Most studies on Excel only used it as a research tool rather than a learning subject. Lee et al., (2019) applied game-based learning, such as Jeopardy to learn Excel shortcuts and the result found that games can improve learners’ understanding of shortcuts and most of the feedback was positive. Li, Meng, & Qu’s (2012) designed an educational game of Excel in the Role-Playing Game (RPG) model and the results indicated that the game can become more interactive and entertaining than traditional Excel teaching. Under the background of campus life, the game we designed lets students immerse in the story. They must learn skills, upgrade levels, and solve problems by playing the game.

In summary, this study took the above as references and constructed a learning game aimed at learning Excel skills. We explored learners’ experience and effectiveness in the Excel learning game and compared them with the other learning method of textbook learning in this study.

2.3 Self-directed learning

Self-directed learning can refer to active learning, independent learning, self-evaluation learning, or self-orientation learning. Although the terms are slightly different, it depends on what issue the research wants to focus on (Zimmerman, 1995). In self-directed learning, we see learners as an individual, who can learn without and with little others’ assistance. Compared with the traditional learning mode, self-directed learners must undertake more learning responsibilities (Steinke, 2012).

Our definition of Self-directed Learning mainly adopted American scholar Malcolm Knowles’ (1975) idea, and we designed curriculum activities based on the ideology. Self-directed learning is a process of learning, that whether others assist or not, an individual spontaneously analyzes learning needs to form a learning goal, find relevant resources, make appropriate learning strategies, and finally self-assessing the learning outcomes (Knowles, 1975). Pressly (1995) further explained Knowles’ concept, indicating that during self-directed learning activities, the following five points should be executed systematically: (1) Evaluate learning needs according to the criteria. (2) Develop meaningful learning goals. (3) Identify helpful resources for achieving learning goals. (4) Choose appropriate learning strategies for different learning tasks. 5 Examine and evaluate learning effectiveness. Self-directed learning is a process in that learners spontaneously transform external knowledge into skills through their mental thinking. During the process, learners can seek help timely, use learning strategies, set learning goals, and manage time. (Pintrich, 2000; Zimmerman, 2002; Blaschke, 2012).

During our study’s learning process, the instructor is a passive bystander, never actively intervenes in the learner’s learning but provides technical assistance, and gives hints to guide learners who take the initiative to ask questions. In our study, we formulated the learning objectives for Excel certification skills and provided reference books and games as learning resources to carry out self-directed learning activities.

2.4 Scaffolding theory

Scaffolding means that learners get the appropriate support to complete the task or learning. Psychologist Vygotsky (1978) advocated the theory of Zone of Proximal Development (ZPD), which refers to the gap in individual’s problem-solving performance between the actual levels of development by themselves independently and the potential level of development with others’ help and guidance. Zone of Proximal Development is a dynamic zone that is constantly changing along with teachers’ help and guidance, which can assist learners’ cognitive development.

Wood et al., (1976) observed if the adults directly help 3–5 years old children or remind errors, children can gradually complete the pyramid building by blocks. Therefore, teachers’ help and guidance look like the “scaffolding” in building architecture. Learners learn things under the teacher’s support and guidance. After their ability is enhanced, the help and guidance will be gradually reduced, just as a building will slowly withdraw from the scaffolding while progressing.

According to Wood et al.‘s (1976)’ results, scaffolding includes six functions: (1) To trigger engagement: It attracts learning interest; (2) To reduce learning burden: It simplifies learning content and pops out the themes, and learners can focus on it; (3) Clear goals: It plans the learning direction and themes; (4) To highlight key points: It helps learners to distinguish characteristics; (5) To control frustration: It gives feedback on experience through interaction with learners; (6) To demonstrate what they have learned.

Scaffolding is an important topic when designing and develo** educational games. Scholars believe that scaffolding must provide temporary assistance when learners encounter obstacles, and so it can improve learners’ ability (Puntambekar & Hubscher, 2005). In the process of using educational games, learners may not be able to cope with the challenges of some levels, due to insufficient prior knowledge, unfamiliarity with user interfaces, and wrong thinking directions. If the complexity of the learning process is higher than the cognitive load that the learner can handle, it will have serious negative effects on the learner’s comprehension, learning ability, and problem-solving ability (Sweller et al., 1998). Therefore, when the learner encounters obstacles in the game and the scaffolding does not intervene appropriately, it will cause learners unable to continue the game, having negative emotions and frustrations, and unable to achieve their learning purpose.

Scaffolding is very important in the learning process, so how to provide the right scaffolding is teachers’ professionalism. Vygotsky (1978) stated that the scaffolding must meet learners’ needs, otherwise, it will lead to poor learning status and negative emotions, resulting in poor learning effect.

Hannafin et al., (1999) pointed out that in facing different learning situations and teaching environments, instructors and learners will use different learning scaffolding, which includes Conceptual Scaffolding, Metacognitive Scaffolding, Procedural Scaffolding, and Strategic Scaffolding:

• 1. Conceptual Scaffolding: In learning activities, Conceptual Scaffolding can help students to think. Hannafin et al., (1999) emphasized that this Conceptual Scaffolding is very appropriate for the challenge which can be solved in a variety of ways and methods.

• 2. Metacognitive Scaffolding: Metacognitive Scaffolding can guide learners on how to think in the process and help them to develop metacognition.

• 3. Procedural Scaffolding: Procedural Scaffolding assists learners in using learning tools or adapting to a new learning environment.

• 4. Strategic Scaffolding: Strategic Scaffolding mainly guides learners when choosing strategies, and is appropriate for open learning situations such as planning, analysis, and decision-making.

Our study used Conceptual Scaffolding and Procedural Scaffolding in the learning resources, and also configured Metacognitive Scaffolding in the course activities. Through the development of Excel digital education games, we inserted the scaffolding in the process of the game, so that learners can get help when encountering difficulties.

2.5 Game design

Based on the above theoretical background, this study made the game “Legendary Wizard Excel” to be the self-directed learning material for the experimental group. This research is written in the C# programming language, with Visual Studio 2019 as the development platform and .NET Framework 4.7.2 as the development framework for game development. This game is a single-player game.

This game is based on the player identity and plot guidance given by the level. After the player enters the game, relevant materials (such as pictures, external data sources, etc.) will be automatically loaded into the specified folder, and the game will automatically open the Excel file to be edited by the player. Therefore, the player must operate the real Excel Application to complete the task. The excel in the game is not a simulation, but a real application that allows players to complete tasks autonomously. For the operation of the Excel application interface in the game, as shown in Fig. 1.

Fig. 1

Students enter the excel application into the game. Note: On the screen of the students in the actual game, the players operate the real Excel application to complete the level challenge according to the guidance of the situation

The purpose is to help learners learn the 4 core skills in MOS 2016 Excel Expert and familiarize themselves with the exam process so that they can learn from this game to prepare for the certification exam. The game interface configuration refers to the actual test system interface, and its interactive components are cartoonized, and elements such as story background, level setting, prompt function, audio-visual teaching, and scoring mechanism are added to simulate the actual system environment so that users can familiarize themselves with the test process through the game. We referred to Prensky’s (2001) Game-based Learning theory for the game elements and Wood et al.‘s (1976) Scaffolding theory for hel** students learn the Excel application. Through the game levels, learners can not only learn Excel skills but also be familiar with the overall scores of the exam through the game. With the additional examination mechanism, learners can understand their current skill indicators, so that they can strengthen their insufficient abilities.

Fig. 2

Game opening. Note: The image on the left shows the title of the game “Legendary Wizard Excel”, while the image on the right shows the description of the story “In a faraway land, there is a magic tribe where people are born to use magic”

The game uses the magic tribe as the background through narration at the game opening (Fig. 2). In the game scenario, we metaphor “magic” as operating Excel skills and “the elf trial” as the certification exam. In the story, there are a total of five magic islands for players to break through. The narration in the game-opening is as follows:

In a faraway kingdom, there was a magical tribe, where people were born with magic. You were born here too but without magic. One day, the elder Office told you,

“Your MANA was sealed at birth...“

“You can only become a wizard once you find Legendary Wizard Excel and learn magic from him.“

But... finding Legendary Wizard Excel won’t be easy. You must pass the Elf Trial, the Wizard will appear...

Let’s begin the journey.

After the story narration and before going to the first level, players will see the first island light up. Once a level is completed, the next level will be unlocked. The game adopted a one-way design, which is reversible to go back to the past levels that have been practiced. Players will have to pass five islands on the road. There are different elements of Elf Trial for each island waiting for players to challenge: (Fire) Fire trial, (Wind) Wind test, (Sand) Sand tribulation, (Water) Water discipline, (Gold) Pioneering alchemy. The elements islands above are from left to right in Fig. 3.

Fig. 3

The level-opening screen. Note: Five islands show the five different levels: (Fire) Fire trial, (Wind) Wind test, (Sand) Sand tribulation, (Water) Water discipline, (Gold) Pioneering alchemy

Players must pass the Elf Trial and obtain the “Magic Inheritor” identity certificate to unlock the seal. The elf of each island will award the player Identification. See Fig. 4 for the role play settings on each island.

Fig. 4

Role play settings on each Island

Players perform island tasks in order from left to right: (1) Fire Island: The player plays the role of winemaker’s assistant of the winery, and assists the winemaker to complete the data analysis of alcohol concentration; (2) Wind Island: The player becomes a band manager, and checks the financial status of the concert through Excel; (3) Sand Island: The player plays the role of a news reporter, and organizes the related data of the sports winning awards in the Elf Games; (4) Water Island: The player is the person in charge of a gym center, and tracks and manages the cooperation project with manufacturers; (5) Pioneering Island: Players must complete the financial annual report of the Elf Amusement Park.

Our study referred to the related scaffolding views of Wood et al., (1976), Hannafin et al., (1999), and Saye & Brush (2002), and we build the scaffolding hints in the game. Figure 5 is a schematic diagram of the game level; we labeled the interactive component by number to illustrate the functions and scaffolding.

Fig. 5

Schematic diagram of the events in the game and interactive items. Note: (1) work area; (2) the book; (3) dreamland fruit; (4) mission area; (5) redo; (6) next step

Taking learning and entertainment into account, we added task prompt buttons on the screen and fine-tuned and cartooned each component. To reduce the impact of sound and light effects on concentration during the game, the interface configuration of each level is consistent. We only changed the background and added visual effects for the different islands of elements. We listed the scaffolding of each interactive component in Table 3.

Table 3 Function and scaffolding for each interactive component

At the end of the game, we designed the examination mechanism with the “Identity Certificate (Fig. 6).“ The rating points are the same as those of the Microsoft Office Specialist certification. We designed the result screen as the official exam score report, which includes the core skills score percentage and the total scores. The Identity Certificate at the end of the game is divided into four levels according to the degree of scoring: Magic Novice (not passed the certification), Magic Practitioner, Magic Eugenic, and Magic Inheritor (full score). That is, in addition to the lowest level, the other results are certifiable corresponding to the official examinations by Microsoft Office Specialists. We allowed players to cross-reference between levels and core skills to identify levels or skills that they are less familiar with so that they can enhance the training of that knowledge point.

Fig. 6

Identity certificate for game score results

2.6 Textbook material

Since the game content developed in this study contains learning tips such as scaffolding guidance and video-teaching, to prevent the experimental group and the control group from having the same learning method, this study used textbooks as the control group. This textbook was compiled by question-group arrangement. The examples were designed with three levels of difficulty: easy, medium, and difficult. Reference results and problem-solving steps were also included. The core skills of certification are integrated into the examples so that readers can learn additional practical techniques other than the certification content.

There are similarities and differences between the game and the textbook, we list them in Table 4.

Table 4 Comparison between the games and the textbook

The materials we selected for this study were all for core skills in certification and included practical files for learners to use. But there are also some differences. For example, another learning purpose of the game is familiarity with the examination interface and process, while the textbook work on the skills of Excel to deepen and broaden knowledge side, to be close to the application in enterprise practices. The types of scaffolding are also slightly different. The game was based on the initial text prompts and video tutorials taught by professional teachers. The two stages of scaffolding are different. The textbook assisted learning with solving steps by pictures and texts as scaffolding. The biggest difference between the game and the textbook is the examination Mechanism. The game included it, which is what the textbook does not have. So that in the game, learners can better understand the items they are not familiar with and then learn more intensively in this field.

3 Research Method

In our study, we adopted Quasi-experiment Research (Yeh, 2001) with a nonequivalent pre-test and post-test groups design. Our participants were students from five different classes at two universities. We divided the participants into an experimental group and a control group according to their odd and even student numbers. We administered a pre-test before the activity, two groups conducted self-directed learning with different teaching materials during the activity, and asked the participants to fill out the activity-related questionnaire and administer the post-test after the activity. Our experiment flow is shown in Fig. 7.

Fig. 7

The module of nonequivalent pre-test and post-test groups design

Our experimental group was the learners using the game to learn, while the control group was the learners using the textbook. Before the start of the experiment, we explained the overall activity process and the purpose of the experiment to the participants so that they can fully understand the scope of the data usage and processing of the information, and then we asked them to fill in the consent form as well. After completing the consent form and agreeing to the experiment, we tested the learners’ level of prior knowledge before the activity. After the activity, we assessed the learners with the activity questionnaire. Later based on the received data, we compared the learning effectiveness and the difference in their flow states, and the Technology Acceptance Model in our game.

4 Scale Tools

In addition to the pre-test and post-test of the learning assessment, we also used the Flow State questionnaire and the Technology Acceptance Model questionnaire as the analysis tools for the activity experience.

4.1 Learning assessment

To understand the effectiveness of the learning activities designed for this study, a Learning Assessment, which compares pre-and post-tests, will be used. The pre-test and post-test are based on the core skills of MOS 2016 Excel Expert in the Microsoft Office Specialist (MOS) official Microsoft Office certification. Three question types included Multiple Choice, Multiple Response, and Short-Answer. The participants must edit the file according to the task instructions, and then answer questions. One score per task and the total scores are 16. The composition of the pre-and post-test questions is shown in Table 5.

Table 5 The elements of pre- and post-test in learning assessment

Because the pre-and post-tests were completely taken from the MOS book without any modification, the pre-and post-test assessments in this study are tests of high reliability and validity.

4.2 Flow state questionnaire

The flow state scale in our study is based on the scale by Killi (2006). The theory of the scale and the dimensions are described below.

4.2.1 Flow state theory

Flow State refers to the mental state in which an individual is fully engaged, efficient, and creative when extremely focused, and under this status, the individual will not be able to perceive the inner and outer messages. This status is also known as the Flow Experience. American Psychologist Csikszentmihalyi’s (1975) research found that Flow State occurs when an individual reaches a balance between challenge and one’s skills; if the challenge is not balanced with the skills, one may feel anxious or bored (Fig. 8).

Fig. 8

Flow state theory

Csikszentmihalyi (1975) surveyed human daily activities such as painting, composing, playing chess, etc., and inducted four characteristics of individual behaviors in the Flow State: Automatic operation, ignoring time, excluding other things, and feeling of pleasure. Csikszentmihalyi (1996) further divided the Flow State into 9 items: (1) Challenge-skills balance, (2) Clear Goals, (3) Unambiguous feedback, (4) Sense of control, (5) Action-awareness merging, (6) Concentration on the task at hand, (7) Transformation of time, (8) Autotelic experience, (9) Loss of self-consciousness (Csikszentmihalyi, 1996).

Killi (2006) compiled them into the Flow Scale for Game and further divided the above 9 items into 2 main dimensions, Flow Antecedents and Flow Experience, in which the 1-5th items are the Flow Antecedents, and the 6-9th items are in the Flow Experience (Killi, 2006). Our study used the modified Chinese version by Hou & Chou (2012). It is a five-point Likert scale with a total of 22 questions. The overall reliability is Cronbach’s α = 0.936 with high internal consistency. The Flow State Questionnaire used in this study is shown in Table 6.

Table 6 The flow state questionnaire

The Flow Theory has developed many related studies on its correlation between learning effectiveness and flow state. Guo & Ro (2008) argued that in the field of business education, the Flow Experience which is characterized by concentration, control, and enjoyment of activities, can lead to better learning performance. From the information management course in graduate school, Guo et al., (2007) also found Flow State has a positive impact on students’ learning effectiveness and satisfaction. In addition to the influence on learning effectiveness, in the learning and education training in the workplace, the Flow Experience can also let learners explore and do more (Ghani & Deshpande, 1994; Ho & Kuo 2010).

4.3 Technology acceptance model questionnaire

In our study, we explored learners’ Technology Acceptance Model based on Davis’ (1989) Technology Acceptance Model, in which we draw Perceived Usefulness, Perceived Ease of Use, and the System elements in the External Variable.

Technology Acceptance Model (TAM) is used to explore the user’s acceptance of computer science, digital information, and other related technologies. The scholar Davis proposed this model through the Theory of Reasoned Action (TRA) in 1986. His purpose was to find an effective behavior pattern to explain users’ acceptance behaviors of the use of information technology, and to analyze a variety of factors that may affect users. The Technology Acceptance Model provides a theoretical basis to broadly explain the reasons for the low use of technology and predict the influencing factors. Davis (1989) believes that the use of information systems is determined by people’s wishes and attitudes, so he further proposed the framework of the Technology Acceptance Model (Fig. 9).

Fig. 9

The framework of technology acceptance model

To test the learner’s acceptance of the game and the implementation on computers, our study refers to Davis’ (1989) framework of the Technology Acceptance Model Scale, to develop a questionnaire. We divided it into three dimensions according to the content: (1) Perceived Usefulness (PU): It is to explore whether users can improve work efficiency or learning effectiveness by using the information system. This variable will have a direct impact on user attitudes and behavioral intentions. (2) Perceived Ease of Use (PEOU): It refers to whether users feel that the information system is easy to use. It will directly impact the Perceived Usefulness and usage attitudes. (3) External Variables: It refers to all external factors that may impact the user, such as system environment, operation interface, etc. This variable will directly affect Perceived Usefulness and Perceived Ease of Use. Our study used system elements as the external variable. The questionnaire used Five-point Likert Scale, with a total of 10 questions. The overall reliability is Cronbach’s α = 0.909, with high internal consistency. The Technology Acceptance Model Questionnaire used in this study is shown in Table 7.

Table 7 Technology acceptance model questionnaire

With the development of technology and the internet, integrating digital technology into learning and training is a topic many scholars discussed. Chiou (2004) stated that the outcome of digital learning is not the technology itself, but how instructors and learners use it. Therefore, many researchers discussed the relationship between the Technology Acceptance Model and learning effectiveness. Chen & Tsai (2009) found that the Perceived Usefulness in the Technology Acceptance Model will directly affect learning effectiveness. While Salloum et al., (2019) mentioned that Perceived Usefulness and Perceived Ease of Use are the strongest predictors of usage willingness; therefore system developers have the responsibility to provide a useful and easy-to-use system.

5 Research participants

Our 187 research participants were from five different classes in two universities in Taiwan. We take actual university courses for the selection of our research subjects. The courses we choose are general studies at the university, and these students are from different university departments. Following the COVID-19 epidemic prevention regulations promulgated by the Taiwan Ministry of Education at that time, distance teaching was adopted for the first to fourth weeks of the course (see Table 8). Therefore, all students have 4 weeks of an online learning experience, but none of the students have contacted the MOS certification in EXCEL. There were 67 participants’ samples turned into invalid due to the absence of pre-test or post-test, incomplete answers, contradictory positive and negative answers in the questionnaires, and failure to complete it within time. In addition, since this research is performed in an actual classroom, this class is a general education class. In general education courses, students can add or withdraw from courses by themselves. On the day of the study, because some students were absent from school, there was an invalid sample, which affected the difference in the number of people between the two groups. Therefore, we received 120 valid samples to analyze the gender, age, and college distribution. According to Table 5, the participants were aged 18–22, and they were mainly students in the College of Business and Management. The structure of the study participants are presented in Table 8; Fig. 10.

Table 8 Structure of the research subject

Fig. 10

Composition chart of research subjects’ gender, age, and college

For the research reliability of the participants in the experimental and control group, we grouped them by simple randomization to balance the complex factors of both groups, to improve the reliability of the data (Yeh, 2017). We examined the grou** by ANCOVA, with the Pre-test score as the covariance (F = 0.490, p = .485). We found no significance, which indicates that there was no difference in the basic Excel ability between both groups before the experiment.

6 Experiment implementation

We experimented in classrooms on ordinary days. The procedure took about 170 min. They had to finish tasks through the classroom computer (see Fig. 11).

Fig. 11

The student was learning through the scaffolding in the game

We received a total of 187 samples, of which 120 were valid samples (68 in the experimental group; 52 in the control group). We analyzed valid samples’ data and answered our research questions in the following sections.

7 Analysis and result

In this study, we employed SPSS Statistic 27 as the analysis software, and One-Sample T-test to analyze the Flow State and the Technology Acceptance Model to examine whether there were differences in each group’s overall performance and sub-dimensions. We used Independent Sample T-test to compare the difference between the experimental with the control group to see whether it was significant, and examined the learning effectiveness by Paired Samples T-Test of their pre-test and post-test. We used Pearson Correlation to analyze the correlations and pathways. According to the assumptions and results of Hou, Wu, & Chou’s (2014) Path Analysis Model, we predicted the effects between sub-dimensions, then we draw the path through linear regression analysis.

According to the research questions, we investigated the differences between the game and the textbook group among the three dimensions of Flow State, Technology Acceptance Model, and learning effectiveness. We then explored the correlations and pathways among the three dimensions in the game group.

Research Question 1: What are the similarities and differences between game-based learning students and textbook-based learning students in their Flow State, Technology Acceptance Model, and learning effectiveness?

Game-based learning was for the experimental group (N = 68), while textbook learning was for the control group (N = 52). We summed up all the comparisons of the three major dimensions between both groups in Table 9.

Table 9 The flow state, technology acceptance model, and learning effectiveness of the experimental and the control group

7.1 Compare the flow state

Table 9 shows that the Flow State for both groups had reached significant levels. However, the average values of all dimensions in the experimental group were significantly higher than those in the control group (Experimental group: M = 3.94, SD = 0.39; Control group: M = 3.67, SD = 0.53; t = 3.06, p < .01).

Flow Antecedents and Flow Experience had also reached significant level (Flow Antecedents in Experimental group: M = 3.93, SD = 0.45; in Control group: M = 3.65, SD = 0.62; t = 2.79, p < .01. Flow Experience in Experimental group: M = 3.95, SD = 0.45; in Control group: M = 3.70, SD = 0.55; t = 2.73, p < .01).

For the sub-dimensions, “Autotelic Experience” had reached the highest significant difference (Experimental group: M = 3.99, SD = 0.52; Control group: M = 3.64, SD = 0.65; t = 3.18, p < .001). Most of the sub-dimensions had reached significant differences, except for the following two: “Transformation of Time” (Experimental group: M = 3.71, SD = 0.82; Control group: M = 3.57, SD = 0.80; t = 0.93, p > .05) and “Loss of Self-consciousness” (Experimental group: M = 4.15, SD = 0.65; Control group: M = 4.08, SD = 0.67; t = 0.58, p > .05).

7.1.1 Compare the technology acceptance model

Table 9 shows that Technology Acceptance Model for both groups had reached significant levels. However, there was no significant differences between both groups (Experimental group: M = 4.03, SD = 0.64; Control group: M = 3.85, SD = 0.57; t = 1.66).

In the analysis of the sub-dimension, there is no significant difference in “Perceived Usefulness” between both groups (Experimental group: M = 4.09, SD = 0.65; Control group: M = 3.96, SD = 0.61; t = 1.19). However, both groups had achieved significant differences in “Perceived Ease of Use” (Experimental group: M = 3.88, SD = 0.76; Control group: M = 3.51, SD = 0.77; t = 2.58, p < .01).

“System Element” in External Variable had also reached significant differences (Experimental group: M = 4.09, SD = 0.67; Control group: M = 3.67, SD = 0.60; t = 3.58, p < .001).

7.1.2 Compare the learning effectiveness

In the analysis of the individual groups, both groups had reached significant levels of learning effectiveness. The pre-test shows no significant difference between both groups’ overall performance. As for each core skill, both of them do not reach a significant level, which means that the simple randomization we used can balance the known and unknown factors between both groups to improve the data’s reliability (Yeh, 2017).

The post-test results show the significant differences in the overall performance of both groups (Experimental group: M = 12.25, SD = 2.58; Control group: M = 10.04, SD = 3.54; t = 3.80, p < .001). There are also significant differences among sub-dimensions. In the further analysis of the core skills, “Create Advanced Formulas” has the most significant difference (Experimental group: M = 8.13, SD = 2.22; Control group: M = 6.69, SD = 2.54; t = 3.25, p < .001). “Create Advanced Charts and Tables” has the lowest significant difference (Experimental group: M = 0.76, SD = 0.43; Control group: M = 0.62, SD = 0.49; t = 1.75, p < .05).

Research Question 2: What are the correlations and pathways between each dimension of Flow State, Technology Acceptance Model, and learning effectiveness for students in game-based learning?

Our research hypothesized the possible path model as follows.

• 12. 1. The Pre-test scores of learning effectiveness may have predictive effects on Perceived Usefulness and Perceived Ease of Use in the Technology Acceptance Model.

• 13. 2. The Pre-test scores of learning effectiveness, Perceived Usefulness, and Perceived Ease of Use may have predictive effects on Flow Antecedents.

• 14. 3. The Pre-test scores of learning effectiveness, Perceived Usefulness, Perceived Ease of Use, and Flow Antecedents may have predictive effects on the Flow Experience.

• 15. 4. The Pre-test scores of learning effectiveness, Perceived Usefulness, Perceived Ease of Use, Flow Antecedents, and Flow Experience may have predictive effects on the post-test scores of learning effectiveness.

Based on the above four hypothesized path predictions, we analyzed by linear regression and output the significant results of the game group as follows.

Fig. 12

The path analysis of the experimental group

According to Fig. 12, we conducted path regression analysis with Flow Antecedents as the dependent variable, and Perceived Usefulness and Perceived Ease of Use as the independent variables. We found that Perceived Ease of Use (β = 0.543, t = 3.668, p = .000 < .001) has a highly significant effect on Flow Antecedents, and 24.1% of the variation in Flow Antecedents can be explained by the least-squares regression line (R² = 0.241). With Flow Experience as the dependent variable, and Pre-test, Perceived Usefulness, Perceived Ease of Use, and Flow Antecedents as the independent variables, we found “Perceived Usefulness (β = .543, t = 3.668, p = .000 < .001)” and “Flow Antecedents (β = .350, t = 3.062, p = .003 < .01)” has a highly significant effect on Flow Experience. 37.6% of the variation in Flow Antecedents can be explained by the least-squares regression line (R² = 0.376). At last, with Post-test as the dependent variable, and Pre-test, Perceived Usefulness, Perceived Ease of Use, Flow Antecedents, and Flow Experience as independent variables, we found Pre-test (β = 0.637, t = 6.148, p = .003 < .01) has a highly significant effect on Post-test, and 39.6% of the variation in Post-test can be explained by the least-squares regression line (R² = 0.396).

8 Discussion

Research Question 1: What are the similarities and differences between game-based learning students and textbook-based learning students in their Flow State, Technology Acceptance Model, and learning effectiveness?

8.1 Compare the flow state

According to Guo Y.M., & Ro, Y.K. (2008), and Guo et al., (2007), games can enable students to enjoy activities and bring about better learning outcomes for students. In our study, Both groups of learners had a mean score higher than the median on the Flow state questionnaire by 3 (Experimental group: M = 3.94; Control group: M = 3.67). Through the flow state questionnaire, we can show that both groups of learners have engaged in the activity. However, Game-based learners (experimental group) have higher mean scores than textbook learners (control group) in all Flow State dimensions and have significant differences in the overall Flow State (p < .01). This means that Game-based learners are more engaged in the activities that are going on in front of them than textbook learners. This result is the same as Guo, Y. M., & Ro, Y. K. (2008) and Guo et al., (2007), that the learner is focused and more engaged in games can be achieved better in the activity.

Among the significant sub-dimensions, the “Autotelic Experience” is the most significant (p < .001), which means game-based learners enjoy the activity presently more than textbook-based learners. In this way, they engaged more in the activity and acquired more Excel-related knowledge. This is consistent with the results of Ghani & Deshpande (1994) that learners explore Excel more to complete the challenge during the process. This process is similar to the self-directed learning defined by Knowles (1975). In the game, learners constantly try and make mistakes, revise and adjust their learning strategies to achieve their learning goals.

“Transformation of Time” and “Loss of Self-consciousness” do not reach significant levels, which indicates that the performances of the two groups have no significant differences. We assumed that both groups felt their perceptions of time changed during the activity (Transformation of Time), and had no desire to feel other external unrelated things (Loss of Self-consciousness).

Through qualitative observation and analysis, we found that (1) The students in the game group are more attentive, and they will try to apply the game situation to solve problems, which is in line with the experience that games can flow (Csikszentmihalyi, 1975). (2) Although the students in the book group know how to use tools better, without the scaffolding in the game, the students must solve problems by themselves, and the experience of guidance is lacking.

8.1.1 Compare technology acceptance model

In all sub-dimensions and in overall, there is no significant difference between the experimental group and the control group. Based on the analysis of each sub-dimension, we assumed that both groups can acquire Excel knowledge from their materials. In addition, both learners reach significant differences in “Perceived Ease of Use” (p < .01) and “System Element” (p < .001), which may indicate that learners find the system design of the digital game more user-friendly and easier to work with than the textbook-based materials while comparing the materials with the real Excel interface. Such results are consistent with the views of Chen & Tsai (2009), indicating that Cognitive usefulness in technology acceptance directly affects learners’ learning outcomes.

Through qualitative observation and analysis, we found that the students in the book group and the game group were able to complete the task independently without any problems, and the observation results were the same as the results of the questionnaires filled out by the students.

8.1.2 Compare learning effectiveness

We found that the learning effectiveness in both groups reaches significant levels. It indicates that whether the material interface is a gaming system or a textbook, their Excel skills can improve effectively after learning.

We found that the game-based learners’ have significantly better performance than the textbook learners’. We can assume that Self-directed Learning via the game can receive deeper and more accurate assistance through the interactive component, which is particularly prominent in the core skill of “Create Advanced Formulas” (p < .001). It is possible that, while learning to create formulas in Function and Logical Thinking, users can absorb the knowledge clearly by the hint button, “Dreamland Fruit” (item 3 in Fig. 5), to watch the video instruction. The results are similar to Prez, Duque, & Garca’s (2018) finding that students applied game-based learning to learn logical mathematics better.

The core skill of “Create Advanced Charts and Tables” reaches significance but has the lowest level of significance compared to the other skill (p < .05). We assumed that although the game provided the operation instructions and the presentation of the results via videos, the textbook also provided text descriptions and step illustrations to assist learners. It means that both materials can allow Self-directed learners to self-examine the core skill of “Create Advanced Charts and Tables.“ As researchers Hannafin et al., (1999) mentioned, learners can cross-refer the practiced results with the reference results, and the examination results can further lead them to think and solve problems. Therefore, the learning effectiveness of both groups is comparable and so the significance is lower than other skills.

Research Question 2: What are the correlations and pathways between each dimension of Flow State, Technology Acceptance Model, and learning effectiveness for students in game-based learning?

According to the path analysis diagram in Fig. 12, we found the Pre-test positively influences the Post-test, which indicates that the learners’ basic scores can be improved. The Perceived Ease of Use positively influences the Flow Antecedents, indicating that the more fluent the game interface for learners to operate, the better game experience they can have. Perceived Usefulness and Flow Antecedents positively influence the Flow Experience, indicating that once learners have a good experience and absorb new knowledge in the game, they can engage more in the game. The result is consistent with Salloum et al., (2019) that when a digital learning system has practical utility and is easy to use, learners’ will increase their willingness to use it, which in turn improves their learning effectiveness. These results are also the same as our qualitative observations.

9 Limitations

Due to research needs, this study adopted a simple random grou** method to determine the learning materials used by learners in activities. In self-learning defined by scholars Knowles (1975) and Pressly (1995), learners should establish appropriate learning resources and make learning strategies for learning with or without the help of others, rather than specifying teaching materials by others. Therefore, the learning materials assigned to the learners in this study may not be able to meet the learners’ learning styles or needs.

In addition, due to the arrangement of the school’s class hours, each class has only 2 classes per week, with a total of 100 min of class hours. Therefore, the learner may not have enough time for the second game and book practice.

Based on the above limitations, we hope to improve the limitations in the future, allowing students to (1) choose which group they want to study in and (2) give students more time to study.

10 Recommendations

Through our research, we have come up with some findings that can be used for future reference. Here are our recommendations:

• 1. Our game can apply to other related software applications, such as Word, PowerPoint, Access, etc. Based on our design, developers can then design games for different skill examinations with the official core skills demanded by schools, institutions, or enterprises.

• 2. We suggest that it is possible to integrate the strengths of various fields into game-based learning or apply it in assessment. Therefore, to turn the game into a tool to assist learners, the classroom can be more fun. By adding user profile managing functions, learners can keep their learning history by e-mail.

• 3. Flipped Classroom becomes popular in recent years, which allows for self-directed learning in the classroom. It can break the traditional teacher-oriented model and stimulate students’ interest in self-directed learning (Bergmann & Sams, 2012). Our model can more commonly be applied to enterprise education and training, workshops, and other practical courses in Excel teaching. The campus side can bring this game and use the flipped classroom method to attract students’ interest in self-directed learning. Through our game, they can understand the practicality of Excel in the workplace.

• 4. We expect to bring Game-based Excel Learning into enterprises. Through workshops, talent training, and employee education, we can provide a way of Self-directed learning with entertainment in the high-pressure employment environment, and allow the employee to improve their skills in play.

• 5. In the future, if there is any relevant research, we can design to increase the analysis of learners’ behavioral patterns, motivation, and autonomy, and we can conduct audio and video recordings of the activities, as well as conduct more studies on different levels through questionnaires and interviews.

11 Conclusion

Our study purpose is to help learners to prepare for Excel certification exams more efficiently through game-based learning, so we developed the Excel learning game “Legendary Wizard Excel.“ We concluded our findings as follows.

• 1. Through the technology acceptance results of the experimental group, it was found that learners’ Technology Acceptance Model in the game “Legendary Wizard Excel” has positive significance and feedback.

• 2. Through the analysis of the flow status and learning effect of the two groups, it was found: that in terms of overall flow state, the mean of both groups was 3 above the median (Experimental group: M = 3.94; Control group: M = 3.67). This represents that whether learning through the game or textbooks, students can be engaged better in the activity and have learning effectiveness.

• 3. Through the comparison of the learning effect between the two groups, it was found that game-based learning is more effective than the textbook.

• 4. Through the comparison of the flow status between the two groups, the mean of the overall flow state for both was higher than the median 3, but there is still a significant difference (p < .01). This means that Game-based learners are more engaged in their learning activities than textbook learners.

It is hoped that through this research, we can make more research pay attention to the significance of tool self-learning, and use games to assist learners in learning Our research hopes to provide the Game-based Excel learning model as a new option and reference for software learning, workplace skill development, and related fields.