Introduction

The vertiginous development of technology in recent decades has surprised humanity with various proposals to improve the quality of life, which, of course, extend to the educational field. Such is the case of the different Adaptive Learning (AL) platforms developed to improve the learning process. According to Moskal et al. (2017) AL Platforms. Use a data-driven—and, in some cases, nonlinear—approach to instruction and remediation. They dynamically adjust to student interactions and performance levels, delivering the types of content in an appropriate sequence that individual learners need at specific points in time to make progress. These systems employ algorithms, assessments, student feedback, instructor adjustments/interventions, and various media to deliver new learning material to students who have achieved mastery and remediation to those who have not. These platforms support the massive application of this adaptability, which saves the professor’s time compared to doing it manually thanks to an infrastructure or specialized equipment to analyze each student’s information.

Adaptive learning platform

One of the educational pillars of the university where this research occurred is flexibility, i.e., offering the students meaningful options regarding what, when, and where they learn (Tecnologico de Monterrey, 2016). Therefore, the educational model enabled the construction of an ecosystem of educational technologies that support delivering personalized experiences to students and professors. This ecosystem of innovative educational technologies can improve teaching–learning and deliver personalized digital experiences. One of the components of this ecosystem is the delivery of content and learning resources; this is where technologies related to learning platforms and the management of learning resources play a relevant role. In this sense, the researchers in this study identified the need to look for technologies that allow personalization, such as AL Platforms. In the market, several providers offer platforms that describe incorporating AL; however, evaluating these platforms detects different ways to implement and adapt them. Some platforms offer Adaptive Content and do not require or do not allow the editing of such content; others have authoring functionality to create content and incorporate the Adaptive Learning Strategy (ALS) with the author’s content.

For the implementation of this strategy, AL Technologies were evaluated that could offer a better learning experience to students through an AL platform that was intuitive and interactive, from which they could obtain feedback in real-time and that generated a Personalized Learning Path. For the professors, it should mean support in their teaching activities. The requirements were as follows:

Student experience

The diagnostic evaluation results indicated that the student must obtain feedback and content recommendations through continuous assessments throughout a personalized content path. There must be the possibility of evaluating the resources and accessibility of carrying out exercises or activities that the professor establishes for learning topics.

Professor experience

The professors must obtain the necessary information to monitor the students’ progress and easily adjust the suggested contents or make interventions.

Curriculum design/syllabus

Functionalities should allow generating a map of competencies or objectives of the course, its relationships, requirements, prerequisites, and expected learning.

Adaptability

For recommendations to the student for the best content according to their performance using some algorithm such as Machine Learning or Artificial Intelligence.

Reports and analytics

The platform should produce student consumption and progress reports that give the institution information for continuous improvement.

Content types

The platform should support multiple formats such as HTML, multimedia (images, audio, and video), video game activities, scripts, and links to external sites.

Finally, after assessing 14 providers, Realizeit technology was selected because it offered the highest level of functionalities; its distinctions compared to others included:

  • Personalized Learning Paths: Each student has an individual path.

  • Artificial Intelligence Algorithm: Capable of analyzing student demographics, performance (pre- and on-course), and preferences; adaptable to student assessment performance.

  • Formative and summative assessments: Offers different types of parameterized questions and exams with immediate and personalized feedback and evaluation rubrics.

  • Real-time analytics for students and professors to track progress in Personalized Learning Paths. Institutional analytics and recommendations for improvement at the end of the course.

  • Learning content: Allows various types and formats (HTML, multimedia, documents, and open resources).

  • Integration with the Learning Platform and the ecosystem of educational technologies, making it easier for the grade obtained in the AL Platform to transfer to the course grade freedom (CGF).

The ALS addressed in this research developed from the data generated in the student’s interaction with the contents and activities previously developed in the AL Platform for data analysis to create Personalized Learning Paths (see Fig. 1). The analysis identified the students’ needs, strengths, and areas of opportunity through a diagnostic exercise, providing different alternative content options for advancing through the class topics according to their level of learning.

Fig. 1
figure 1

Learning Map’s view in realizeit adaptive learning platform

Full size image

The ALS’s design aims to support the students’ performance and experiences and enrich the professors’ teaching processes. The selected courses incorporating the strategy met the criteria of having complex content, low grades, or knowledge leveling.

Didactic model using adaptive learning

DMUAL follows a general model that combines activities before and after the class and actions that occur during the class. The lower part of the model represents the lower-order cognitive processes of Bloom’s Taxonomy revision (Anderson & Krathwohl, 2001), where the activities concentrate on using the AL Platform to review information through digital resources such as videos, readings, podcasts, review of developed examples, and resolution of exercises with automated feedback (see Fig. 2).

Fig. 2
figure 2

Combining adaptive learning in an Active classroom

Full size image

With the previous study of the AL Contents, the student can actively participate during the class in activities that involve higher-order cognitive levels such as problem-solving, elaboration of complex exercises, and resolution of challenges or problem situations.

Finally, given economic, human, and time resources costs, it was decided to evaluate the impact of ALS on the level of learning, achievement of sub-competencies, and students’ learning experience. The ALS involves develo** and designing activities that feed the AL Platform, didactic strategies, and follow-up with professors and designers. The Methodology section provides additional details.

Theoretical framework

Adaptive learning

To speak of AL, one must first speak of Personalized Learning as an encompassing educational strategy. Personalization of learning is more of an "umbrella" that covers several approaches and models, including Competency-based Learning, Differentiated Instruction, Tutorial Models, and AL (Observatorio of Institute for the Future of Education, 2014). Schmid and Petko (2019) pointed out that international research literature shows that Personalized Learning is a multilayered construct with numerous definitions and implementation forms. Niknam and Thulasiraman (2020) argued that educational society has been interested in having a Personalized Learning System that adjusts the pedagogy, curriculum, and learning environment for learners to meet their learning needs and preferences.

The authors define the term Personalized Learning as an educational approach that adapts each student’s learning based on their individual needs, strengths, abilities, and interests, providing them flexibility in what, when, how, and where to learn during their curriculum and various training experiences. Different strategies and technologies provide degrees of autonomy and choices for the students to own their learning process in these settings.

According to Smyrnova-Trybulska et al. (2022), AL consists of Adapting the Learning Process according to the student’s needs, competencies, and abilities. Morze et al. (2021) declare that Adaptive Learning technologies aim to provide students with the means to acquire information according to their training needs and cognitive differences. Lagubeau et al. (2020) state that these technologies also promote active learning, and González Fernández et al. (2018) indicate that they also promote self-regulation. Students take control of their learning process; they can access learning resources according to their needs and study them at their own pace.

Liu et al. (2017) state that most educators recognize the advantages of AL, but evidence-based research is limited as AL is still evolving. Moskal et al. (2017) indicate that for students, AL respects their prior knowledge, responds to their learning needs, and reduces gaps in their understanding. By ensuring that students attain mastery before moving on, AL avoids “teaching to the middle”, which fails to consider advanced or lagging students. Meanwhile, instructors can more easily monitor which students need assistance, measure curriculum performance, and maximize learning outcomes.

Kara and Sevim (2020) claim that these systems employ algorithms, assessments, feedback, adjustments, instructor intervention, and different means to deliver new learning materials to students who have reached the desired level or provide remediation to those who have not. Regarding the instructors, according to Moskal et al. (2017), they can have a better sense of content areas where students are struggling, and system metrics allow intervention before individual students are at risk of withdrawal or failure. In these ways, the role of the instructor changes from the content provider to the learning facilitator. However, to achieve these efficiencies, instructors must know how to use the systems properly, say Cavanagh et al. (2020).

The authors define Adaptive Learning as an educational strategy that uses technology based on data analytics to adapt education and create a Personalized Learning path whose contents were previously designed by the professor to be effective and efficient for each student based on their performance level, profile, and learning needs. This allows the professor to identify gaps in the group’s understanding to establish improvement actions and adjust their educational practice, optimizing student performance. In addition, the ALS can coexist with other strategies, techniques, and methodologies, such as the following.

Flipped classroom

Bergmann and Sams (2012) indicate that "flip**" the classroom establishes a structure to ensure students receive a personalized education tailored to their needs. Lagubeau et al. (2020) indicate that Flipped Learning aims to optimize class time by promoting active learning. Van Alten et al. (2019) say that some studies have found that using Flipped Classroom strategies improves learning compared to traditional teaching methods. However, these studies emphasize the importance of having an adequate design of activities during classes to succeed.

The application of Flipped Classroom considers the following aspects, known as the four pillars of Flipped Learning (Flipped Learning Network, 2014): (a) Flexible environment: Creating flexible spaces allows the students to choose when and where to learn and the facilitator to choose where and how to apply learning assessments. (b) Learning culture: The Flipped Learning Model deliberately shifts instruction to a learner-centered approach, where in-class time explores topics in greater depth and creates rich learning opportunities. As a result, students are actively involved in knowledge construction as they participate in and evaluate their learning in a personally meaningful manner.

(c) Intentional content: Educators use intentional content to maximize classroom time to adopt student-centered, active learning strategies, depending on grade level and subject matter. (d) Professional educator: During the class, the educator should observe, provide feedback, and continually evaluate the student’s work. Professional educators reflect on their practices, connect to improve instruction, accept constructive criticism, and tolerate controlled classroom chaos.

The application of elements of the Flipped Classroom predominates in the delivery of the ALS by allowing students to perform exercises and review the contents of their classes beforehand, creating personalized paths to bring their questions and contributions to the classroom and resolve issues with their professors and classmates.

Self-regulation of learning

Another critical element in the ALS is self-regulation, which is the ability to control and manage thoughts, emotions, and actions through personal strategies that allow both the achievement of objectives and the avoidance of undesired results. It is noteworthy that self-regulation allows the analysis of the environment, responds to it, and modulates the consequent reaction to adapt to the environment. This ability significantly affects the individual’s personal development, social adjustment, and general well-being. It is essential to point out three areas in which self-regulation manifests: behavior, learning, and emotions (Castro & Gallardo, 2021).

Self-regulated Learning is a process of self-reflection and action in which the learner structures, monitors, and evaluates his or her learning. Self-regulated Learning is associated with better content retention, greater engagement with studies, and improved academic performance (Ganda & Boruchovitch, 2018).

There are different models of Self-regulated Learning, each dealing with different phases and activities. However, Zimmerman (2013) and Pintrich (2000) identified common meanings among the different models: first, most models assume that learners are active in constructing their meanings and goals and are influenced by various factors in the environment and their cognitive system. Second, individuals can monitor and control learning’s cognitive, motivational, behavioral, and contextual aspects. Third, regulation can be driven or facilitated by intra-individual factors (biological and developmental, for example) and extra-individual, contextual influences. Fourth, Self-regulated Learning Models emphasize the individual’s ability to set learning goals and monitor their learning against these goals through control processes influenced by assessment outcomes. Finally, these authors position Self-regulated Learning as a mediator between personal and contextual influences and actual learning performance.

Self-regulation is closely related to AL by allowing students to take control of their learning and tailor it to their needs. In contrast, adaptive learning offers the possibility of creating a personalized and effective pathway by using technology to tailor the content and activities the student can perform, so both help achieve academic success.

Microlearning

Microlearning is learning through small, well-planned modules and short-term learning activities (Allela, 2021). It is an activity-oriented approach capable of providing learning in small parts (Skalka et al., 2021), allowing a regular rotation of microcontent and micro-activities; these authors specify that microcontent is usually short text, sometimes enriched with images, tables, diagrams, or source codes, while micro-activities require user interactions. Among the primary needs addressed by Microlearning is the search for training strategies that avoid affecting the cognitive load or attention threshold that the student may have. Similarly, it is associated with strategies for the development of training processes in areas of knowledge subject to permanent changes (Allela, 2021).

For Göschlberger et al. (2019), the design of a microcontent should consider that it is self-contained and self-explanatory, features a single learning activity, is usually achievable in a matter of seconds, and provides immediate feedback on performance.

Trabaldo et al. (2017) comment that Microlearning materials are brief, continuous, contextual, gradual, informal, and granular. Brief: These are microcontents of information with short tasks. Continuous: The contents are flexible, can be accessed every time a concept or procedure needs remembering, and are assimilated in the long term. Contextual: Microlearning is distributed in diverse contexts and with technological tools appropriate to the situation and circumstances. Gradual: Microcontents within the capsule are presented from simple to complex. Informal: It favors informal learning based on concrete information to support decision-making or skills acquisition. Granular: Microcapsules are interconnected to generate new learning.

The Microlearning approach presents an alternative to create didactic sequences for a specific topic, with small content units that can be consumed quickly in micro time frames. These learning pieces can be connected, assembled as single puzzle pieces to build in their unique expression as resources, in a minimal expression as learning objects, and in a more comprehensive construction as learning paths that allow for personalization and adaptability of learning.

Delivery model

Before implementing a course with AL, the professor receives training on the general strategy, the didactic and delivery model, and information on using the Adaptive platform. The delivery model comprises three moments. Within each one are activities to be carried out: (1) Before the class: the professor monitors the learning analytics and adjusts the interaction instruction based on the group’s level of understanding and the strategies available. (2) During the class: the professor applies Active learning strategies, takes content from the AL to support instruction, and gives instructions for pre-study for the next class. (3) After class: the professor provides personalized follow-up to students who require intervention and again monitors analytics to adjust instruction (see Fig. 3).

Fig. 3
figure 3

Original model of application of the adaptive learning strategy

Full size image

Combining AL and the element of the Flipped Classroom allows the benefits of both strategies to enrich the teaching–learning process. With the previous study of the digital resources published in the AL Platform, the students go to class prepared by studying the content, practicing with exercises with automatic feedback, and taking to class what is not understood, which allows them to participate actively in the classroom and receive personalized follow-up per their progress. For professors, having learning analytics reports on the progress of their class and students makes it easier to make timely decisions during the process and adapt instruction based on specific learning needs.

Adaptive learning in higher education

In Engineering Education, different application lines are found in the ALS. One of these lines of applying AL is supported by data analytics of the student’s level of knowledge or skill when they are asked a question. This modality analyzes and categorizes the error, concept, or repetition of incorrect answers to the topics reviewed. Based on this data, intelligent Machine Learning algorithms allow teachers to suggest a new line or route (see Fig. 1) that allows the student to strengthen his deficiencies and gradually achieve his performance objectives (Waters, 2014). This first line, the adoption rate has grown commercially, and various providers of digital resources offer platform and monitoring dashboard services or delivery of personalized content supported by the results of diagnostic points in the learning process. We can find some reports on the impact of these commercial platforms, such as the intelligent tutorials report by Weltman et al. (2018), who evaluates the impact of Adaptive Content using the Smart Sparrow platform on learning. Matayoshi et al. (2021) show the effectiveness of Adaptive Algorithms on the ALEKS platform by evaluating the learning of mathematics concepts. Conklin (2016) documents the advantages and customization models of the Knewton platform, which relies on customization elements to improve learning.

A second line of development of AL is "facilitator-driven" at the orientation involving adaptation from the professor’s actions. This line aims to generate the necessary actions to adapt to the student’s profile, thereby improving learning. This modality is supported by dashboards, indicators, and records of the progress of a population or group of students, allowing the personalization of the process in particular cases or a target population. The adaptive teaching processes stem from the professors’ actions; different authors have studied them and have shown their efficiency. The Adaptive Teaching Processes based on the professors’ actions have been used in a group manner, perhaps based on their difficulty in achieving an individual impact on students. Under this modality, reports on the application of adaptive teaching published by Rincon-Flores et al. (2022) show how Adaptive Learning Algorithms, through performance forecasts, allow generating predictions to influence students’ positive and negative trends in the learning process. The process shows how, using AI algorithms, such as Random Forest, it is possible to train this algorithm to predict the performance of a population of students, supported by historical records of previous groups under the same activity and evaluation schemes. The evidence reflects that they suit the professor’s actions and improve student learning. Figure 4 shows the gradual training process of the Random Forest algorithms that are the basis for performance forecasting of the population under study.

Fig. 4
figure 4

AI algorithm training in Adaptive Teaching Course. Algorithm Training with a Quizzes. b Quizzes + Homework (HW). c Quizzes + HW + Students surveys to evaluate professors (ECOA). d Quizzes + HW + ECOA + Biometrics (Rincon-Flores et al., 2022)

Full size image

The model supported by neurocognition measurements is found in recent studies of the ALS under the professor’s actions. At Tecnologico de Monterrey, Olmos López et al. (2018) have incorporated models and experiences to observe and record the cognitive response of students to learning stimuli, producing an innovative AL modality with biometric measurements for the best characterization of the learning profile within the ALS. The Adaptive Learning Model supported by the neuro-cognitive response seeks to relate specific parameters involved in the learning process (Brusilovsky & Peylo, 2003) and the student’s learning profile. This process occurs through correlation algorithms and the classification of decision trees from unassisted deep learning (Crockett et al., 2011) (See Fig. 5). In addition, Artificial Intelligence supports the classification process and tries to search if there exists a classification pattern from, or coincidences with, historical data (Polson & Richardson, 1988).

Fig. 5
figure 5

Algorithms: Iterative and unassisted random trees for classification and determination of data set patterns

Full size image

The algorithm seeks a regression model from the student data that can serve as the basis for the study and who can be expected to be in performance groups with distinctive characteristics. Figure 6a shows the graphs of the unclassified student data, while Fig. 6b displays the data of the students already grouped in performance clusters.

Fig. 6
figure 6

Student clustering data in an Adaptive Learning Course. a Students without clustering, b Student performance clusters

Full size image

Finally, other experiences and efforts to develop Adaptive Tools are expert systems that systematically offer a primary degree of personalization supported by logical performance indicators to make learning reinforcement more efficient in some aspects of difficulty in the courses. Jaquez et al. (2015) developed an “Adaptive training platform for Tec Eval science courses”, where an online diagnosis and evaluation process occurs through binary decision trees, classifying the relevance of the knowledge acquired by the student. The platform offers a learning route, which is modified depending on each topic’s responses generated in the evaluation process. Defining a student’s learning profile is just as relevant as the development given to adaptive learning. Crespo et al. (2014), in a Novus proceeding, proposed an "Intelligent Platform for determining learning styles of TEC students for use in adaptive learning" to recognize student profiles as an element in course actions to improve learning.

AL Models aim to personalize the content review in a course and thus assign improvement activities to the student, identifying the learning needs that generate personalized remediation related to the student’s learning profile (Noguez et al., 2013). In the different cases shown as alternatives for the ALS in engineering courses, the constant is collecting performance data and identifying performance patterns and trends supported by forecasts or results analytics from previously analyzed populations (Olmos-López et al., 2018).

One of the goals of adaptive learning is to shift the teacher-centered model to one that focuses on the student. Therefore, one of the key elements in its implementation is to use the flipped classroom, so that the teacher can optimize teaching time and student attention (Alamri et al., 2021). In the study by Contrino et al. (2024), the CogBooks platform was used for university students to review content and complete activities. Then, the teacher reviewed the analytics, allowing them to design and plan the class according to the results, thus adapting the class based on CogBooks’ outcomes. Similarly, in the case presented by Olmos-López et al., (2023), an algorithm based on decision trees, trained with biometrics and academic history, was used to predict student performance, and based on this, teachers designed adaptive strategies. However, there was no access to automated content and assessments.

Thanks to technological advances, particularly the development of AI, the use of adaptive platforms is expanding in various countries, such as the United States and China, where millions of unique users per platform have been recorded in recent years (Wang et al., 2020). In this regard, it is important not only to evaluate the impact of Adaptive Learning on the teaching–learning process but also to simultaneously identify the key elements of an effective and successful delivery model. Therefore, as a result of this research, this manuscript offers a measurement of the impact of ALS as well as the key elements of the delivery model.

Study objective and research questions

The main objective of this study was to determine the impact of ALS on the teaching–learning process in basic science courses of engineering careers assisted by an adaptive platform, where, based on the results, a teaching model that incorporates Adaptive Learning can be offered.

The following research questions guided the study:

  1. 1.

    What is the impact of the educational strategy of AL supported by an AL Platform on the learning level of students in four courses of the School of Engineering?

  2. 2.

    What is the impact of the ALS supported by an Adaptive Platform in achieving the disciplinary and transversal sub-competencies of the students in four courses of the School of Engineering?

  3. 3.

    What was the experience of the ALS by students and professors based on usability, teaching, learning, engagement, and user experience?

  4. 4.

    What are the didactic elements to consider when incorporating ALS into the teaching–learning process?

Methodology

Methodology approach

The study employed a methodological approach that mixed sequential Quant- > Qual so that the qualitative results explained the quantitative ones (Creswell & Poth, 2016). Likewise, the study was quasi-experimental with control and experimental groups. The research occurred in two stages. The first occurred in August–December 2022, and the second in February–June 2023. In the first stage, the professor and content variables were controlled in all the participating courses; the same professor oversaw the control and experimental groups. In the second stage, this was only possible in one course. However, care was taken that the professors of the other courses had similar teaching experiences. The participating courses were Computational Thinking (10 weeks long), Physics I (10 weeks long), Physics II (5 weeks long), and Fundamental Mathematical Modeling (10 weeks long), in both stages 1 and 2.

Sample

In both stages, the sample was non-probabilistic, as the selection of teachers and groups of students was not random (Dahlberg & McCaig, 2015). The selection decision was made by the regional directors of the science departments of the various campuses who voluntarily decided to participate in the project. In the first stage, 1281 first-semester students and 24 professors from the School of Engineering participated. Stage 2 involved 230 students and five professors from the Engineering school.

Data collection techniques

To answer the first research question in stage 1, a pre and post-test was applied to measure the students’ learning level in both the control and experimental groups. The pre-test aimed to determine whether the groups of both treatments had the same knowledge baseline, while the post-test allowed us to compare both groups after the intervention, as well as to measure learning gain. The development of these instruments and their piloting are explained in Sect. "Data analysis" of this section.

To answer the second question, the researchers collected evidence of learning in both treatments to measure the achievement of the sub-competencies, which were evaluated by a professor external to the project. The knowledge evidence consists of a final project in which students integrate the contents of the course to solve a real problem. These activities are pre-designed, so teachers who participated in the ALS intervention selected the same learning evidence for each course.

To answer the third question, a questionnaire was applied to the students of the experimental group to evaluate the learning experience regarding usability, learning process, engagement, user experience, and teaching process. Validating this instrument yielded a Cronbach’s Alpha of 0.90, in addition, four focus groups were applied to the professors, one for each course. The same instruments were applied in the second stage except for the focus groups, which had semi-structured interviews.

Finally, to answer the fourth question, after the results of both stage 1 and stage 2, the best courses evaluated by students and professors were selected to determine the elements that contributed to their success.

Project development and educational intervention

The Adaptive Platform’s content planning and design process occurred between January and June 2022. Figure 7 shows the project development process, from planning to impact assessment.

Fig. 7
figure 7

Project Timeline

Full size image

Stage 1 involved five campuses in different cities, while Stage 2 involved only one campus because it involved an irregular semester. Notably, some of the findings and recommendations found in Stage 1 were applied in Stage 2; those not applied would be applied in Stage 3.

Instrument piloting

During the planning and design stage, the professors designed diagnostic tests, which would be part of the instruments for collecting information. The research team validated them; the process is detailed next.

A Multiple-Choice Question (MCQ) test served as the pre and post-measurement of the students’ learning level in each course selected for the present study and after following the next steps.

  1. 1.

    Professors attended a four-hour workshop to write four-choice items with one correct answer. Specific educational objectives of this training were twofold: the first objective was to identify the concepts of assessment construct (López, 2013) and specification matrix (Kubiszyn & Borich, 2003), and the second was to write multiple-choice questions. The participants developed the test specification matrix during the workshop based on Bloom’s Taxonomy revision (Anderson & Krathwohl, 2001). They also received several recommendations on how to write multiple-choice questions correctly. The target MCQ test included 16 items since this number of questions were sufficient to evaluate the desired learning objectives in 30 min or less. A first draft of the MCQ test was produced at the end of the workshop.

  2. 2.

    Next, professors worked autonomously to complete the first version of the items. The professors were advised to design more than 16 items, as each would be validated qualitatively and qualitatively. Therefore, the participating professors wrote at least 30 items for each of the designed MCQ tests.

  3. 3.

    Subsequently, two rounds of qualitative validation considered the 31 criteria suggested by Haladyna et al. (2002). During this process, the professors revised the items regarding their content, format, stem, and answer options; they received classifications as either high, middle, or low quality. Consequently, professors were advised to revise the structure of the items rated as middle, while low-quality items were discarded.

  4. 4.

    The next step was the test’s quantitative validation. A group of students with similar characteristics to those participating in the study was invited to answer the adjusted test version. Results were examined using Cronbach’s Alpha, discrimination, and difficulty indexes. Table 1 displays the results from the quantitative validation of the test.

  5. 5.

    Items were modified to increase the test quality for reliability, discrimination, and difficulty. Next, the final version of the test was uploaded to Canvas so students participating in the study could answer it before (pre-test) and after (post-test) the intervention.

Table 1 Quantitative validation of the test
Full size table

Data analysis

Several data analyses were conducted in SPSS 29 to answer the research questions in the present study. To answer research questions one and two, the researchers performed One-Way ANOVAs to compare the students’ learning level and their development of sub-competencies, respectively. To answer research question three, they calculated descriptive statistics such as mean and standard deviation for each dimension in the perception questionnaire.

Ethical aspects

During the investigation, the anonymity of the students was preserved, as well as that of the professors, at no time were sensitive data published.

Results

The results are presented below based on the research questions with descriptions of the adjustments made in the second stage.

Results of research question 1: what is the impact of the ALS supported by an AL platform on the learning level of students in four courses of the school of engineering?

Quantitative results

The pre-and post-tests of course knowledge were applied for each course and treatment (control and experimental intervention). The pre-test results allowed us to verify that both treatments started from the same baseline of knowledge. The results revealed no significant difference, so the groups of both treatments started from the same knowledge base. On the other hand, the post-test results indicated significant differences between the treatments. Table 2 presents the overall results of each stage and course.

Table 2 Post-test results between Control and Experimental Groups
Full size table

The results show that the students receiving the experimental treatment in the course Computational Thinking had a significantly better performance than the control groups in both stages, while in the course Physics I, no significant difference was found between the treatments of both stages. Concerning Physics II, a significant difference was found in favor of the control students, but only in the first stage. In the second stage, there was no difference. Finally, in Mathematics, there was a significant difference in favor of the control treatment in the first stage and oppositely in the second; however, the scores in both treatments were very low. Table 3 shows the learning gain in both stages and the four courses.

Table 3 Control and experimental group learning gain
Full size table

Table 3 shows that only Physics II in the first stage had a significant difference from the control group; however, descriptively, the learning gain was more significant in the experimental group in most of the stages and courses.

Qualitative results

The focus groups with professors and the student responses to open questions identified that pre-reading is a determining factor; in the first stage, pre-reading content through the AL Platform was not mandatory, so it was tested to make it mandatory in the second stage. The professors of the first stage focus groups commented that most students did not do pre-reading; likewise, some students in the first stage commented that the pre-reading helped them understand the class topics better.

Another determining factor in Stage 1 was the content design, so the mathematics course was redesigned entirely during the second stage; it has not been tested yet. In Physics I and Physics II courses, some adjustments were made to the contents before stage 2; all adjustments to the content design of these courses were completed in the summer of 2023. The Computational Thinking course did not need adjustments for the second stage; however, some contents are currently being enriched.

Finally, another critical factor observed in the second stage was the professors’ learning curve because the professor of the Computational Thinking course returned to teach the course in the second stage, and, as can be seen in Tables 2 and 3, this course resulted with better evaluations in both stages.

Results of research question 2: what is the impact of the ALS supported by an AL platform in achieving students’ disciplinary and transversal sub-competencies in four courses of the school of engineering?

Answering this question required collecting the evidence of each course. Computational Thinking involved a gaming programming project; in the other courses, it was an argumentative exam where students had to apply their knowledge and argue their answers. Based on this evidence, the sub-competencies corresponding to each course were evaluated. A professor external to the project who had already taught the course evaluated the evidence.

The results revealed no significant difference between the control and experimental groups of each course and stage. For example, Table 4 shows the results of the transversal sub-competency of Scientific Thinking shared by Physics I, II, and Mathematics. Computational Thinking displays the results of cutting-edge technologies. In the first stage, sampling was done because the external reviewer could not review all the evidence of each course (more than 300), while the second stage only had one group per treatment and course, so all were reviewed.

Table 4 Sub-competency results
Full size table

As can be seen, no significant difference was found. The second-stage evaluations were lower in all courses. One explanation may be that the second stage mainly had repeating students, and the professors indicated in the interviews that they noticed some student apathy.

Results of research question 3: what was the experience of the ALS by students and professors based on usability, teaching, learning, engagement, and user experience?

Quantitative results

To answer this question, a questionnaire applied to the students had a continuous Likert scale ranging from Strongly disagree (0) to Strongly agree (100). The purpose of the instrument was to assess students’ learning experience with ALS. This instrument evaluated the following dimensions: Usability (AL Platform), Learning Process, Engagement, User Experience, and Teaching Process.

The Usability dimension measured how easy it was to use the platform. Learning Process evaluated the impact of the ALS (content on the platform) on learning. Engagement evaluated how much the student committed to the ALS. User Experience measured their perception of having a 24/7 platform, and, finally, Teaching Process measured if the professor linked the platform’s contents in the class. Table 5 shows the overall results.

Table 5 Learning experience evaluation
Full size table

The results revealed that the Computational Thinking course had a better evaluation in the second stage in almost all dimensions. As mentioned before, the second-stage professor had participated in the first, so the professor’s learning curve may be one of the determining factors for the success of the second stage. The second stage of Physics II also had higher means in all dimensions. In this case, the second-stage professor did not participate in the first, but the contents of the adaptive platform were enriched for the second stage, which did not happen in Physics I and Mathematical Modeling. Their contents had not been modified, nor were they in charge of professors participating in the first stage. This may indicate that the content design on the platform is another determining factor for the success of the ALS.

In the questionnaire, students assessed the AL Platform (0 as the lowest grade, 100 as the highest), Table 6 shows the results.

Computational Thinking and Physics II courses obtained the best means. The students agreed, except for those of the Mathematical Modeling course, on the platform’s strengths: flexibility and usability. What motivated them to consult it were learning, their grade, and pre-reading (they found value in arriving at the class with a knowledge base). Regarding the suggestions, there was agreement to include videos, games, concise readings, and automated feedback. In the case of Mathematical Modeling, most students expressed that they did not feel motivated or see strengths and suggested redesigning the contents, which is already in process for Stage 3.

Qualitative results

Finally, focus groups were developed with the professors of each course to collect information in the first stage and interviews in the second since there was only one group per course. Figure 8 summarizes the comments of the professors of Computational Thinking, Physics I, and Physics II.

Fig. 8
figure 8

Focus group and interview results

Full size image

The results show that the professors of the courses Computational Thinking, Physics I, and Physics II agreed that pre-reading favored the participation and performance of the students. Other comments were that the platform was excellent support for introducing the topics and that the contents are generally good, although they suggest enriching them and making them more attractive to students, which is currently underway. Areas of opportunity identified some errors in exercises and the results in the Mathematical Modeling course, which puzzled and demotivated the students.

Also, the professor of the first stage agreed that the students did not do pre-reading, so the professors suggested closing the date of the activities to just before the topics were addressed in class, as this could motivate students to do the pre-reading, particularly those who did not usually do it. This suggestion was applied in the second stage, which positively affected the courses Computational Thinking and Physics II (see Table 6).

Table 6 Students’ assessment of the adaptive learning platform
Full size table

Discussion

Learning level

Regarding the first research question, level of learning, results identified that ALS impacted the level of learning in two of the four courses (see Tables 2 and 3). One of these courses was Computational Thinking, which was characterized by having a sound content design in stage 1, and it was not necessary to adjust them in stage 2. Note that stage 2 attained better results despite the groups mainly being comprised of repeating students. Another important aspect was that in stage 2, the pre-reading (pre-study) was mandatory, and the professor had participated in stage 1.

This result is contradictory to what was proposed by White (2020) and Contrino et al. (2024) where it is stated that the optional consultation of the contents is better for the learning process, in this study it was found that establishing pre-reading as mandatory before going to class, was more successful for students and even allowed teachers to finish the course contents. Perhaps this is due to the age of the students or the culture, it is certainly a topic for another research. On the other hand, Alamri et al. (2021) agrees that one of the important elements of Adaptive Learning is the pre-reading or flip** classroom.

The second course was Physics II, in which a significant difference was in the level of knowledge by the experimental group in stage 1 but not in stage 2. This could have some explanations. One may be the difference in the sample size between the control and experimental groups (see Table 2). The professors were different, and another possible explanation may be that it was an irregular semester with mostly recursive students. In the case of Physics, I and Fundamental Mathematical Modeling, no significant difference was found between the control and experimental groups in any of the stages. Here, the explanations are different. Both Physics I and Physics II are courses of five weeks long. Stage I had some errors in the AL Platform in the Physics I course, although they were promptly corrected; given the course duration, the situation discouraged students and teachers. In stage 2, the platform did not present technological problems; however, similar results were obtained in both stages; the causes may be similar to those of the stage 2 Physics II course.

On the other hand, in the case of Fundamental Mathematical Modeling, the causes of the results are more complex because, from the beginning, the content design was inadequate, which caused dissonance between the contents of the AL Platform and the content addressed in class. This situation discouraged both professors and students. Undoubtedly, the results in both stages have laid the foundations for improvements in the four courses, particularly the contents of the AL Platform for the Fundamental Mathematical Modeling course, which is redesigned. All the adjustments and improvements will appear in Stage 3. Concerning learning gain, Table 3 descriptively shows that it tends to be higher among experimental groups than control groups except for Fundamental Mathematical Modeling in stage 1.

Considering the results presented, it can be stated that ALS tends to have a positive impact on the level of student learning if the content design is appropriate for students, pre-reading (previous study or flipped classroom) is promoted, and there is a commitment from the professors. In this way, ALS allows the professor to optimize teaching time and that the student, when they arrive prepared, can better understand the topics or ask specific questions to improve the topic comprehension, this finding coincides with that found by Hwang et al., (2020), Olmo-Muñoz et al., (2022), Contrino et al., (2024) and Alamri et al. (2021), likewise, this study confirms, as Watson and Watson (2017) that technology is a necessary resource to facilitate the personalization of learning.

Achievement of sub-competencies

The results related to the achievement of the sub-competencies showed no significant difference between the control and experimental groups, in any course and stage. This result was somehow expected because the design of the contents in the AL Platform focused on knowledge and not on the development of the sub-competencies declared in each course. In fact, AL Platforms such as Aleks (Matayoshi et al., 2021), Knewton (Conklin, 2016) or CogBooks (Contrino et al., 2024) aim to enable students to carry out internships in basic sciences and thus advance and improve their learning. However, in the medium term, promoting pre-reading (previous study) through the AL Platform and improving content can indirectly impact students’ achievement of sub-competencies; that is, pre-reading could optimize teaching time in class, leaving more time to design activities that develop sub-competencies. Maybe, we can see the results in the long-term, but not now.

Learning experience

Intriguing aspects stand out concerning the learning experience of both students and professors. The professors of all the courses, except for Mathematical Modeling, agreed that the platform is easy to use and would recommend it to other colleagues (see Table 4 and Fig. 8). Regarding the contents, professors and students of the Computational Thinking course agreed that they are sound, sufficient, and well-dosed, although both suggested diversifying their presentation, for attending different learning styles, in this sense, Contrino et al. (2024) find that one advantage of CogBooks is to offer students the same content in a variety of ways.

Physics I and II professors considered the content good, although they suggested that the exercises offer different difficulty levels, while students suggested including automated feedback beyond merely indicating right or wrong. Although it is known that adaptive platforms such as Aleks (Matayoshi et al., 2021) or Knewton (Conklin, 2016) offer immediate feedback and have a large exercise bank, it is important to specify that ALS uses the RealizeIt platform that allows the content to be developed from scratch and thus that it is aligned with the course program, in this sense, the ALS project holds the promise of enriching and improving resources each semester.

Regarding pre-reading, the professors agreed that the students who did it participated more in class; therefore, their performance was better than those who did not. In the first stage, the professors complained that, since it was optional, few students did the pre-lecture, so they decided to make it compulsory for the second stage. It will continue in the third stage. On the other hand, the students commented that pre-reading made them feel more confident in class and that they better understood the topics, in addition to having the content on the platform to review or consult. Pre-lecture works in the design of the ALS as a Flipped Classroom, so when performed, it optimizes the teaching–learning process; however, the design of the content plays a relevant role in motivating and engaging the student, as well as the commitment of the teacher to link these contents with the classroom dynamics. In Alamri et al. (2021) study establishes that one of the important elements of Adaptive Learning is the flipped classroom, in this way teachers can optimize class time and attend to students in a personalized way (Watson & Watson, 2017).

Regarding analytics, the professors recognized the value of having information on the group and individual performances in their courses to follow up with students and motivate them promptly. Likewise, the students highlighted that the knowledge determination exams within the adaptive platform allowed them to start from their level of knowledge and visualize their learning paths. Olmos-Lopez et al. (2023) found the same advantages when professors were able to adapt the teaching process based on the information obtained after applying a predictive algorithm of student performance.

On the other hand, the students in the Computational Thinking course evaluated the AL Platform the best, followed by those taking Physics II, which aligns with the above discussion (see Table 5). Finally, ALS is a network of pedagogical resources enabled by the professor and the AL Platform to improve student learning. Therefore, based on the research results, it was determined that the factors associated with the success of the ALS concerning the learning level and gain were content design, pre-reading (previous study), interaction with the platform, an evaluation greater than 80 for the ALS, teacher commitment, and an evaluation greater than 80 for the AL Platform. Adaptive learning must be implemented correctly in such a way as to ensure a good learning experience (Alamri et al., 2021) by selecting the technology that favors its implementation and didactic needs (Watson & Watson, 2017).

Implementation model of the adaptive learning strategy

After the results and discussion, and responding to the last research question, we have integrated the following model that encompasses what we believe an ALS should have (see Fig. 9).

Fig. 9
figure 9

Final implementation model of the adaptive learning strategy

Full size image

Within the Final Implementation Model of the Adaptive Learning Strategy they involve both student and professor activities in these moments: before, during and after class. Before and after the class, the student in the AL platform takes a diagnostic test and studies the contents designed under Microlearning which are available 24/7; subsequently takes quick knowledge check quizzes. It is at this point, where the student puts into practice skills of Self-regulated learning and the Flipped classroom model. With the previous study (pre-reading) of the digital resources published in the AL Platform, the students go to class prepared by studying the content, practicing with exercises with automatic feedback, and repeating the content that is not understood. During class, the student performs a pre-reading check activity and actively participates in class based on what has been studied in the AL platform. Arriving prepared for class allows the student to participate actively in the classroom and receive personalized follow-up.

For the professor the activities are as follows: Before and after the class, the professor monitors the learning analytics and adjusts the interaction instruction based on the group’s level of understanding and the strategies available. During the class, applies strategies for verification of pre-reading, applies Active learning strategies, takes content from the AL Platform to support instruction and evidence resolution, finally, gives instructions for review of content in the AL Platform for the next session. In a cross-cutting manner during this cycle the professor provides personalized follow-up to students who require intervention.

Conclusions

When applied correctly, the ALS can make a significant difference in impacting students’ learning levels. The findings of this study reveal the novelty that the implementation of an ALS tends to positively impact student learning, when combined with Flipped Classroom, Teaching Strategies, Self-regulated Learning Microlearning and AL platform aligned with this strategy, in addition to placing special emphasis on the Content Design and an adequate Implementation Model by the professor where the required actions are established before, during and after the time of each class.

Regarding teaching strategies, an important finding is that the pre-reading (previous study) element of the Flipped Classroom is a determining factor that promotes better results when it is mandatory since it allows the student to achieve better results. In addition, the student can arrive to the class more prepared, and the professor can optimize the time in class.

About the design of content, the most relevant factors for success include: (a) pedagogical planning, which establishes the purposes of learning and the structure of content; (b) the design of content and learning paths made by experts following the guidelines established in the Pedagogical Model; (c) the use of relevant strategies such as Microlearning and Self-regulated Learning; and (d) design content in different formats so that they are available 24/7 on the AL Platform.

With respect to the Implementation Model of the Adaptive Learning Strategy, what this study proposes is that the professor, before and after each class, continuously monitors the performance of the students through the analytics of the AL Platform to identify the performance of the group and adapt the instructional plan for each class, according to the needs of the students. In addition, it is very important that during class the professor links the contents of the platform, applies Reading Verification Strategies, Active Learning and provides instructions on what the student must do for the next class. To ensure that this model is implemented, it is important to train the professor on the ALS, the Didactic Model, which contains the associated strategies and the Delivery Model, as well as provide the required technological and pedagogical support, especially when implementing this model for the first time. In this regard, this study revealed the professor’s learning curve as a key factor, showing that better results are obtained as the professor gains more experience in applying the ALS.

None of the elements mentioned above alone could achieve a good result. It won´t be successful to have an adaptive platform aligned with the learning experience the institution look for, full of well-structured and designed content if the professor does not make sure to link this content in class and promote its use, since that could demotivate the students. The authors conclude that to achieve a successful ALS it is necessary to combine it with various elements that are related to an adequate content design strategy, incorporation of didactic strategies and an adequate implementation model by the professor in addition to supporting it with the appropriate technology that offers key aspects such as performance and learning analytics, personalized learning routes, support for multiple contents in different formats, ease of use, diagnostic exams, integration with the EdTech Ecosystem, among other things.

Although the study is robust, interdisciplinary, and interesting, it has some limitations, first, although the sample size is representative of the university’s first-year student population, a probabilistic sample could not be performed because only a few campuses were selected for the study. Likewise, some professors did not participate voluntarily, which caused some discomfort that was transferred to the students, that is, those professors who gotQ17 involved with ALS had a positive response from the students and vice versa. Nevertheless, Adaptive learning is an excellent alternative to positively impact on the Personalized Learning of the student, and the Teaching and Learning Process if its design includes an appropriate pedagogical and implementation strategy to achieve a successful student experience.