Keywords

1 Introduction

Human-Computer Interaction is a multidisciplinary field focused on human aspects of the development of computer technology, which combines the theories and practices from different disciplines (e.g., computer science, cognitive and behavioral psychology, ergonomics). In particular, computer-learning technologies, such as Learning Management Systems (LMSs), within an Online Learning Environment (OLE), can provide educators an environment to place their online course materials and for students to receive that education while interacting with other students/teachers; however, students’ interactions, attention and communications are seen as relatively low in the LMSs [1].

Nevertheless, it seems fair to say that higher education institutions (HEIs) are facing the need of constant monitoring of users’ interaction with LMS, in order to identify key areas for potential improvement. This could be expressed in terms of quality of interaction (QoI) through the LMS use within a blended (b-) learning environment [2–6]. The fundamental challenge would be to provide flexible solutions, where intelligent models could contribute, involving artificial intelligence and incertitude modelling, e.g., via Fuzzy Logic (FL) [7, 8]. In this line, a novel approach within the field of FL is proposed here, exploring the potentiality of the Fuzzy Cognitive Map (FCM) [9], in order to be used as a structural element of a new modelling scheme, namely FCM-QoI.

The proposed FCM-QoI model was drawn from a real-life LMS Moodle use case of a public Higher Education Institution (HEI), for effectively estimating the LMS users’ QoI within a b-learning context. It involves a FCM with 14 input-one output concepts, dependences and trends considering the time period of the 51 weeks of the LMS use (one academic year), from two academic teaching of dance disciplines, including rare and contemporary dances, respectively. The latter are used as paradigms that comply with the educational scenarios of the i-Treasures project (www.i-treasures.eu), which is situated in the area of Intangible Cultural Heritage (ICH) education and it is aimed to: develop an open and extendable platform to provide access to ICH resources; enable knowledge exchange between researchers; and contribute to the transmission of rare know-how from Living Human Treasures to apprentices.

Two training/testing scenarios were explored, i.e., time-in/dependent, using pre-validated QoI data. Seen as a holistic and dynamic model, the FCM-QoI approach has the potential to explore possibilities and scenarios from different perspectives; for instance, pedagogical planners and decision makers can (re)adjust online tools, towards maximum use of the LMS Moodle within the teaching/learning practices.

The experimental results have shown that the proposed FCM-QoI model can provide concepts interconnection and causal dependencies representation of LMS users’ interaction behaviour, contributing to the analysis and modelling of different ICHs (e.g., in the field of dance), thus, supporting learning of the rare know-how behind these cultural expressions and their passing down to new generations.

2 Methodology

2.1 The Fuzzy Cognitive Map (FCM) Concept

Originated from the theories of FL, neural networks, soft computing and computational intelligence techniques, FCMs can be understood as a modelling methodology based on exploiting knowledge and experience. In general, FCMs belong to the granular computing field, which refers to the conceptualization/processing of information granules (i.e., concepts). Based on binary values, Kosko [9] suggested the use of fuzzy causal functions considering numbers within [-1, 1], modifying, in this way, the Axelrod’s cognitive maps perspective and introducing the FCM concept. In addition, a relevant update of cognitive maps combined with FL was examined by Kosko [10]. Basically, FCMs express causality over time and are intended to model causality, not merely semantic relationships between concepts, facilitating the exploration of the implications of complex conceptual models, with greater flexibility [11]. FCM is a qualitative modelling tool and can describe any system using a model with three main features, more specifically [11]:

  • the signed causality, indicating positive/negative relationship;

  • the strengths of the causal relationships, taking fuzzy values; and

  • the dynamic causal links, where the effect of a change in one node influences other nodes (i.e., a feedback mechanism that captures the dynamic relationship of all the nodes).

Seen as a dynamic modelling technique, FCMs have gained considerable research interest in multiple scientific areas from knowledge modelling to decision making (e.g., [12, 13]). At the same time, FCMs have been studied resorting to data mining techniques for promoting user’s expert knowledge ([14, 15]). In general, FCM is considered an efficient inference engine to easily model complex causal relationships in a qualitatively and quantitatively way, and to express the dynamic behavior of a set of related concepts.

2.2 The Proposed FCM-QoI Model

A schematic representation of the proposed FCM-based modeling approach, namely FCM-QoI model, is depicted in Fig. 1. From the Fig. 1(a), it is clear that the user interacts with the LMS Moodle and 110 metrics are acquired, exactly the same as the ones used in the FuzzyQoI model ([7] -Table 1). These metrics are then categorized into 14 categories, denoted as C1,C2,…,C14, corresponding to the 14 categories used in the FuzzyQoI model [7], i.e., C1: {Journal/Wiki/Blog/Form (J/W/B/F)}, C2: {Forum/Discussion/Chat (F/D/C)}, C3: {Submission/Report/Quiz/Feedback (S/R/Q/F)}, C4: {Course Page (CP)}, C5: {Module (M)}, C6: {Post/Activity (P/A)}, C7: {Resource/Assignment (R/A)}, C8: {Label (L)}, C9: {Upload (UP)}, C10: {Update (U)}, C11: {Assign (A)}, C12: {Edit/Delete (E/D)}, C13: {Time Period (TP)}, and C14: {Engagement Time (ET)}. These 14 concepts are considered the inputs of the FCM within the FCM-QoI model and the additional FCM-QoI concept is considered its output (see Fig. 1(b)). The FuzzyQoI model (Fig. 1(a)) outputs the \( QoI^{FIS} \), which is used in the training phase of the FCM within the FCM-QoI model (Fig. 1(b)). In the proposed FCM-QoI model, the Nonlinear Hebbian Rule (NHB)-based learning algorithm [16] is adopted. There, the \( W:(Ci,Cj) \to w_{ij} \) is estimated, which is a function that associates a causal value \( w_{ij} \in [ - 1,1] \) to each pair of nodes of the connection matrix, denoting the weight of the directed edge from \( Ci \) to \( Cj \), representing the causality degree between the interconnected concepts; hence, the weight matrix \( W_{N \times N} \) gathers the system causality, usually determined by knowledge experts.

Fig. 1.
figure 1

A schematic representation of the proposed FCM-QoI model, with the 110 LMS Moodle user’s interaction metrics, categorized into 14 input parameters (C1,…,C14) fed to: (a) the FuzzyQoI model [7] outputting the estimated \( QoI^{FIS} \) and (b) the FCM-QoI model as input concepts interconnected with the FCM-QoI as an output concept to estimate the \( QoI^{FCM} \). Note that the estimated \( QoI^{FIS} \) is fed to the FCM-QoI model during the training phase only, to correctly adjust the interconnection weights of the latter towards the minimization of the error in the knowledge representation process.

Full size image

During the iterative learning process (\( k = 1,2, \ldots ,K \) maximum iterations), the weights \( w_{ji}^{\left( k \right)} \), along with the corresponding concept labels \( A_{i}^{{\left( {k + 1} \right)}} \), are updated, towards the satisfaction of a convergence criterion [16]. When the latter is met, the FCM is considered trained and the updated weights (\( w_{ji}^{up} \)) are used in the testing phase of the FCM-QoI model, towards the final estimation of the \( QoI^{FCM} \) output of the FCM-QoI model.

As it is clear from the aforementioned description of the FCM-QoI model, the involved FCM plays the role of a system representation, dismantling the 14 inputs-1 output relations, as represented by the estimated \( W^{up} \) matrix. This actually reflects the expert’s knowledge representation hidden in the IF/THEN fuzzy rules of the FuzzyQoI model, yet in a more quantitative way, i.e., in the form of the interconnection weight values. The efficiency of the proposed FCM-QoI model has been evaluated through its application to the same data used in the FuzzyQoI model [7], drawn from a real-life LMS Moodle use case from higher education, involving both professors and students, as thoroughly described in the succeeding section.

3 Validation of the FCM-QoI Model

3.1 Data Characteristics

The proposed FCM-QoI model was applied to LMS Moodle data from two dance disciplines, i.e., Rare and Contemporary Dances, drawn from the Faculdade de Motricidade Humana (FMH), Universidade de Lisboa (Portugal), where the corresponding dance disciplines are realized within the b-learning context. Rare Dances (RD) discipline belongs to the Social Dances at the FMH, aiming to provide and develop ways to dance, able to contribute to a students’ education in a more complete, comprehensive and multifaceted way, through the diversity of approaches and multiplicity of perspectives developed in each dance form. Moreover, the social dimension and respect of the act of dance are taken into account to enhance the knowledge and extend the application domain with multicultural approaches, revealing the nature and specificity of their contents. The planning of this discipline aims to construct a place of experience and experimentation with different materials, choreographic and contextual, along with specific techniques for analysis, leading to “know-how” and the enlargement and consolidation of formal and expressive repertoire of the students. On the other hand, Contemporary Dances (CD) discipline is included in the Techniques of Theater Dances at the FMH, which aims to promote the analysis and study of motor vocabulary characteristic of modern and classical dance forms. The corresponding planning includes practice of standardized modeling steps organized in simple exercises with repetitions and chained in sequence dances increasing complexity. Moreover, training skills of observation in situations of mutual learning, are also considered, being consistent with the principles and quality of dance movements.

For each paradigm (RD and CD), the 110 LMS Moodle metrics data for one academic year (2009-2010) from Professors (P) (RD: 2; CD: 2) and Students (S) (RD: 29; CD: 43), were used and analyzed. In order to identify any possible changes in the users’ interaction behavior correlated with a specific time-period section, a time-period segmentation was adopted. The latter has resulted in time-period sections (e.g., semesters (S1: 2-16, S2: 23-38 weeks), exam periods (1st: 18-23, 2nd: 38-46 weeks), interruptions (16-18, 24-25; 30-31 weeks)) that served as landmarks in 51-week total examined period.

3.2 Training/Testing Configuration

The whole dataset of the 14 input-1 output data from the FuzzyQoI model was randomly split into 75 % as a training dataset and 25 % as a testing dataset for the FCM-QoI model. Two training/testing scenarios were conducted, i.e., time-dependent and time-independent. In particular, in the time-dependent training scenario, the time unit of analysis of one week is taken into account, in order to obtain the best estimation of \( QoI^{FCM} \) per week. To this end, 50 randomized selections of the 75 % of the training set were considered and, for each random selection, the mean value across the users per input/output per week was estimated, setting, in this way, the initial concept values of the \( A_{i}^{(1)} , i = 1,2, \ldots ,15 \), per week. The initial values of the \( W \) matrix (\( w_{ji}^{(0)} , j \ne i \) per week were randomly selected from the range of \( [ - 1, 1] \) (apparently \( diagonal\left( W \right) = 0) \). Consequently, the training process in the time-dependent scenario outputted a \( W^{up} \) per week \( (W^{up,w} ) \). In the time-independent training scenario, the same process as in the case of the time-dependent scenario was followed, yet here, for each random selection of the 50 randomized selections of the 75 % of the training set, the mean value across the users per input/output and across the weeks was estimated. In this way, the training process in the time-independent scenario outputted a \( W^{up} \) per academic year \( (W^{up,y} ) \). In the time-dependent and time-independent testing scenarios, the estimated \( W^{up,w} \) and \( W^{up,y} \) were used upon the initially selected testing set (25 % of the initial data) to infer the \( QoI^{FCM} \) per week and per academic year, respectively.

All above scenarios were applied to each RD and CD selected discipline. The evaluation of the performance of the FCM-QoI model was realized via the estimation of the Root Mean Squared Error (RMSE) between the estimated \( QoI^{FCM} \) and the \( QoI^{FIS} \) derived from the FuzzyQoI model [7].

3.3 Implementation Issues

The implementation of the whole analysis of the FCM-QoI model was carried out in Matlab 2014a (The Mathworks, Inc., Natick, USA), using custom-made programming code. The archived data in the LMS Moodle repository were exported from .xml to .xlsm (Microsoft Excel format) and imported to the Matlab environment and archived as .mat files. The values used for the updating process of the FCM were selected as the optimum ones that minimize the number of iterations for meeting the termination criteria \( ({\rm K} \le 20000) \).

4 Results and Discussion

4.1 Time-Dependent Scenario

Training Phase. The RMSE between the estimated \( QoI^{FCM} \) and the \( QoI^{FIS} \) across 50 iterations during the time-dependent training phase of the FCM-QoI model for the P and S cases and for the RD and CD disciplines are depicted in Fig. 2, with the corresponding horizontal lines denoting the mean RMSE, accordingly. Clearly, the low mean RMSE values and limited RMSE range across the 50 repetitions denote an efficient performance and consistency in the behavior of the FCM-QoI model during the time-dependent training phase.

Fig. 2.
figure 2

The RMSE between the estimated \( QoI^{FCM} \) and the \( QoI^{FIS} \) across 50 iterations during the time-dependent training phase of the FCM-QoI model for the P (1st row) and S (2nd row) cases and for the RD (1st column) and CD (2nd column) disciplines (horizontal lines denote the mean RMSE).

Full size image

The estimated \( W^{up,w} \) matrices (corresponding to the minimum RMSE values of Fig. 2), for P/S and RD/CD are illustrated in Fig. 3. Due to the difference in the LMS interaction across the examined cases, some \( W^{up,w} \) matrices show a degree of sparseness across the weeks-axis. Moreover, it seems that the CD discipline exhibits more negative than positive weight values (both in P and S cases), whereas the RD one shows mainly positive ones (both in P and S cases).

Fig. 3.
figure 3

The estimated interconnection weight \( W^{up,w} \) 4D-matrices across the 51 weeks of the academic year between the 15 \( Ci \) concepts (14 input concepts and \( QoI^{FCM} \) as the output concept) of the time-dependent training phase of the FCM-QoI model, which correspond to the iterations of Fig. 2 that exhibit the minimum RMSE, accordingly. P (1st row) and S (2nd row) denote cases; RD (1st column) and CD (2nd column) denote disciplines.

Full size image

Testing Phase. Figure 4 illustrates the results from the testing phase of the time-dependent scenario of the FCM-QoI model, based on the \( W^{up,w} \) 4D-matrices depicted in Fig. 3, displaying, in a superimposed, the estimated \( QoI^{FCM} \) (dashed thick line) and the corresponding \( QoI^{FIS} \) (solid thin line), estimated from the FuzzyQoI model [7]. From Fig. 4 it is clear that the estimated \( QoI^{FCM} \) efficiently captures the morphology of the \( QoI^{FIS} \), showing an efficient generalization of the FCM-QoI model in predicting the \( QoI \) after a training procedure based on historical data.

Fig. 4.
figure 4

The estimated \( QoI^{FCM} \) values (dashed thick line) during the testing phase of the time-dependent scenario, along with the corresponding \( QoI^{FIS} \) (solid thin line) derived from the FuzzyQoI model [7], for P/S cases and RD/CD disciplines. The vertical lines illustrate the time-period segmentation of Sect. 3.1.

Full size image

4.2 Time-Independent Scenario

Training Phase. The RMSE between the estimated \( QoI^{FCM} \) and the \( QoI^{FIS} \) across 50 iterations during the time-independent training phase of the FCM-QoI model for the P/S cases and RD/CD disciplines are depicted in Fig. 5, with the corresponding horizontal lines denoting the mean RMSE, accordingly. Similarly to Fig. 2, low mean RMSE values (yet slightly higher than those of Fig. 2) and limited RMSE range across the 50 repetitions are derived, showing satisfactory training of the FCM-QoI model, based on the data across the whole academic year. Figure 6 presents the estimated FCMs with the interconnection weights \( W^{up,y} \), which correspond to the iterations that exhibit the minimum RMSE in Fig. 5, between the 15 concepts (14 input concepts and \( QoI^{FCM} \) as the output concept) of the time-independent training phase of the FCM-QoI model, for the P/S cases and RD/CD disciplines, respectively. From Fig. 6, the interdependencies and causalities amongst the concepts can be identified.

Fig. 5.
figure 5

The RMSE between the estimated \( QoI^{FCM} \) and the \( QoI^{FIS} \) across 50 iterations during the time-independent training phase of the FCM-QoI model for the P (1st row) and S (2nd row) cases and for the RD (1st column) and CD (2nd column) disciplines (horizontal lines denote the mean RMSE).

Full size image
Fig. 6.
figure 6

The FCMs corresponding to the optimum interconnection weights matrices \( W^{up,y} \) between the 15 concepts [14 input concepts and \( QoI^{FCM} \) as the output concept of the FCM-QoI model], for the P/S cases and RD/CD disciplines.

Full size image

Testing Phase. Figure 7 illustrates the results from the testing phase of the time-independent scenario of the FCM-QoI model, based on the \( W^{{up,{\text{y}}}} \) (not shown here), displaying, in a superimposed way, the estimated \( QoI^{FCM} \) (thick line) and the corresponding \( QoI^{FIS} \) (thin line), estimated from the FuzzyQoI model [7]. From Fig. 6 it is clear that, like in Fig. 4, the estimated \( QoI^{FCM} \) efficiently captures, in most cases, the morphology of the \( QoI^{FIS} \), showing again satisfactory generalization of the FCM-QoI model in predicting the \( QoI \), even after a training procedure based on historical data averaged across the whole academic year.

Fig. 7.
figure 7

The estimated \( QoI^{FCM} \) values (thick line) during the testing phase of the time-independent scenario, along with the corresponding \( QoI^{FIS} \) (thin line) derived from the FuzzyQoI model [7], for P/S cases and RD/CD disciplines. The vertical lines illustrate the time-period segmentation of Sect.3.1. 

Full size image

The described FCM-QoI model provides valuable information to both educational decision-makers and software developers towards more appropriate software development efforts, assisting to more effective management of the main activities that are essential towards optimization of the OLEs functionality. The FCM-QoI model allows dynamic monitoring of LMS users’ QoI across the academic year; hence, contributing to the perspective of OLEs from a dynamic rather static view, taking into account the alterations in the QoI of the LMS users (both P and S) with time. Finally, the findings here support distinct perspectives between RD and CD disciplines, as reflected into the realization of the online component of the b-learning context. The latter could be very useful for effective designing of educational scenarios within the concept of sustaining the cultural heritage, such as teaching rare dances to young generations (New Millennium Learners) and build upon tradition to create a contemporary output, as in the case of the i-Treasures project (http://i-treasures.eu/). There, the RD and CD use-cases are realized via a LMS-based OLE platform, which supports sensorimotor learning, incorporating cutting-edge technology and pedagogical planning according to the experts; hence, combined with the FCM-QoI model output, more enhanced feedback could be provided to the users.

5 Conclusion

A FCM-based estimation of the QoI of both professors and students, when interacting with the LMS Moodle within a b-learning context (RD/CD disciplines), has been presented here. The FCM-QoI model provided an easy way to represent LMS Moodle academic community understanding, in a form of scaled up mental modeling, as a kind of internal representation of external reality. The FCM-QoI model was validated on real data, proving its potentiality to represent the LMS users’ attitude in terms of their QoI. This could facilitate the dynamic design of educational scenarios and strategies in many disciplines/courses/fields (e.g., engineering, social sciences) within advanced OLEs, such as the one of the i-Treasures.