can act as a bridge between the faculty and the rest of the students, even receiving special roles to be able to curate forum messages. Some other research studies focused on analyzing the overall class mood from social interactions, calculating the polarity of messages (positive, neutral, negative) posted by students in the course forum. The polarity of messages was calculated by applying word dictionaries and syntax rules, and the aim was to detect parts of the course in which the overall class mood was more positive or more negative to take corrective measures in the second case (Moreno-Marcos, Alario-Hoyos, Muñoz-Merino, Estévez-Ayres, & Delgado Kloos, 2018). Finally, all the data collected from social interactions can be used as input to develop chatbots or conversational agents programmed to give support to students in specific courses (Delgado Kloos, Catalán-Aguirre, Muñoz-Merino, Alario-Hoyos, 2018).
An additional relevant trend in data-driven innovation in engineering education is the study, characterization, and support of students’ development of self-regulated learning (SRL) skills. SRL skills are particularly important in engineering education due to, among other things, the complexity of the contents and the permanent retraining demanded in today’s engineers. Appropriate strategies to self-regulate each one’s learning should be applied in every learning stage (before, during, and after each learning activity), including, for instance, setting reasonable and measurable objectives (before), seeking help when necessary (during), self-reflecting on the work done and the objectives achieved (after), etc. (Alonso-Mencía, Alario-Hoyos, Maldonado-Mahauad, Estévez-Ayres, Pérez-Sanagustín, & Delgado Kloos, 2019). The study and characterization of SRL skills in engineering courses led to the conclusion that strategies related to appropriate time management are the most problematic ones for learners. Therefore, specific interventions need to be done to facilitate time management, both when designing a course and when developing support tools (Alario-Hoyos, Estévez-Ayres, Pérez-Sanagustín, Delgado Kloos, & Fernández-Panadero, 2017). It is also important to support students’ self-reflection by offering high-level visualizations based on low-level data with the aim to increase students’ awareness on the desired level to be achieved and that of their classmates, both for an entire course and for each module or part of a course (Ruipérez-Valiente, Muñoz-Merino, Leony, & Delgado Kloos, 2015).
One last, but not least, relevant trend in data-driven innovation in engineering education is the prediction of students’ behavior from previously collected data and appropriately trained machine learning models. The variables that are typically predicted through these models refer to students’ partial or final grades in a course; and also to whether a student will abandon a course or not; and, by extension, to whether a student will abandon a complete study program (bachelor’s degree or master’s degree) or not. The aim of these prediction models is to take corrective measures in order to prevent students from failing a course, or from dropping out of a course or study program. Studies on prediction in education have detected that, in general, low-level data, such as the interaction with educational content (e.g., videos and exercises), have greater predictive power than data collected from self-reported questionnaires (such as students’ intentions and motivations) (Moreno-Marcos, Alario-Hoyos, Muñoz-Merino, & Delgado Kloos, 2018). There are still important gaps in this research line, including proposing generalizable models applicable to different educational contexts and areas of knowledge, and developing predictive models and tools for real-time data collection and processing in order to improve the implementation of corrective measures.
3. SOME RISKS OF DATA-DRIVEN INNOVATION IN ENGINEERING EDUCATION
The collection and processing of data for decision-making in engineering education is not without problems. In fact, there are important risks to worry about, which have been highlighted by numerous experts on learning analytics, educational data mining, and data protection policies, (Dringus, 2012) (Khalil, Taraghi, & Ebner, 2016), as well as certain ethical considerations (Slade, & Tait, 2019). Some of these risks are briefly described in this section, although each of them would deserve an entire paper for discussion.
A first problem refers to the secure storage of collected data. Many institutional education systems are not prepared to store large amounts of data in a secure way and are likely to have breaches that can compromise important data, including students’ personal data. Relying on third-party services for data storage in the cloud can also lead to an inappropriate use of the data collected by these service providers.
A second problem refers to data privacy. It is very important to have an institutional strategy that clearly states what data needs to be collected from students and who has access to the data. Many educational institutions are not aware of all the data they collect (or can potentially collect); do not have mechanisms to control who has access to the data collected; and/ or do not have an internal policy to facilitate that the right people can make proper informed decisions, being able to access the data they are entitled to.
A third problem refers to always getting explicit consent from the students for collecting data. It is very important that students (who actually own the data) know at all times what data are being collected from them and for what purpose the data are going to be used. In addition, there are international laws such as the GDPR (General Data Protection Regulation) in Europe that require, among other things, the removal of data collected upon request by its owner. Many educational institutions do not have data collection ethics committees and are not prepared to delete collected data upon request.
A fourth problem refers to transparency, or rather lack of transparency, of many algorithms and systems which are private and whose code is not open. The lack of transparency prevents algorithms and systems from being audited to better understand how they work. Even if one relies on third-party algorithms and systems to make informed decisions, it is important to know how the results and visualizations they provide were obtained.
A fifth problem refers to bias. Many artificial intelligence systems use data collected in the past to make their calculations and predictions. However, data collected in the past may have important biases, such as a gender imbalance, which could lead to promoting more male students versus female students or vice versa. In general, it is important to take minorities into account when using data from the past as input so that these are not penalized.
Finally, it is important to bear in mind that humans may misinterpret the results and visualizations obtained from the data processed. Sometimes there are people who deliberately twist the data to fit a pre-designed theory, instead of discarding this theory if data advise to do so. Actually, from a very large dataset, and taking only a subset of it, erroneous and unreproducible conclusions can be very easily reached.
4. CONCLUSIONS
Innovation in engineering education must be informed by data. Teachers must be aware of their students’ performance (individually and at the class level) to support those who need more help as well as offer top quality educational resources progressively improved according to the data collected. Students should define a curriculum adapted to their particular needs and develop their SRL skills. Institutions must create specialized units for data collection and processing, and adequately train their staff (including teachers) for a data management culture. Nevertheless, there are numerous risks to be aware of, and a trade-off is needed so that these risks do not slow down innovation in educational institutions.
REFERENCES
Alario-Hoyos, C., Pérez-Sanagustín, M., Delgado-Kloos, C., Parada G., H. A., & Muñoz-Organero, M. (2014). Delving into participants’ profiles and use of social tools in MOOCs. IEEE Transactions on Learning Technologies, 7(3), 260-266.
Alario-Hoyos, C., Muñoz-Merino, P. J., Pérez-Sanagustín, M., Delgado Kloos, C., & Parada G., H. A. (2016). Who are the top contributors in a MOOC? Relating participants’ performance and contributions. Journal of Computer Assisted Learning, 32(3), 232-243.
Alario-Hoyos,
