We hold our inaugural FLoRA symposium: “New Ways of Measuring, Analysing and Scaffolding Self-Regulated Learning” at the 19 th Biennial European Association for Research on Learning and Instruction – EARLI 2021 conference (link: https://earli.org/EARLI2021). In light of the COVID-19 pandemic, the EARLI 2021 conference will take place online with the conference theme – Education and Citizenship: learning and Instruction and the Shaping of Futures. We look forward to seeing you at the conference!
Besides, Prof. Maria Bannert will also join the other members of the EARLI-Centre for Innovative Research (E-CIR), “Measuring and Supporting Student’s Self-Regulated Learning in Adaptive Educational Technologies“, in an expert panel discussion. In light of the COVID-19 pandemic, the EARLI 2021 conference will take place online with the conference theme – Education and Citizenship: learning and Instruction and the Shaping of Futures.
The contributions are as follows:
Project FLoRA EARLI Symposium: New Ways of Measuring, Analysing and Scaffolding Self-Regulated Learning
Organizer: Joep van der Graaf, Radboud University, Netherlands
Discussant: Philip Winne, Simon Fraser University, Canada
Date & Time: 26 Aug 2021, 15:45 – 16:45
Location: Session Room 5 – Session T
New ways of measuring and analysing Self-Regulated Learning (SRL) are rapidly emerging. This has important implications for theoretical frameworks of SRL, methodological approaches, and for current educational practices. The first aim of this symposium is to present and discuss new approaches to measurement and analysis of SRL. The second aim is to provide suggestions on the design of educational materials that provide additional insights into students’ learning processes and support their SRL. The presenters are a group of international researchers, who have a strong interest in learning analytics and/or SRL. The presentations are incrementally ordered, moving from measurement and analysis of SRL (1, 2) to learning outcomes (3) and ending with a digital learning tool (4). The four presentations revolve around the following main questions: a) How can multimodal data improve the granularity of measurement of SRL? b) How does SRL unfold in high versus low performing students? c) How do SRL activities relate to different learning outcomes? And d) How can we support students by visualising their SRL process? By addressing these questions, this symposium provides suggestions for theory and methodological development as well as educational practice.
 Improving the granularity for the measurement of self-regulated learning using multi-channel data.
Yizhou Fan, The University of Edinburgh, United Kingdom; Lyn Lim, Technical University of Munich, Germany; Joep van der Graaf, Radboud University, Netherlands; Jonathan Kilgour, The University of Edinburgh, United Kingdom; Johanna Moore, The University of Edinburgh, United Kingdom; Dragan Gasevic, Monash University, Australia; Maria Bannert, Technical University of Munich, Germany; Inge Molenaar, Radboud University, Netherlands.
In recent years, unobtrusive measures of self-regulated learning (SRL) processes based on log data recorded by digital learning environments have attracted increasing attention. However, researchers have also recognised that simple navigational log data or time spent on pages are often not fine-grained enough to study complex SRL processes. Recent advances in data-capturing technologies enabled researchers to go beyond simple navigational logs to measure SRL processes with multi- channel data. Though, to what extent can the addition of peripheral and eye-tracking data with navigational data improve the granularity of measurement of SRL are key questions that require further investigation. Hence, we conducted a study that aimed to address this problem by enhancing navigational log data with peripheral and eye-tracking data. Based on the measurement protocol proposed in this study, we were able to compare the process models of SRL of n=25 students across different data channels. The results revealed that by adding new data channels, we improved the capture of learning actions and detected SRL processes while enhancing the granularity of the measurement. In addition, we also concluded that eye-tracking data is valuable for measuring and extracting SRL processes, and it should receive more attention in the future.
 Understanding Self-Regulated Learning Processes through Process Mining.
Lyn Lim, Technical University of Munich, Germany; Maria Bannert, Technical University of Munich, Germany; Joep van der Graaf, Radboud University, Netherlands; Yizhou Fan, The University of Edinburgh, United Kingdom; Jonathan Kilgour, The University of Edinburgh, United Kingdom; Inge Molenaar, Radboud University, Netherlands; Johanna Moore, The University of Edinburgh, United Kingdom; Dragan Gasevic, Monash University, Australia.
Self-regulated learning (SRL) is related to better learning outcomes and observation of SRL using think aloud data has been shown to be more insightful in determining SRL activities and predicting students’ learning achievements than self-reports. Educational process mining, moreover with think aloud data, enables a deeper understanding and a more fine-grained analysis of SRL processes. This study based on a pre-post design aimed to investigate how students differ in SRL learning processes and how this affects learning performance. There were 32 university students who participated in the study to learn about the theme, “Artificial Intelligence in Education”, and they had to write an essay in a digital learning environment within a 45-minute learning session while thinking aloud. The results showed that there is a significant learning gain in the knowledge test. Besides, the top performers showed more metacognitive and cognitive activities during learning. Furthermore, process mining using HeuristicMiner algorithm based on post hoc coded think aloud protocols examined differences in process structures of SRL for the high and low performers. In general, comparing resulting process mining models with prior process mining models will help to better generalize findings of prior research.
 How Self-Regulated Learning Affects Different Learning Outcomes.
Joep van der Graaf, Radboud University, Netherlands; Lyn Lim, Technical University of Munich, Germany; Yizhou Fan, The University of Edinburgh, United Kingdom; Jonathan Kilgour, The University of Edinburgh, United Kingdom; Johanna Moore, The University of Edinburgh, United Kingdom; Dragan Gasevic, Monash University, Australia; Maria Bannert, Technical University of Munich, Germany; Inge Molenaar, Radboud University, Netherlands.
Self-regulated learning (SRL) fosters transfer, but effects on other learning outcomes, such as domain knowledge are mixed. SRL potentially has a differential impact on learning outcomes with different characteristics, deep vs surface knowledge, and independent vs connected concepts. Therefore, we assessed how surface knowledge measured with a domain test (independent), and a concept map (connected) and deep knowledge measured with a transfer test (independent) and an essay (connected) are associated to SRL activities during learning and to prior metacognitive knowledge. Forty-five university students performed a 45 minute problem-solving task integrating three topics into a vision on future of education. SRL activities were measured using think aloud. Results revealed learning occurred. Surface knowledge measures, independent and connected concepts, were related to each other and associated with low cognitive activities during learning. Deep knowledge of independent concepts was associated with low cognitive processes, while deep knowledge of connected concepts was associated with a mixture of low and high cognitive processes. In addition, we found that prior metacognitive knowledge was associated with deep knowledge of independent concepts. To conclude, taking the level and structure of knowledge into account helps to specify effects of SRL processes on learning outcomes.
 Visualising student’s learning strategies in online learning to support self-regulation.
Shaveen Singh, Monash University, Australia; Mladen Rakovic, Monash University, Australia, Yizhou Fan, The University of Edinburgh, United Kingdom; Lyn Lim, Technical University of Munich, Germany; Joep van der Graaf, Radboud University, Netherlands; Jonathan Kilgour, The University of Edinburgh, United Kingdom; Inge Molenaar, Radboud University, Netherlands; Johanna Moore, The University of Edinburgh, United Kingdom; Maria Bannert, Technical University of Munich, Germany; Dragan Gasevic, Monash University, Australia.
Visualisations provide an effective way for learners to gain insight into their learning process which, in turn, may promote their self-regulated learning. Yet few learner-facing visualisations have been developed to support learners’ self-regulation. To this purpose, we propose a collection of personalised, theory-based and empirically driven visual interfaces. We harnessed trace data from multiple channels to generate clear and actionable recommendations for learners to improve their regulation. Guided by a quasi-experimental study in an university context (n=25), we investigated the student’s critical learning processes in SRL, such as, planning, content consumption, working on task, monitoring and evaluation. In the presentation, we describe the learning environment to collect data about those processes, and suggest visualizations that rely upon these data sources. In an ongoing study, we will prompt learners to engage in metacognitive monitoring of their learning using visualisations to support their regulation and learning.
We are delighted to be presenting our work at LAK 21. LAK21 is organised by the Society for Learning Analytics Research (SoLAR) with location host University of California, Irvine. This year, the conference will be fully online from April 12-16, 2021.
Attend our sessions to know more about our work:
 van der Graaf, J., Molenaar, Lim L., I., Fan, Y.,Kilgour, J., Moore, J., Dasevic, D. & Bannert, M. (2021, April). Do Instrumentation Tools Capture Self-Regulated Learning? In Proceedings of the 11th International Conference on Learning Analytics & Knowledge (pp. 438-448).
Abstract: Researchers have been struggling with the measurement of SelfRegulated Learning (SRL) for decades. Instrumentation tools have been proposed to help capture SRL processes that are difficult to capture. The aim of the present study was to improve the measurement of SRL by embedding instrumentation tools in a learning environment and validating the measurement of SRL with these instrumentation tools using think aloud. Synchronizing log data and concurrent think-aloud data helped identify which SRL processes were captured by particular instrumentation tools. One tool was associated with a single SRL process: the timer co-occurred with monitoring. Other tools co-occurred with a number of SRL processes, i.e., the highlighter and note-taker captured superficial writing down, organizing, and monitoring, whereas the search and planner tools revealed planning and monitoring. When specific learner actions with the tool were analyzed, a clearer picture emerged of the relation between the highlighter and note taker and SRL processes. By aligning log data with think-aloud data, we showed that instrumentation tool use indeed reflects SRL processes.
The main contribution is that this paper is the first to show that SRL processes that are difficult to measure by trace data can indeed be captured by instrumentation tools such as high cognition and metacognition. Future challenges are to collect and process log data in real-time with learning analytic techniques to measure ongoing SRL processes and support learners during learning with personalized SRL scaffolds.
 Singh, S., Rakovic, M., Lim, K.P., van der Graaf, J., Fan, Y.,Kilgour, J., Bannert, M., Molenaar, I., Gasevic, D. & Moore, J. (2021, April). Using Enhanced Learner-facing Visual Interfaces to support Self-regulated Learning. In Companion Proceedings of the 11th International Conference on Learning Analytics & Knowledge (pp. 661-63).
Abstract: Visualisations provide a rapid way for learners to see and understand their learning metrics. Yet few learner-facing interfaces have been developed to support learners’ self-regulation. This paper proposes the application profile of personalised visual interfaces to support learners in self-regulated learning (SRL). Our design is theoretically based and empirically driven, and utilises trace data from multiple channels to provide clear actionable recommendations for learners to improve regulation. Guided by a quasi-experimental study in a university context, we survey the critical learning processes in SRL, describe the environment to collect multimodal and multichannel data about those processes, and suggest visualizations that can rely upon these data sources— to prompt learners to engage in metacognitive monitoring of their learning and adapt their learning. We conclude with future directions of contributing to the interdisciplinary efforts of addressing the use of cognitive and meta-cognitive data traces and visualisations to foster self-regulation to support optimal and successful learning.
See you there !
FLoRA-Project featured in the EDUCAUSE 2021 Horizon Report
Our ORA research project, FLoRA – Facilitating Self-Regulated Learning with Personalized Scaffolds on Student’s own Regulation Activities (funded by the DFG, ESRC, NWO) was selected to be featured in the EDUCAUSE 2021 Horizon Report | Teaching and Learning Edition.
The EDUCAUSE 2021 Horizon Report profiles key trends and emerging technologies and practices shaping the future of teaching and learning and envisions a number of scenarios and implications for that future. It is based on the perspectives and expertise of a global panel of leaders from across the higher education landscape. Six out of 141 key technologies and practices were voted to have a significant impact on the future of higher education teaching and learning. The report serves as one vantage point on where our post-pandemic future of teaching and learning may be headed.
The report was officially published on 26 April and can be downloaded from here: