We hold our inaugural FLoRA symposium: “New Ways of Measuring, Analysing and Scaffolding Self-Regulated Learning” at the 19 th Biennial European Association for Research on Learning and Instruction – EARLI 2021 conference (link: https://earli.org/EARLI2021). In light of the COVID-19 pandemic, the EARLI 2021 conference will take place online with the conference theme – Education and Citizenship: learning and Instruction and the Shaping of Futures. We look forward to seeing you at the conference!
Besides, Prof. Maria Bannert will also join the other members of the EARLI-Centre for Innovative Research (E-CIR), “Measuring and Supporting Student’s Self-Regulated Learning in Adaptive Educational Technologies“, in an expert panel discussion. In light of the COVID-19 pandemic, the EARLI 2021 conference will take place online with the conference theme – Education and Citizenship: learning and Instruction and the Shaping of Futures.
The contributions are as follows:
Project FLoRA EARLI Symposium: New Ways of Measuring, Analysing and Scaffolding Self-Regulated Learning
Organizer: Joep van der Graaf, Radboud University, Netherlands
Discussant: Philip Winne, Simon Fraser University, Canada
Date & Time: 26 Aug 2021, 15:45 – 16:45
Location: Session Room 5 – Session T
New ways of measuring and analysing Self-Regulated Learning (SRL) are rapidly emerging. This has important implications for theoretical frameworks of SRL, methodological approaches, and for current educational practices. The first aim of this symposium is to present and discuss new approaches to measurement and analysis of SRL. The second aim is to provide suggestions on the design of educational materials that provide additional insights into students’ learning processes and support their SRL. The presenters are a group of international researchers, who have a strong interest in learning analytics and/or SRL. The presentations are incrementally ordered, moving from measurement and analysis of SRL (1, 2) to learning outcomes (3) and ending with a digital learning tool (4). The four presentations revolve around the following main questions: a) How can multimodal data improve the granularity of measurement of SRL? b) How does SRL unfold in high versus low performing students? c) How do SRL activities relate to different learning outcomes? And d) How can we support students by visualising their SRL process? By addressing these questions, this symposium provides suggestions for theory and methodological development as well as educational practice.
[1] Improving the granularity for the measurement of self-regulated learning using multi-channel data.
Yizhou Fan, The University of Edinburgh, United Kingdom; Lyn Lim, Technical University of Munich, Germany; Joep van der Graaf, Radboud University, Netherlands; Jonathan Kilgour, The University of Edinburgh, United Kingdom; Johanna Moore, The University of Edinburgh, United Kingdom; Dragan Gasevic, Monash University, Australia; Maria Bannert, Technical University of Munich, Germany; Inge Molenaar, Radboud University, Netherlands.
In recent years, unobtrusive measures of self-regulated learning (SRL) processes based on log data recorded by digital learning environments have attracted increasing attention. However, researchers have also recognised that simple navigational log data or time spent on pages are often not fine-grained enough to study complex SRL processes. Recent advances in data-capturing technologies enabled researchers to go beyond simple navigational logs to measure SRL processes with multi- channel data. Though, to what extent can the addition of peripheral and eye-tracking data with navigational data improve the granularity of measurement of SRL are key questions that require further investigation. Hence, we conducted a study that aimed to address this problem by enhancing navigational log data with peripheral and eye-tracking data. Based on the measurement protocol proposed in this study, we were able to compare the process models of SRL of n=25 students across different data channels. The results revealed that by adding new data channels, we improved the capture of learning actions and detected SRL processes while enhancing the granularity of the measurement. In addition, we also concluded that eye-tracking data is valuable for measuring and extracting SRL processes, and it should receive more attention in the future.
[2] Understanding Self-Regulated Learning Processes through Process Mining.
Lyn Lim, Technical University of Munich, Germany; Maria Bannert, Technical University of Munich, Germany; Joep van der Graaf, Radboud University, Netherlands; Yizhou Fan, The University of Edinburgh, United Kingdom; Jonathan Kilgour, The University of Edinburgh, United Kingdom; Inge Molenaar, Radboud University, Netherlands; Johanna Moore, The University of Edinburgh, United Kingdom; Dragan Gasevic, Monash University, Australia.
Self-regulated learning (SRL) is related to better learning outcomes and observation of SRL using think aloud data has been shown to be more insightful in determining SRL activities and predicting students’ learning achievements than self-reports. Educational process mining, moreover with think aloud data, enables a deeper understanding and a more fine-grained analysis of SRL processes. This study based on a pre-post design aimed to investigate how students differ in SRL learning processes and how this affects learning performance. There were 32 university students who participated in the study to learn about the theme, “Artificial Intelligence in Education”, and they had to write an essay in a digital learning environment within a 45-minute learning session while thinking aloud. The results showed that there is a significant learning gain in the knowledge test. Besides, the top performers showed more metacognitive and cognitive activities during learning. Furthermore, process mining using HeuristicMiner algorithm based on post hoc coded think aloud protocols examined differences in process structures of SRL for the high and low performers. In general, comparing resulting process mining models with prior process mining models will help to better generalize findings of prior research.
[3] How Self-Regulated Learning Affects Different Learning Outcomes.
Joep van der Graaf, Radboud University, Netherlands; Lyn Lim, Technical University of Munich, Germany; Yizhou Fan, The University of Edinburgh, United Kingdom; Jonathan Kilgour, The University of Edinburgh, United Kingdom; Johanna Moore, The University of Edinburgh, United Kingdom; Dragan Gasevic, Monash University, Australia; Maria Bannert, Technical University of Munich, Germany; Inge Molenaar, Radboud University, Netherlands.
Self-regulated learning (SRL) fosters transfer, but effects on other learning outcomes, such as domain knowledge are mixed. SRL potentially has a differential impact on learning outcomes with different characteristics, deep vs surface knowledge, and independent vs connected concepts. Therefore, we assessed how surface knowledge measured with a domain test (independent), and a concept map (connected) and deep knowledge measured with a transfer test (independent) and an essay (connected) are associated to SRL activities during learning and to prior metacognitive knowledge. Forty-five university students performed a 45 minute problem-solving task integrating three topics into a vision on future of education. SRL activities were measured using think aloud. Results revealed learning occurred. Surface knowledge measures, independent and connected concepts, were related to each other and associated with low cognitive activities during learning. Deep knowledge of independent concepts was associated with low cognitive processes, while deep knowledge of connected concepts was associated with a mixture of low and high cognitive processes. In addition, we found that prior metacognitive knowledge was associated with deep knowledge of independent concepts. To conclude, taking the level and structure of knowledge into account helps to specify effects of SRL processes on learning outcomes.
[4] Visualising student’s learning strategies in online learning to support self-regulation.
Shaveen Singh, Monash University, Australia; Mladen Rakovic, Monash University, Australia, Yizhou Fan, The University of Edinburgh, United Kingdom; Lyn Lim, Technical University of Munich, Germany; Joep van der Graaf, Radboud University, Netherlands; Jonathan Kilgour, The University of Edinburgh, United Kingdom; Inge Molenaar, Radboud University, Netherlands; Johanna Moore, The University of Edinburgh, United Kingdom; Maria Bannert, Technical University of Munich, Germany; Dragan Gasevic, Monash University, Australia.
Visualisations provide an effective way for learners to gain insight into their learning process which, in turn, may promote their self-regulated learning. Yet few learner-facing visualisations have been developed to support learners’ self-regulation. To this purpose, we propose a collection of personalised, theory-based and empirically driven visual interfaces. We harnessed trace data from multiple channels to generate clear and actionable recommendations for learners to improve their regulation. Guided by a quasi-experimental study in an university context (n=25), we investigated the student’s critical learning processes in SRL, such as, planning, content consumption, working on task, monitoring and evaluation. In the presentation, we describe the learning environment to collect data about those processes, and suggest visualizations that rely upon these data sources. In an ongoing study, we will prompt learners to engage in metacognitive monitoring of their learning using visualisations to support their regulation and learning.