Curricular Concept Maps as Structured Learning Diaries: Collecting Data on Self-Regulated Learning and Conceptual Thinking for Learning Analytics Applications




Self-regulated learning, Concept mapping, Curriculum, Learning diary, Self-monitoring, Learning analytics dashboard


The collection and selection of the data used in learning analytics applications deserve more attention. Optimally, selection of data should be guided by pedagogical purposes instead of data availability. Using design science research methodology, we designed an artifact to collect time-series data on students’ self-regulated learning and conceptual thinking. Our artifact combines curriculum data, concept mapping, and structured learning diaries. We evaluated the artifact in a case study, verifying that it provides relevant data, requires a limited amount of effort from students, and works in different educational contexts. Combined with learning analytics applications and interventions, our artifact provides possibilities to add value for students, teachers, and academic leaders.


Ausubel, D. P. (1963). The Psychology of Meaningful Verbal Learning. Oxford, U.K.: Grune & Stratton.

Azevedo, R., Harley, J., Trevors, G., Duffy, M., Feyzi-Behnagh, R., Bouchet, F., & Landis, R. (2013). Using trace data to examine the complex roles of cognitive, metacognitive, and emotional self-regulatory processes during learning with multi-agent systems. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 427–449). New York: Springer.

Bodily, R., & Verbert, K. (2017). Review of research on student-facing learning analytics dashboards and educational recommender systems. IEEE Transactions on Learning Technologies, 10(4), 405–418.

Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59.

Chard, S. M. (2017). Using design science in educational technology research projects. University of Mindanao International Multidisciplinary Research Journal, 2(1).

Chmielewski, T. C., & Dansereau, D. F. (1998). Enhancing the recall of text: Knowledge mapping training promotes implicit transfer. Journal of Educational Psychology, 90(3), 407–413.

Daley, B. J., & Torre, D. M. (2010). Concept maps in medical education: An analytical literature review. Medical Education, 44(5), 440–448.

Dignath, C., & Büttner, G. (2008). Components of fostering self-regulated learning among students. A meta-analysis on intervention studies at primary and secondary school level. Metacognition and Learning, 3(3), 231–264.

Dignath-van Ewijk, C., Fabriz, S., & Büttner, G. (2015). Fostering self-regulated learning among students by means of an electronic learning diary: A training experiment. Journal of Cognitive Education and Psychology, 14(1), 77–97.

Fabriz, S., Dignath-van Ewijk, C., Poarch, G., & Büttner, G. (2014). Fostering self-monitoring of university students by means of a standardized learning journal—A longitudinal study with process analyses. European Journal of Psychology of Education, 29(2), 239–255.

Geiser, C., Götz, T., Preckel, F., & Freund, P. A. (2017). States and traits. European Journal of Psychological Assessment: Official Organ of the European Association of Psychological Assessment, 33(4), 219–223.

Greene, J. A., & Schunk, D. H. (2017). Historical, contemporary, and future perspectives on self-regulated learning and performance. In Handbook of Self-Regulation of Learning and Performance (pp. 17–32). New York: Routledge.

Gurlitt, J., & Renkl, A. (2010). Prior knowledge activation: How different concept mapping tasks lead to substantial differences in cognitive processes, learning outcomes, and perceived self-efficacy. Instructional Science, 38(4), 417–433.

Hadwin, A. F., Järvelä, S., & Miller, M. (2011). Self-regulated, co-regulated, and socially shared regulation of learning. Handbook of Self-Regulation of Learning and Performance, 30, 65–84.

Hellas, A., Ihantola, P., Petersen, A., Ajanovski, V. V., Gutica, M., Hynninen, T., Knutas, A., Leinonen, J., Messom, C., &

Liao, S. N. (2018). Predicting academic performance: A systematic literature review. In Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE 2018 Companion), 2–4 July 2018, Larnaca, Cyprus (pp. 175–199). New York: ACM.

Hevner, A. R. (2007). A three cycle view of design science research. Scandinavian Journal of Information Systems, 19(2), Article 4.

Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design science in information systems research. MIS Quarterly, 28(1), 75–105.

Järvelä, S., Kirschner, P. A., Panadero, E., Malmberg, J., Phielix, C., Jaspers, J., Koivuniemi, M., & Järvenoja, H. (2015). Enhancing socially shared regulation in collaborative learning groups: Designing for CSCL regulation tools. Educational Technology Research and Development: ETR & D, 63(1), 125–142.

Jivet, I., Scheffel, M., Drachsler, H., & Specht, M. (2017). Awareness is not enough: Pitfalls of learning analytics dashboards in the educational practice. In Data Driven Approaches in Digital Education, Proceedings of the 12th European Conference on Technology Enhanced Learning (EC-TEL 2017), 12–15 September 2017, Tallinn, Estonia (pp. 82–96). Lecture Notes in Computer Science, Springer.

Kerr, J. F. (1968). Changing the Curriculum. (J. F. Kerr, Ed.). London, U.K.: University of London Press.

Ketonen, E. E., Dietrich, J., Moeller, J., Salmela-Aro, K., & Lonka, K. (2018). The role of daily autonomous and controlled educational goals in students’ academic emotion states: An experience sampling method approach. Learning and Instruction, 53, 10–20.

Knight, S., Wise, A. F., & Chen, B. (2017). Time for change: Why learning analytics needs temporal analysis. Journal of Learning Analytics, 4(3), 7–17.

Kovanović, V., Joksimović, S., Mirriahi, N., Blaine, E., Gašević, D., Siemens, G., & Dawson, S. (2018). Understand students’ self-reflections through learning analytics. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, NSW, Australia (pp. 389–398). New York: ACM.

Klug, J., Ogrin, S., Keller, S., Ihringer, A., & Schmitz, B. (2011). A plea for self-regulated learning as a process: Modelling, measuring and intervening. Psychological Test and Assessment Modeling, 53(1), 51–72.

Marzouk, Z., Rakovic, M., Liaqat, A., Vytasek, J., Samadi, D., Stewart-Alonso, J., Ram, I., Woloshen, S., Winne, P. H., & Nesbit, J. C. (2016). What if learning analytics were based on learning science? Australasian Journal of Educational Technology, 32(6).

Munezero, M., Montero, C. S., Mozgovoy, M., & Sutinen, E. (2013). Exploiting sentiment analysis to track emotions in students’ learning diaries. In Proceedings of the 13th Koli Calling International Conference on Computing Education Research, 14–17 November 2013, Koli, Finland (pp. 145–152). New York: ACM.

Nesbit, J. C., & Adesope, O. O. (2006). Learning with concept and knowledge maps: A meta-analysis. Review of Educational Research, 76(3), 413–448.

Novak, J., & Musonda, D. (1991). A twelve-year longitudinal study of science concept learning. American Educational Research Journal, 28(1), 117–153.

Panadero, E. (2017). A review of self-regulated learning: Six models and four directions for research. Frontiers in Psychology, 8, 422.

Panadero, E., & Alonso-Tapia, J. (2013). Self-assessment: Theoretical and practical connotations. When it happens, how is it acquired and what to do to develop it in our students (in Spanish). Electronic Journal of Research in Educational Psychology, 11(2), 551–576.

Panadero, E., Jonsson, A., & Botella, J. (2017). Effects of self-assessment on self-regulated learning and self-efficacy: Four meta-analyses. Educational Research Review, 22, 74–98.

Panadero, E., Klug, J., & Järvelä, S. (2016). Third wave of measurement in the self-regulated learning field: When measurement and intervention come hand in hand. Scandinavian Journal of Educational Research, 60(6), 723–735.

Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2017). Using learning analytics to scale the provision of personalised feedback. British Journal of Educational Technology: Journal of the Council for Educational Technology, 50(1), pp. 128–138.

Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist, 37(2), 91–105.

Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M.

Zeidner (Eds.), Handbook of Self-Regulation (pp. 451–502). San Diego, CA, USA: Academic Press.

Puustinen, M., and Pulkkinen, L. (2001). Models of self-regulated learning: A review. Scandinavian Journal of Educational Research, 45, 269–286.

Schmitz, B., Klug, J., & Schmidt, M. (2011). Assessing self-regulated learning using diary measures with university students. In D. H. Schunk & B. Zimmerman (Eds.), Handbook of Self-Regulation of Learning and Performance (pp. 251–266). New York: Routledge.

Schmitz, B., & Perels, F. (2011). Self-monitoring of self-regulation during math homework behaviour using standardized diaries. Metacognition and Learning, 6(3), 255–273.

Schmitz, B., & Wiese, B. S. (2006). New perspectives for the evaluation of training sessions in self-regulated learning: Time-series analyses of diary data. Contemporary Educational Psychology, 31(1), 64–96.

Schumacher, C., & Ifenthaler, D. (2018). Features students really expect from learning analytics. Computers in Human Behavior, 78, 397–407.

Sedrakyan, G., Malmberg, J., Verbert, K., Järvelä, S., & Kirschner, P. A. (2018). Linking learning behavior analytics and learning science concepts: Designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior, in press.

Siemens, G. (2013). Learning analytics: The emergence of a discipline. The American Behavioral Scientist, 57(10), 1380–1400.

Sitzmann, T., & Ely, K. (2011). A meta-analysis of self-regulated learning in work-related training and educational attainment: What we know and where we need to go. Psychological Bulletin, 137(3), 421–442.

Strautmane, M. (2012). Concept map-based knowledge assessment tasks and their scoring criteria: An overview. In A. J. Cañas, J. D. Novak, & J. Vanhear (Eds.), Concept Maps: Theory, Methodology, Technology — Proceedings of the Fifth International Conference on Concept Mapping (Vol. 2, pp. 80–88), 17–20 September 2012, Valletta, Malta.

van den Akker, J. (2003). Curriculum perspectives: An introduction. In J. van den Akker, W. Kuiper, & U. Hameyer (Eds.), Curriculum Landscapes and Trends (pp. 1–10). Dordrecht, Netherlands: Springer.

Weber, S. (2012), Comparing key characteristics of design science research as an approach and paradigm. Proceedings of the 2012 Pacific Asia Conference on Information Systems (PACIS 2012), 11–15 July 2012, Ho Chi Minh City, Vietnam (paper 180).

Willcox, K. E., & Huang, L. (2017). Network models for mapping educational data. Design Science, 3, E18.

Winne, P. H. (2010). Bootstrapping learner’s self-regulated learning. Psychological Test and Assessment Modeling, 52(4), 472–490.

Winne, P. H. (2017). Learning analytics for self-regulated learning. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), Handbook of Learning Analytics (pp. 241–249). Society for Learning Analytics Research.

Winne, P. H., & Baker, R. S. J. d. (2013). The potentials of educational data mining for researching metacognition, motivation and self-regulated learning. JEDM|Journal of Educational Data Mining, 5(1), 1–8.

Winne, P. H., & Hadwin, A. F. (2013). nStudy: Tracing and supporting self-regulated learning in the internet. In R. Azevedo & V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 293–308). New York: Springer.

Zimmerman, B. J., & Paulsen, A. S. (1995). Self-monitoring during collegiate studying: An invaluable tool for academic self-regulation. New Directions for Teaching and Learning, 1995(63), 13–27.




How to Cite

Kivimäki, V., Pesonen, J., Romanoff, J., Remes, H., & Ihantola, P. (2019). Curricular Concept Maps as Structured Learning Diaries: Collecting Data on Self-Regulated Learning and Conceptual Thinking for Learning Analytics Applications. Journal of Learning Analytics, 6(3), 106–121.