Analytics of Learning Strategies: Role of Course Design and Delivery Modality


  • Wannisa Matcha School of Informatics, University of Edinburgh, and Faculty of Communication Sciences, Prince of Songkla University
  • Dragan Gašević Faculty of Information Technology, Monash University, and School of Informatics, University of Edinburgh
  • Nora'ayu Ahmad Uzir School of Informatics, University of Edinburgh, and Universiti Teknologi MARA
  • Jelena Jovanović Faculty of Organizational Sciences, University of Belgrade
  • Abelardo Pardo University of South Australia
  • Lisa Lim University of South Australia
  • Jorge Maldonado-Mahauad Pontificia Universidad Cat´ólica de Chile, and Universidad de Cuenca
  • Sheridan Gentili Teaching Innovation Unit, University of South Australia
  • Mar Pérez-Sanagustín Pontificia Universidad católica de Chile, and Université Paul Sabatier Toulouse III, Institute de Recherche en Informatique de Toulouse (IRIT)
  • Yi-Shan Tsai School of Informatics, University of Edinburgh



learning strategies, learning tactics, self-regulated learning, data mining, modality, course design


Generalizability of the value of methods based on learning analytics remains one of the big challenges in the field of learning analytics. One approach to testing generalizability of a method is to apply it consistently in different learning contexts. This study extends a previously published work by examining the generalizability of a learning analytics method proposed for detecting learning tactics and strategies from trace data. The method was applied to the datasets collected in three different course designs and delivery modalities, including flipped classroom, blended learning, and massive open online course. The proposed method combines process mining and sequence analysis. The detected learning strategies are explored in terms of their association with academic performance. The results indicate the applicability of the proposed method across different learning contexts. Moreover, the findings contribute to the understanding of the learning tactics and strategies identified in the trace data: learning tactics proved to be responsive to the course design, whereas learning strategies were found to be more sensitive to the delivery modalities than to the course design. These findings, well aligned with self-regulated learning theory, highlight the association of learning contexts with the choice of learning tactics and strategies.


Baker, R. S. (2019). Challenges for the future of educational data mining: The Baker Learning Analytics Prizes. Journal of Educational Data Mining, 11(1), 1–17.

Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., & Abrami, P. C. (2014, 4). A meta-analysis of blended learning and technology use in higher education: from the general to the applied. Journal of Computing in Higher Education, 26(1), 87–122.

Biggs. (1987). Student approaches to learning and studying. Retrieved from

Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64(1), 120928131529005.

Broadbent, J. (2017). Comparing online and blended learner’s self-regulated learning strategies and academic performance. The Internet and Higher Education, 33, 24–32.

Broadbent, J., & Poon, W. L. (2015). Self-regulated learning strategies & academic achievement in online higher education learning environments: A systematic review. Internet and Higher Education, 27, 1–13.

Byrne, M., Flood, B., & Willis, P. (2010). The relationship between learning approaches and learning outcomes: A study of Irish accounting students. Accounting Education, 11, 27–42.

Chonkar, S. P., Ha, T. C., Chu, S. S. H., Ng, A. X., Lim, M. L. S., Ee, T. X., . . . Tan, K. H. (2018). The predominant learning approaches of medical students. BMC Medical Education, 18(1), 1–8.

Clarebout, G., Elen, J., Collazo, N. A. J., Lust, G., & Jiang, L. (2013, 1). Metacognition and the use of tools. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 187–195). New York: Springer.

Dabbagh, N. (2007). The online learner: Characteristics and pedagogical implications. Contemporary Issues in Technology and Teacher Education, 7(3), 217–226. Retrieved from

Derry, S. J. (1989). Putting learning strategies to work. Educational Leadership, 47(5), 4–10.

DiFrancesca, D., Nietfeld, J. L., & Cao, L. (2016). A comparison of high and low achieving students on self-regulated learning variables. Learning and Individual Differences, 45, 228–236.

Diseth, A., & Martinsen, (2003). Approaches to learning, cognitive style, and motives as predictors of academic achievement. Educational Psychology, 23(2), 195–207.

Dunlosky, J. (2013). Strengthening the student toolbox: Study strategies to boost learning. American Educator, 37(3), 12–21. Retrieved from

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013, 1). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4–58.

Entwistle, N. J. (1991). Approaches to learning and perceptions of the learning environment: Introduction to the special issue. Higher Eduation, 22(3), 201–204.

Entwistle, N. J. (2007, 10). Research into student learning and university teaching. The British Psychological Society, 1–18. Retrieved from

Eriksson, T., Adawi, T., & St¨ohr, C. (2017, 4). “Time is the bottleneck”: A qualitative study exploring why learners drop out of MOOCs. Journal of Computing in Higher Education, 29(1), 133–146.

Ferreira, D. R., & Gillblad, D. (2009). Discovering process models from unlabelled event logs. Business Process Management, 5701, 143–158.

Fincham, O. E., Gaˇsevi´c, D. V., Jovanovi´c, J. M., & Pardo, A. (2018). From study tactics to learning strategies: An analytical method for extracting interpretable representations. IEEE Transactions on Learning Technologies, 1382(c), 1–14.

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410–8415.

Gabadinho, A., Ritschard, G., Studer, M., & Muller, N. S. (2008). Mining sequence data in R with the TraMineR package: A user’s guide. Department of Econometrics and Laboratory of Demography, University of Geneva, Switzerland, 1, 1–124. Retrieved from

Gaševi´c, D., Dawson, S., Rogers, T., & Gasevi´c, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. Internet and Higher Education, 28, 68–84.

Gaševi´c, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1).

Gaševi´c, D., Jovanovi´c, J., Pardo, A., & Dawson, S. (2017). Detecting learning strategies with analytics: Links with self-reported measures and academic performance. Journal of Learning Analytics, 4(2), 113–128.

Gaševi´c, D., Kovanovi´c, V., & Joksimovi´c, S. (2017). Piecing the learning analytics puzzle: A consolidated model of a field of research and practice. Learning: Research and Practice, 3(1), 63–78.

Gaševi´c, D., Mirriahi, N., Dawson, S., & Joksimovi´c, S. (2017). Effects of instructional conditions and experience on the adoption of a learning tool. Computers in Human Behavior, 67, 207–220.

Gatta, R., Lenkowicz, J., Vallati, M., & Stefanini, A. (2017). pMineR: Processes mining in medicine. Retrieved from

Hadwin, A. F., Nesbit, J. C., Jamieson-Noel, D., Code, J., &Winne, P. H. (2007). Examining trace data to explore self-regulated learning. Metacognition and Learning, 2(2-3), 107–124.

Joksimovi´c, S., Kovanovi´c, V., & Dawson, S. (2019, 1). The journey of learning analytics. HERDSA Review of Higher Education, 6, 37–63. Retrieved from

Jovanovi´c, J., Gaševi´c, D., Dawson, S., Pardo, A., & Mirriahi, N. (2017). Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education, 33, 74–85.

Kizilcec, R. F., P´erez-Sanagust´ın, M., & Maldonado, J. J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in massive open online courses. Computers & Education, 104, 18–33.

Kovanovi´c, V., Gaˇsevi´c, D., Joksimovi´c, S., Hatala, M., & Olusola, A. (2015). Analytics of communities of inquiry: Effects of learning technology use on cognitive presence in asynchronous online discussions. Internet and Higher Education, 27, 74–89.

Lai, C. L., & Hwang, G. J. (2016, 9). A self-regulated flipped classroom approach to improving students’ learning performance in a mathematics course. Computers and Education, 100, 126–140.

Lee, J.-s. (2014). The relationship between student engagement and academic performance: Is it a myth or reality? The Journal of Educational Research, 107(3), 177–185.

Lim, L. A., Gentili, S., Pardo, A., Kovanovi´c, V., Whitelock-Wainwright, A., Gaˇsevi´c, D., & Dawson, S. (2019). What changes, and for whom? A study of the impact of learning analytics-based process feedback in a large course. Learning and Instruction.

Lust, G., Elen, J., & Clarebout, G. (2013a). Regulation of tool-use within a blended course: Student differences and performance effects. Computers and Education, 60(1), 385–395.

Lust, G., Elen, J., & Clarebout, G. (2013b). Students’ tool-use within a web enhanced course: Explanatory mechanisms of students’ tool-use pattern. Computers in Human Behavior, 29, 2013–2021.

Maldonado-Mahauad, J., P´erez-Sanagust´ın, M., Kizilcec, R. F., Morales, N., & Munoz-Gama, J. (2018). Mining theory-based patterns from Big Data: Identifying self-regulated learning strategies in massive open online courses. Computers in Human Behavior, 80, 179–196.

Malmberg, J., J¨arvel¨a, S., & Kirschner, P. A. (2014). Elementary school students’ strategic learning: Does task-type matter? Metacognition and Learning, 9(2), 113–136.

Malmberg, J., J¨arvenoja, H., & J¨arvel¨a, S. (2010). Tracing elementary school students’ study tactic use in gStudy by examining a strategic and self-regulated learning. Computers in Human Behavior, 26(5), 1034–1042.

Marton, F., & S¨alj¨o, R. (1976). On qualitative differences in learning: I—Outcome and process. British Journal of Educational Psychology, 46(1), 4–11.

Matcha, W., Ahmad Uzir, N., Gaˇsevi´c, D., & Pardo, A. (2019). A systematic review of empirical studies on learning analytics dashboards: A self-regulated learning perspective. IEEE Transactions on Learning Technologies, 1382(c), 1.

Matcha, W., Gaˇsevi´c, D., Ahmad Uzir, N., Jovanovi´c, J., & Pardo, A. (2019). Analytics of learning strategies: Associations with academic performance and feedback. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK ’19), 4–8 March 2019, Tempe, Arizona, USA (pp. 461–470). New York: ACM.

Matcha, W., Gaˇsevi´c, D., Ahmad Uzir, N., Jovanovi´c, J., Pardo, A., Maldonado-Mahauad, J., & P´erez-Sanagust´ın, M. (2019). Detection of learning strategies: A comparison of process, sequence and network analytic approaches. In European Conference on Technology Enhanced Learning (EC-TEL 2019), 16–19 September, Delft, Netherlands (pp. 525–540). Springer.

Mattick, K., Dennis, I., & Bligh, J. (2004). Approaches to learning and studying in medical students: Validation of a revised inventory and its relation to student characteristics and performance. Medical Education, 38(5), 535–543.

McCabe, J. (2011, 4). Metacognitive awareness of learning strategies in undergraduates. Memory & Cognition, 39(3), 462–476.

Molenaar, I. (2014). Advances in temporal analysis in learning and instruction. Frontline Learning Research, 6, 15–24.

Nandagopal, K., & Ericsson, K. A. (2012). An expert performance approach to the study of individual differences in self-regulated learning activities in upper-level college students. Learning and Individual Differences, 22(5), 597–609.

Nugent, G., Guru, A., & Namuth-Covert, D. (2018). Students’ approaches to e-learning: Analyzing credit/noncredit and high/low performers. Interdisciplinary Journal of e-Skills and Lifelong Learning, 14, 143–158.

O’Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. Internet and Higher Education, 25, 85–95.

Olney, T., Rienties, B., & Toetenel, L. (2018). Gathering, visualising and interpreting learning design analytics to inform classroom practice and curriculum design: A student-centred approach from the Open University. In From Data and Analytics to the Classroom: Translating Learning Analytics for Teachers. London: Routledge. Retrieved from

Pardo, A. (2018). A feedback model for data-rich learning experiences. Assessment & Evaluation in Higher Education, 43(3), 428–438.

Pardo, A., Gaˇsevi´c, D., Jovanovi´c, J. M., Dawson, S., & Mirriahi, N. (2018). Exploring student interactions with preparation activities in a flipped classroom experience. IEEE Transactions on Learning Technologies.

Pardo, A., & Mirriahi, N. (2017). Design, deployment and evaluation of a flipped learning first year engineering course. In C. Reidsema, L. Kavanagh, R. Hadgraft, & N. Smith (Eds.), The Flipped Classroom: Practice and Practices in Higher Education (pp. 177–191). Singapore: Springer.

P´erez-Sanagust´ın, M., Hilliger, I., Alario-Hoyos, C., Kloos, C. D., & Rayyan, S. (2017, 4). H-MOOC framework: Reusing MOOCs for hybrid education. Journal of Computing in Higher Education, 29(1), 47–64.

Pressley, M., Borkowski, J. G., & Schneider, W. (1987). Cognitive strategies: Good strategy users coordinate metacognition and knowledge. Annals of Child Development, 4(2), 89–129.

Proctor, B. E., Prevatt, F. F., Adams, K. S., Reaser, A., & Petscher, Y. (2006). Study skills profiles of normalachieving and academically-struggling college students. Journal of College Student Development, 47(1), 37–51.

Rachal, C. K., Daigle, S., & Rachal, W. S. (2007). Learning problems reported by college students: Are they using learning strategies? Journal of Instructional Psychology, 34(4), 191–199. Retrieved from

Rahman, A. A., Aris, B., Rosli, M. S., Mohamed, H., Abdullah, Z., & Zaid, N. M. (2015). Significance of preparedness in flipped classroom. Advanced Science Letters, 21(10), 3388–3390.

Rodr´ıguez, M. F., Correa, J. H., P´erez-Sanagust´ın, M., Pertuze, J. A., & Alario-Hoyos, C. (2017). A MOOC-based flipped class: Lessons learned from the orchestration perspective. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 10254 LNCS, pp. 102–112). Springer Verlag.

Tessmer, M., & Richey, R. C. (1997). The role of context in learning and instructional design. Educational Technology Research and Development, 45(2), 85–115.

Winne, P. H. (2006). How software technologies can improve research on learning and bolster school reform. Educational Psychologist, 41(1), 5–17.

Winne, P. H. (2013). Learning strategies, study skills, and self-regulated learning in postsecondary education. Higher Education: Handbook of Theory and Research(28), 337–403.

Winne, P. H. (2017). Learning analytics for self-regulated learning. In C. Lang, G. Siemens, A. Wise, & D. Gaševi´c (Eds.), Handbook of learning analytics (pp. 241–249). Society for Learning Analytics Research.

Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in Educational Theory and Practice (Vol. 93, pp. 277–304). Lawrence Erlbaum Associates.

Winne, P. H., Jamieson-Noel, D., & Muis, K. (2002). Methodological issues and advances in researching tactics, strategies, and self-regulated learning. In Advances in motivation and achievement: New directions in measures and methods (Vol. 12, pp. 121–155). Emerald Group.

Yip, M. C. W. (2007). Differences in learning and study strategies between high and low achieving university students: A hong kong study. Educational Psychology, 27(4), 597–606.

Zeegers, P. (2001). Approaches to learning in science: A longitudinal study. British Journal of Educational Psychology, 71(1), 115–132.

Zhou, M., & Winne, P. H. (2012). Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction, 22(6), 413–419.

Zhu, Y., Au, W., & Yates, G. (2016). University students’ self-control and self-regulated learning in a blended course. Internet and Higher Education, 30, 54–62.

Zimmerman, B. J. (1986). Becoming a self-regulated learner: Which are the key subprocesses? Contemporary Educational Psychology, 11(4), 307–313.




How to Cite

Matcha, W., Gašević, D., Ahmad Uzir, N., Jovanović, J., Pardo, A., Lim, L., Maldonado-Mahauad, J., Gentili, S., Pérez-Sanagustín, M., & Tsai, Y.-S. (2020). Analytics of Learning Strategies: Role of Course Design and Delivery Modality. Journal of Learning Analytics, 7(2), 45-71.



Extended Conference Papers

Most read articles by the same author(s)

1 2 3 4 > >>