Analytics of Learning Strategies: Role of Course Design and Delivery Modality
DOI:
https://doi.org/10.18608/jla.2020.72.3Keywords:
learning strategies, learning tactics, self-regulated learning, data mining, modality, course designAbstract
Generalizability of the value of methods based on learning analytics remains one of the big challenges in the field of learning analytics. One approach to testing generalizability of a method is to apply it consistently in different learning contexts. This study extends a previously published work by examining the generalizability of a learning analytics method proposed for detecting learning tactics and strategies from trace data. The method was applied to the datasets collected in three different course designs and delivery modalities, including flipped classroom, blended learning, and massive open online course. The proposed method combines process mining and sequence analysis. The detected learning strategies are explored in terms of their association with academic performance. The results indicate the applicability of the proposed method across different learning contexts. Moreover, the findings contribute to the understanding of the learning tactics and strategies identified in the trace data: learning tactics proved to be responsive to the course design, whereas learning strategies were found to be more sensitive to the delivery modalities than to the course design. These findings, well aligned with self-regulated learning theory, highlight the association of learning contexts with the choice of learning tactics and strategies.
References
Baker, R. S. (2019). Challenges for the future of educational data mining: The Baker Learning Analytics Prizes. Journal of Educational Data Mining, 11(1), 1–17. https://dx.doi.org/10.5281/zenodo.3554746
Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., & Abrami, P. C. (2014, 4). A meta-analysis of blended learning and technology use in higher education: from the general to the applied. Journal of Computing in Higher Education, 26(1), 87–122. https://dx.doi.org/10.1007/s12528-013-9077-3
Biggs. (1987). Student approaches to learning and studying. Retrieved from https://eric.ed.gov/?id=ED308201
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64(1), 120928131529005. https://dx.doi.org/10.1146/annurev-psych-113011-143823
Broadbent, J. (2017). Comparing online and blended learner’s self-regulated learning strategies and academic performance. The Internet and Higher Education, 33, 24–32. https://dx.doi.org/10.1016/j.iheduc.2017.01.004
Broadbent, J., & Poon, W. L. (2015). Self-regulated learning strategies & academic achievement in online higher education learning environments: A systematic review. Internet and Higher Education, 27, 1–13. https://dx.doi.org/10.1016/j.iheduc.2015.04.007
Byrne, M., Flood, B., & Willis, P. (2010). The relationship between learning approaches and learning outcomes: A study of Irish accounting students. Accounting Education, 11, 27–42. https://dx.doi.org/10.1080/09639280210153254
Chonkar, S. P., Ha, T. C., Chu, S. S. H., Ng, A. X., Lim, M. L. S., Ee, T. X., . . . Tan, K. H. (2018). The predominant learning approaches of medical students. BMC Medical Education, 18(1), 1–8. https://dx.doi.org/10.1186/s12909-018-1122-5
Clarebout, G., Elen, J., Collazo, N. A. J., Lust, G., & Jiang, L. (2013, 1). Metacognition and the use of tools. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 187–195). New York: Springer.
Dabbagh, N. (2007). The online learner: Characteristics and pedagogical implications. Contemporary Issues in Technology and Teacher Education, 7(3), 217–226. Retrieved from https://citejournal.org/volume-7/issue-3-07/general/the-online-learner-characteristics-and-pedagogical-implications/
Derry, S. J. (1989). Putting learning strategies to work. Educational Leadership, 47(5), 4–10.
DiFrancesca, D., Nietfeld, J. L., & Cao, L. (2016). A comparison of high and low achieving students on self-regulated learning variables. Learning and Individual Differences, 45, 228–236. https://dx.doi.org/10.1016/j.lindif.2015.11.010
Diseth, A., & Martinsen, (2003). Approaches to learning, cognitive style, and motives as predictors of academic achievement. Educational Psychology, 23(2), 195–207. https://dx.doi.org/10.1080/01443410303225
Dunlosky, J. (2013). Strengthening the student toolbox: Study strategies to boost learning. American Educator, 37(3), 12–21. Retrieved from http://www.aft.org/sites/default/files/periodicals/dunlosky.pdf
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013, 1). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4–58. https://dx.doi.org/10.1177/1529100612453266
Entwistle, N. J. (1991). Approaches to learning and perceptions of the learning environment: Introduction to the special issue. Higher Eduation, 22(3), 201–204. https://dx.doi.org/10.1007/BF00132287
Entwistle, N. J. (2007, 10). Research into student learning and university teaching. The British Psychological Society, 1–18. Retrieved from https://www.ingentaconnect.com/content/bpsoc/slaut/2007/00000001/00000001/art00001
Eriksson, T., Adawi, T., & St¨ohr, C. (2017, 4). “Time is the bottleneck”: A qualitative study exploring why learners drop out of MOOCs. Journal of Computing in Higher Education, 29(1), 133–146. https://dx.doi.org/10.1007/s12528-016-9127-8
Ferreira, D. R., & Gillblad, D. (2009). Discovering process models from unlabelled event logs. Business Process Management, 5701, 143–158. https://dx.doi.org/10.1007/978-3-642-03848-811
Fincham, O. E., Gaˇsevi´c, D. V., Jovanovi´c, J. M., & Pardo, A. (2018). From study tactics to learning strategies: An analytical method for extracting interpretable representations. IEEE Transactions on Learning Technologies, 1382(c), 1–14. https://dx.doi.org/10.1109/TLT.2018.2823317
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410–8415. https://dx.doi.org/10.1073/pnas.1319030111
Gabadinho, A., Ritschard, G., Studer, M., & Muller, N. S. (2008). Mining sequence data in R with the TraMineR package: A user’s guide. Department of Econometrics and Laboratory of Demography, University of Geneva, Switzerland, 1, 1–124. Retrieved from http://mephisto.unige.ch/traminer
Gaševi´c, D., Dawson, S., Rogers, T., & Gasevi´c, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. Internet and Higher Education, 28, 68–84. https://dx.doi.org/10.1016/j.iheduc.2015.10.002
Gaševi´c, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1). https://dx.doi.org/10.1007/s11528-014-0822-x
Gaševi´c, D., Jovanovi´c, J., Pardo, A., & Dawson, S. (2017). Detecting learning strategies with analytics: Links with self-reported measures and academic performance. Journal of Learning Analytics, 4(2), 113–128. https://dx.doi.org/10.18608/jla.2017.42.10
Gaševi´c, D., Kovanovi´c, V., & Joksimovi´c, S. (2017). Piecing the learning analytics puzzle: A consolidated model of a field of research and practice. Learning: Research and Practice, 3(1), 63–78. https://dx.doi.org/10.1080/23735082.2017.1286142
Gaševi´c, D., Mirriahi, N., Dawson, S., & Joksimovi´c, S. (2017). Effects of instructional conditions and experience on the adoption of a learning tool. Computers in Human Behavior, 67, 207–220. https://dx.doi.org/10.1016/j.chb.2016.10.026
Gatta, R., Lenkowicz, J., Vallati, M., & Stefanini, A. (2017). pMineR: Processes mining in medicine. Retrieved from https://cran.r-project.org/package=pMineR
Hadwin, A. F., Nesbit, J. C., Jamieson-Noel, D., Code, J., &Winne, P. H. (2007). Examining trace data to explore self-regulated learning. Metacognition and Learning, 2(2-3), 107–124. https://dx.doi.org/10.1007/s11409-007-9016-7
Joksimovi´c, S., Kovanovi´c, V., & Dawson, S. (2019, 1). The journey of learning analytics. HERDSA Review of Higher Education, 6, 37–63. Retrieved from www.herdsa.org.au/herdsa-review-higher-education-vol-6/37-63
Jovanovi´c, J., Gaševi´c, D., Dawson, S., Pardo, A., & Mirriahi, N. (2017). Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education, 33, 74–85. https://dx.doi.org/10.1016/j.iheduc.2017.02.001
Kizilcec, R. F., P´erez-Sanagust´ın, M., & Maldonado, J. J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in massive open online courses. Computers & Education, 104, 18–33. https://dx.doi.org/http://dx.doi.org/10.1016/j.compedu.2016.10.001
Kovanovi´c, V., Gaˇsevi´c, D., Joksimovi´c, S., Hatala, M., & Olusola, A. (2015). Analytics of communities of inquiry: Effects of learning technology use on cognitive presence in asynchronous online discussions. Internet and Higher Education, 27, 74–89. https://dx.doi.org/10.1016/j.iheduc.2015.06.002
Lai, C. L., & Hwang, G. J. (2016, 9). A self-regulated flipped classroom approach to improving students’ learning performance in a mathematics course. Computers and Education, 100, 126–140. https://dx.doi.org/10.1016/j.compedu.2016.05.006
Lee, J.-s. (2014). The relationship between student engagement and academic performance: Is it a myth or reality? The Journal of Educational Research, 107(3), 177–185. https://dx.doi.org/10.1080/00220671.2013.807491
Lim, L. A., Gentili, S., Pardo, A., Kovanovi´c, V., Whitelock-Wainwright, A., Gaˇsevi´c, D., & Dawson, S. (2019). What changes, and for whom? A study of the impact of learning analytics-based process feedback in a large course. Learning and Instruction. https://dx.doi.org/10.1016/j.learninstruc.2019.04.003
Lust, G., Elen, J., & Clarebout, G. (2013a). Regulation of tool-use within a blended course: Student differences and performance effects. Computers and Education, 60(1), 385–395. https://dx.doi.org/10.1016/j.compedu.2012.09.001
Lust, G., Elen, J., & Clarebout, G. (2013b). Students’ tool-use within a web enhanced course: Explanatory mechanisms of students’ tool-use pattern. Computers in Human Behavior, 29, 2013–2021. https://dx.doi.org/10.1016/j.chb.2013.03.014
Maldonado-Mahauad, J., P´erez-Sanagust´ın, M., Kizilcec, R. F., Morales, N., & Munoz-Gama, J. (2018). Mining theory-based patterns from Big Data: Identifying self-regulated learning strategies in massive open online courses. Computers in Human Behavior, 80, 179–196. https://dx.doi.org/10.1016/j.chb.2017.11.011
Malmberg, J., J¨arvel¨a, S., & Kirschner, P. A. (2014). Elementary school students’ strategic learning: Does task-type matter? Metacognition and Learning, 9(2), 113–136. https://dx.doi.org/10.1007/s11409-013-9108-5
Malmberg, J., J¨arvenoja, H., & J¨arvel¨a, S. (2010). Tracing elementary school students’ study tactic use in gStudy by examining a strategic and self-regulated learning. Computers in Human Behavior, 26(5), 1034–1042. https://dx.doi.org/10.1016/j.chb.2010.03.004
Marton, F., & S¨alj¨o, R. (1976). On qualitative differences in learning: I—Outcome and process. British Journal of Educational Psychology, 46(1), 4–11. https://dx.doi.org/10.1111/j.2044-8279.1976.tb02980.x
Matcha, W., Ahmad Uzir, N., Gaˇsevi´c, D., & Pardo, A. (2019). A systematic review of empirical studies on learning analytics dashboards: A self-regulated learning perspective. IEEE Transactions on Learning Technologies, 1382(c), 1. https://dx.doi.org/10.1109/TLT.2019.2916802
Matcha, W., Gaˇsevi´c, D., Ahmad Uzir, N., Jovanovi´c, J., & Pardo, A. (2019). Analytics of learning strategies: Associations with academic performance and feedback. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK ’19), 4–8 March 2019, Tempe, Arizona, USA (pp. 461–470). New York: ACM. https://dx.doi.org/10.1145/3303772.3303787
Matcha, W., Gaˇsevi´c, D., Ahmad Uzir, N., Jovanovi´c, J., Pardo, A., Maldonado-Mahauad, J., & P´erez-Sanagust´ın, M. (2019). Detection of learning strategies: A comparison of process, sequence and network analytic approaches. In European Conference on Technology Enhanced Learning (EC-TEL 2019), 16–19 September, Delft, Netherlands (pp. 525–540). Springer. https://dx.doi.org/10.1007/978-3-030-29736-739
Mattick, K., Dennis, I., & Bligh, J. (2004). Approaches to learning and studying in medical students: Validation of a revised inventory and its relation to student characteristics and performance. Medical Education, 38(5), 535–543. https://dx.doi.org/10.1111/j.1365-2929.2004.01836.x
McCabe, J. (2011, 4). Metacognitive awareness of learning strategies in undergraduates. Memory & Cognition, 39(3), 462–476. https://dx.doi.org/10.3758/s13421-010-0035-2
Molenaar, I. (2014). Advances in temporal analysis in learning and instruction. Frontline Learning Research, 6, 15–24. https://dx.doi.org/10.14786/flr.v2i4.118
Nandagopal, K., & Ericsson, K. A. (2012). An expert performance approach to the study of individual differences in self-regulated learning activities in upper-level college students. Learning and Individual Differences, 22(5), 597–609. https://dx.doi.org/10.1016/j.lindif.2011.11.018
Nugent, G., Guru, A., & Namuth-Covert, D. (2018). Students’ approaches to e-learning: Analyzing credit/noncredit and high/low performers. Interdisciplinary Journal of e-Skills and Lifelong Learning, 14, 143–158. https://dx.doi.org/10.28945/4133
O’Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. Internet and Higher Education, 25, 85–95. https://dx.doi.org/10.1016/j.iheduc.2015.02.002
Olney, T., Rienties, B., & Toetenel, L. (2018). Gathering, visualising and interpreting learning design analytics to inform classroom practice and curriculum design: A student-centred approach from the Open University. In From Data and Analytics to the Classroom: Translating Learning Analytics for Teachers. London: Routledge. Retrieved from https://www.taylorfrancis.com/books/e/9781351113038/chapters/10.4324/9781351113038-6
Pardo, A. (2018). A feedback model for data-rich learning experiences. Assessment & Evaluation in Higher Education, 43(3), 428–438. https://dx.doi.org/10.1080/02602938.2017.1356905
Pardo, A., Gaˇsevi´c, D., Jovanovi´c, J. M., Dawson, S., & Mirriahi, N. (2018). Exploring student interactions with preparation activities in a flipped classroom experience. IEEE Transactions on Learning Technologies. https://dx.doi.org/10.1109/TLT.2018.2858790
Pardo, A., & Mirriahi, N. (2017). Design, deployment and evaluation of a flipped learning first year engineering course. In C. Reidsema, L. Kavanagh, R. Hadgraft, & N. Smith (Eds.), The Flipped Classroom: Practice and Practices in Higher Education (pp. 177–191). Singapore: Springer. https://dx.doi.org/10.1007/978-981-10-3413-811
P´erez-Sanagust´ın, M., Hilliger, I., Alario-Hoyos, C., Kloos, C. D., & Rayyan, S. (2017, 4). H-MOOC framework: Reusing MOOCs for hybrid education. Journal of Computing in Higher Education, 29(1), 47–64. https://dx.doi.org/10.1007/s12528-017-9133-5
Pressley, M., Borkowski, J. G., & Schneider, W. (1987). Cognitive strategies: Good strategy users coordinate metacognition and knowledge. Annals of Child Development, 4(2), 89–129.
Proctor, B. E., Prevatt, F. F., Adams, K. S., Reaser, A., & Petscher, Y. (2006). Study skills profiles of normalachieving and academically-struggling college students. Journal of College Student Development, 47(1), 37–51. https://dx.doi.org/10.1353/csd.2006.0011
Rachal, C. K., Daigle, S., & Rachal, W. S. (2007). Learning problems reported by college students: Are they using learning strategies? Journal of Instructional Psychology, 34(4), 191–199. Retrieved from https://eric.ed.gov/?id=EJ790467
Rahman, A. A., Aris, B., Rosli, M. S., Mohamed, H., Abdullah, Z., & Zaid, N. M. (2015). Significance of preparedness in flipped classroom. Advanced Science Letters, 21(10), 3388–3390. https://dx.doi.org/10.1166/asl.2015.6514
Rodr´ıguez, M. F., Correa, J. H., P´erez-Sanagust´ın, M., Pertuze, J. A., & Alario-Hoyos, C. (2017). A MOOC-based flipped class: Lessons learned from the orchestration perspective. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 10254 LNCS, pp. 102–112). Springer Verlag. https://dx.doi.org/10.1007/978-3-319-59044-812
Tessmer, M., & Richey, R. C. (1997). The role of context in learning and instructional design. Educational Technology Research and Development, 45(2), 85–115. https://dx.doi.org/10.1007/BF02299526
Winne, P. H. (2006). How software technologies can improve research on learning and bolster school reform. Educational Psychologist, 41(1), 5–17. https://dx.doi.org/10.1207/s15326985ep4101
Winne, P. H. (2013). Learning strategies, study skills, and self-regulated learning in postsecondary education. Higher Education: Handbook of Theory and Research(28), 337–403. https://dx.doi.org/10.1007/978-90-481-8598-6
Winne, P. H. (2017). Learning analytics for self-regulated learning. In C. Lang, G. Siemens, A. Wise, & D. Gaševi´c (Eds.), Handbook of learning analytics (pp. 241–249). Society for Learning Analytics Research. https://dx.doi.org/10.18608/hla17.021
Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in Educational Theory and Practice (Vol. 93, pp. 277–304). Lawrence Erlbaum Associates.
Winne, P. H., Jamieson-Noel, D., & Muis, K. (2002). Methodological issues and advances in researching tactics, strategies, and self-regulated learning. In Advances in motivation and achievement: New directions in measures and methods (Vol. 12, pp. 121–155). Emerald Group.
Yip, M. C. W. (2007). Differences in learning and study strategies between high and low achieving university students: A hong kong study. Educational Psychology, 27(4), 597–606. https://dx.doi.org/10.1080/01443410701309126
Zeegers, P. (2001). Approaches to learning in science: A longitudinal study. British Journal of Educational Psychology, 71(1), 115–132. https://dx.doi.org/10.1348/000709901158424
Zhou, M., & Winne, P. H. (2012). Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction, 22(6), 413–419. https://dx.doi.org/10.1016/j.learninstruc.2012.03.004
Zhu, Y., Au, W., & Yates, G. (2016). University students’ self-control and self-regulated learning in a blended course. Internet and Higher Education, 30, 54–62. https://dx.doi.org/10.1016/j.iheduc.2016.04.001
Zimmerman, B. J. (1986). Becoming a self-regulated learner: Which are the key subprocesses? Contemporary Educational Psychology, 11(4), 307–313. https://dx.doi.org/10.1016/0361-476X(86)90027-5
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) license that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
