Detecting Learning Strategies with Analytics: Links with Self-reported Measures and Academic Performance

Dragan Gasevic
Jelena Jovanovic
Abelardo Pardo
Shane Dawson

Abstract


The use of analytic methods for extracting learning strategies from trace data has attracted considerable attention in the literature. However, there is a paucity of research examining any association between learning strategies extracted from trace data and responses to well-established self-report instruments and performance scores. This paper focuses on the link between the learning strategies identified in the trace data and student reported approaches to learning. The paper reports on the findings of a study conducted in the scope of an undergraduate engineering course (N=144) that followed a flipped classroom design. The study found that learning strategies extracted from trace data can be interpreted in terms of deep and surface approaches to learning. The detected significant links with self-report measures are with small effect sizes for both the overall deep approach to learning scale and the deep strategy scale. However, there was no observed significance linking the surface approach to learning and surface strategy nor were there significant associations with motivation scales of approaches to learning. The significant effects on academic performance were found, and consistent with the literature that used self-report instruments showing that students who followed a deep approach to learning had a significantly higher performance.

Keywords


Learning analytics; learning strategy; approaches to learning; self-reported measures

Full Text:

PDF HTML

References


Azevedo, R. (2015). Defining and measuring engagement and learning in science: Conceptual, theoretical, methodological, and analytical issues. Educational Psychologist, 50(1), 84–94. http://dx.doi.org/10.1080/00461520.2015.1004069

Beheshitha, S. S., Hatala, M., Gašević, D., & Joksimović, S. (2016). The role of achievement goal orientations when studying effect of learning analytics visualizations. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼLA), 25–29 April 2016, Edinburgh, UK (pp. 54–63). New York: ACM. http://dx.doi.org/10.1145/2883851.2883904

Biggs, J. (1987). Student approaches to learning and studying. (Research Monograph). Australian Council for Educational Research. Retrieved from http://eric.ed.gov/?id=ED308201

Biggs, J., Kember, D., & Leung, D. Y. P. (2001). The revised two-factor Study Process Questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71(1), 133–149. http://dx.doi.org/10.1348/000709901158433

Bjork, E. L., & Bjork, R. A. (2011). Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. In M. A. Gernsbacher, R. W. Pew, L. M. Hough, & J. R. Pomerantz (Eds.), Psychology and the real world: Essays illustrating fundamental contributions to society (pp. 56–64).

Bliuc, A.-M., Ellis, R. A., Goodyear, P., & Piggott, L. (2010). Learning through face-to-face and online discussions: Associations between students’ conceptions, approaches and academic performance in political science. British Journal of Educational Technology, 41(3), 512–524. http://dx.doi.org/10.1111/j.1467-8535.2009.00966.x

Colvin, C., Rogers, T., Wade, A., Dawson, S., Gašević, D., Buckingham Shum, S., … Fisher, J. (2015). Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement (Research Report). Canberra, Australia: Office of Learning and Teaching, Australian Government.

Dawson, S., Drachsler, H., Rosé, C. P., Gašević, D., & Lynch, G. (Eds.). (2016). Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK. New York: ACM.

Del Valle, R., & Duffy, T. M. (2009). Online learning: Learner characteristics and their approaches to managing learning. Instructional Science, 37(2), 129–149. http://dx.doi.org/10.1007/s11251-007-9039-0

Elliot, A. J., & McGregor, H. A. (2001). A 2 X 2 achievement goal framework. Journal of Personality and Social Psychology, 80, 501–519.

Ellis, R. A., Goodyear, P., Calvo, R. A., & Prosser, M. (2008). Engineering students’ conceptions of and approaches to learning through discussions in face-to-face and online contexts. Learning and Instruction, 18(3), 267–282. http://dx.doi.org/10.1016/j.learninstruc.2007.06.001

Ellis, R. A., Marcus, G., & Taylor, R. (2005). Learning through inquiry: Student difficulties with online course-based material. Journal of Computer Assisted Learning, 21(4), 239–252. http://dx.doi.org/10.1111/j.1365-2729.2005.00131.x

Entwistle, N. J. (2009). Teaching for understanding at university: Deep approaches and distinctive ways of thinking. Basingstoke, UK: Palgrave Macmillan.

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410–8415, http://dx.doi.org/10.1073/pnas.1319030111

Gabadinho, A., Ritschard, G., Mueller, N. S., & Studer, M. (2011). Analyzing and visualizing state sequences in R with TraMineR. Journal of Statistical Software, 40(4), 1–37. http://dx.doi.org/10.18637/jss.v040.i04

Gašević, D., Dawson, S., Rogers, T., & Gašević, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting learning success. The Internet and Higher Education, 28, 68–84. http://dx.doi.org/10.1016/j.iheduc.2015.10.002

Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64–71. http://dx.doi.org/10.1007/s11528-014-0822-x

Khan, I., & Pardo, A. (2016). Data2U: Scalable real time student feedback in active learning environments. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼLA), 25–29 April 2016, Edinburgh, UK (pp. 249–253). New York: ACM. http://dx.doi.org/10.1145/2883851.2883911

Kovanović, V., Gašević, D., Joksimović, S., Hatala, M., & Adesope, O. (2015). Analytics of communities of inquiry: Effects of learning technology use on cognitive presence in asynchronous online discussions. The Internet and Higher Education, 27, 74–89. http://dx.doi.org/10.1016/j.iheduc.2015.06.002

Levenshtein, V. I. (1966). Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady, 10, 707–710.

Lodge, J., & Lewis, M. (2012). Pigeon pecks and mouse clicks: Putting the learning back into learning analytics. Proceedings of the 33rd Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE 2016). 28–30 November 2016, Wellington, New Zealand (pp. 560–564). Australasian Society for Computers in Learning in Tertiary Education.

Lust, G., Elen, J., & Clarebout, G. (2013a). Regulation of tool-use within a blended course: Student differences and performance effects. Computers & Education, 60(1), 385–395. http://dx.doi.org/10.1016/j.compedu.2012.09.001

Lust, G., Elen, J., & Clarebout, G. (2013b). Students’ tool-use within a web enhanced course: Explanatory mechanisms of students’ tool-use pattern. Computers in Human Behavior, 29(5), 2013–2021. http://dx.doi.org/10.1016/j.chb.2013.03.014

Lust, G., Vandewaetere, M., Ceulemans, E., Elen, J., & Clarebout, G. (2011). Tool-use in a blended undergraduate course: In Search of user profiles. Computers & Education, 57(3), 2135–2144. http://dx.doi.org/10.1016/j.compedu.2011.05.010

Martin, A. J., Papworth, B., Ginns, P., Malmberg, L.-E., Collie, R. J., & Calvo, R. A. (2015). Real-time motivation and engagement during a month at school: Every moment of every day for every student matters. Learning and Individual Differences, 38, 26–35. http://dx.doi.org/10.1016/j.lindif.2015.01.014

Miyamoto, Y. R., Coleman, C., Williams, J. J., Whitehill, J., Nesterko, S., & Reich, J. (2015). Beyond time-on-task: The relationship between spaced study and certification in MOOCs. Journal of Learning Analytics, 2(2), 47–69. http://dx.doi.org/10.18608/jla.2015.22.5

O’Flaherty, J., Phillips, C., Karanicolas, S., Snelling, C., & Winning, T. (2015). The use of flipped classrooms in higher education: A scoping review. The Internet and Higher Education, 25, 85–95. http://dx.doi.org/10.1016/j.iheduc.2015.02.002

Pardo, A., Ellis, R. A., & Calvo, R. A. (2015). Combining observational and experiential data to inform the redesign of learning activities. Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ(L), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 305–309). New York: ACM. http://dx.doi.org/10.1145/2723576.2723625

Pardo, A., Han, F., & Ellis, R. A. (2016). Exploring the relation between self-regulation, online activities, and academic performance: A case study. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼLA), 25–29 April 2016, Edinburgh, UK (pp. 422–429). New York: ACM. http://dx.doi.org/10.1145/2883851.2883883

Pardo, A., & Mirriahi, N. (in press). Design, deployment and evaluation of a flipped learning first year engineering course. In C. Reidsema, L. Kavanagh, R. Hadgraft, & N. Smith (Eds.), Flipping the Classroom: Practice and Practices. Singapore: Springer.

Rogers, T., Gašević, D., & Dawson, S. (2016). Learning analytics and the imperative for theory driven research. In C. Haythornthwaite, R. Andrews, J. Fransma, & E. Meyers (Eds.), The SAGE Handbook of E-Learning Research, 2nd ed. (pp. 232–250). London, UK: SAGE Publications.

Siemens, G., & Gasevic, D. (2012). Guest editorial: Learning and knowledge analytics. Educational Technology & Society, 15(3), 1–2.

Svihla, V., Wester, M. J., & Linn, M. C. (2015). Distributed revisiting: An analytic for retention of coherent science learning. Journal of Learning Analytics, 2(2), 75–101. http://dx.doi.org/10.18608/jla.2015.22.7

Trigwell, K., & Prosser, M. (1991). Improving the quality of student learning: The influence of learning context and student approaches to learning on learning outcomes. Higher Education, 22(3), 251–266. http://dx.doi.org/10.1007/BF00132290

Trigwell, K., Prosser, M., & Waterhouse, F. (1999). Relations between teachers’ approaches to teaching and students’ approaches to learning. Higher Education, 37(1), 57–70. http://dx.doi.org/10.1023/A:1003548313194

van Leeuwen, A. (2015). Learning analytics to support teachers during synchronous CSCL: Balancing between overview and overload. Journal of Learning Analytics, 2(2), 138–162. http://dx.doi.org/10.18608/jla.2015.22.11

Weinstein, C. E., Husman, J., & Dierking, D. R. (2000). Self-regulation interventions with a focus on learning strategies. In P. R. Pintrich & M. Boekaerts (Eds.), Handbook on self-regulation (pp. 727–747). New York: Academic Press.

Winne, P. H., & Jamieson-Noel, D. (2002). Exploring students’ calibration of self reports about study tactics and achievement. Contemporary Educational Psychology, 27(4), 551–572. http://dx.doi.org/10.1016/S0361-476X(02)00006-1

Winne, P. H., & Jamieson-Noel, D. (2003). Self-regulating studying by objectives for learning: Students’ reports compared to a model. Contemporary Educational Psychology, 28(3), 259–276. http://dx.doi.org/10.1016/S0361-476X(02)00041-3

Wise, A. F. (2014). Designing pedagogical interventions to support student use of learning analytics. Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ʼLA), 24–28 March 2014, Indianapolis, IN, USA (pp. 203–211). New York: ACM. http://dx.doi.org/10.1145/2567574.2567588

Wise, A. F., & Shaffer, D. W. (2015). Why theory matters more than ever in the age of big data. Journal of Learning Analytics, 2(2), 5–13. http://dx.doi.org/10.18608/jla.2015.22.2

Wise, A. F., Speer, J., Marbouti, F., & Hsiao, Y.-T. (2013). Broadening the notion of participation in online discussions: Examining patterns in learners’ online listening behaviors. Instructional Science, 41(2), 323–343. http://dx.doi.org/10.1007/s11251-012-9230-9

Zhou, M., & Winne, P. H. (2012). Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction, 22(6), 413–419. http://dx.doi.org/10.1016/j.learninstruc.2012.03.004

Zimmerman, B. J. (2000). Self-efficacy: An essential motive to learn. Contemporary Educational Psychology, 25(1), 82–91. http://dx.doi.org/10.1006/ceps.1999.1016




DOI: http://dx.doi.org/10.18608/jla.2017.42.10

Refbacks

  • There are currently no refbacks.