Synergies of Learning Analytics and Learning Design: A Systematic Review of Student Outcomes




Learning design, Learning analytics, Higher education, Effect size, Analytics framework, Collaborative learning, Learning gain, Personalized learning, Online learning, Self-regulation, Student learning


The field of learning analytics (LA) has seen a gradual shift from purely data-driven approaches to more holistic views of improving student learning outcomes through data-informed learning design (LD). Despite the growing potential of LA in higher education (HE), the benefits are not yet convincing to the practitioner, in particular aspects of aligning LA data with LD toward desired learning outcomes. This review presents a systematic evaluation of effect sizes reported in 38 key studies in pursuit of effective LA approaches to measuring student learning gain for the enhancement of HE pedagogy and delivery. Large positive effects on student outcomes were found in LDs that fostered socio-collaborative and independent learning skills. Recent trends in personalization of learner feedback identified a need for the integration of student-idiosyncratic factors to improve the student experience and academic outcomes. Finally, key findings are developed into a new three-level framework, the LA Learning Gain Design (LALGD) model, to align meaningful data capture with pedagogical intentions and their learning outcomes. Suitable for various settings — face to face, blended, or fully online — the model contributes to data-informed learning and teaching pedagogies in HE.


Adamopoulos, P. (2013). What makes a great MOOC? An interdisciplinary analysis of student retention in online courses. In Proceedings of the International Conference on Information Systems (ICIS 2013): Reshaping Society Through Information Systems Design, 15–18 December 2013, Milan, Italy (Vol. 5, pp. 4720–4740). Retrieved from

Akhtar, S., Warburton, S., & Xu, W. (2017). The use of an online learning and teaching system for monitoring computer aided design student participation and predicting student success. International Journal of Technology and Design Education, 27(2), 251–270.

Alhadad, S. S., Thompson, K., Knight, S., Lewis, M., & Lodge, J. M. (2018). Analytics-enabled teaching as design: Reconceptualisation and call for research. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, Australia (pp. 427–435). New York, NY: ACM.

Baker, C. (2010). The impact of instructor immediacy and presence for online student affective learning, cognition, and motivation. Journal of Educators Online, 7(1).

Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gašević, D., Mulder, R., …, Lockyer, L. (2016). A conceptual framework linking learning design with learning analytics. In Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ’16), 25–29 April 2016, Edinburgh, UK (pp. 329–338). New York, NY: ACM.

Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32(3), 347–364.

Blikstein, P., Worsley, M., Piech, C., Sahami, M., Cooper, S., & Koller, D. (2014). Programming pluralism: Using learning analytics to detect patterns in the learning of computer programming. Journal of the Learning Sciences, 23(4), 561–599.

Blumenstein, M., Liu, D. Y. T., Richards, D., Leichtweis, S., & Stephens, J. M. (2018). Data-informed nudges for student engagement and success. In J. Lodge, J. C. Horvath, & L. Corrin (Eds.), Learning analytics in the classroom: Translating learning analytics research for teachers (pp. 185–207). London, UK: Routledge.

Calvert, C. E. (2014). Developing a model and applications for probabilities of student success: A case study of predictive analytics. Open Learning: The Journal of Open and Distance Learning, 29(2), 160–173.

Casquero, O., Ovelar, R., Romo, J., Benito, M., & Alberdi, M. (2016). Students’ personal networks in virtual and personal learning environments: A case study in higher education using learning analytics approach. Interactive Learning Environments, 24(1), 49–67.

Cheung, A. C., & Slavin, R. E. (2016). How methodological features affect effect sizes in education. Educational Researcher, 45(5), 283–292.

Cohen, J. (2013). Statistical power analysis for the behavioral sciences. New York, NY: Routledge.

Colvin, C., Rogers, T., Wade, A., Dawson, S., Gašević, D., Buckingham Shum, S., & Fisher, J. (2015). Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement. Sydney, NSW: Australian Office for Learning and Teaching. Retrieved from

Conole, G., Gašević, D., Long, P., & Siemens, G. (2011). Message from the LAK 2011 General and Program Chairs. In G. Conole, & D. Gašević (Eds.), Proceedings of the 1st International Conference on Learning Analytics and Knowledge (LAK ’11), 27 February–1 March 2011, Banff, AB, Canada. New York, NY: ACM. Retrieved from

Cooper, A. (2012). What is analytics? Definition and essential characteristics. CETIS Analytics Series, 1(5), 1–10. Retrieved from

Corrigan, O., Smeaton, A. F., Glynn, M., & Smyth, S. (2015). Using educational analytics to improve test performance. In G. Conole, T. Klobučar, C. Rensing, J. Konert, & E. Lavoué (Eds.), Design for teaching and learning in a networked world. Lecture Notes in Computer Science (Vol 9307, pp. 42–55). Cham, Switzerland: Springer.

Dawson, S., Gašević, D., Siemens, G., & Joksimović, S. (2014). Current state and future trends: A citation network analysis of the learning analytics field. In Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ’14), 24–28 March 2014, Indianapolis, IN (pp. 231–240). New York, NY: ACM.

Dawson, S., Jovanović, J., Gašević, D., & Pardo, A. (2017). From prediction to impact: Evaluation of a learning analytics retention program. In Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March, 2017, Vancouver, BC (pp. 474–478). New York, NY: ACM.

D’Mello, S., Dieterle, E., & Duckworth, A. (2017). Advanced, analytic, automated (AAA) measurement of engagement during learning. Educational Psychologist, 52(2), 104–123.

Dodge, B., Whitmer, J., & Frazee, J. P. (2015). Improving undergraduate student achievement in large blended courses through data-driven interventions. In Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ’15), 16–20 March 2015, Poughkeepsie, NY (pp. 412–413). New York, NY: ACM.

Domínguez, C., Jaime, A., Sánchez, A., Blanco, J. M., & Heras, J. (2016). A comparative analysis of the consistency and difference among online self-, peer-, external- and instructor-assessments: The competitive effect. Computers in Human Behavior, 60, 112–120.

Evans, C., Kandiko Howson, C., & Forsythe, A. (2018). Making sense of learning gain in higher education. Higher Education Pedagogies, 3(1), 1–45.

Ferguson, R., & Clow, D. (2015). Examining engagement: Analysing learner subpopulations in massive open online courses (MOOCs). In Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ’15), 16–20 March 2015, Poughkeepsie, NY (pp. 51–58). New York, NY: ACM.

Ferguson, R., & Clow, D. (2017). Where is the evidence? A call to action for learning analytics. In Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 56–65). New York, NY: ACM.

Ferguson, R., Clow, D., Griffiths, D., & Brasher, A. (2019). Moving forward with learning analytics: Expert views. Journal of Learning Analytics, 6(3), 43–59.

Gašević, D., Adesope, O., Joksimović, S., & Kovanović, V. (2015). Externally-facilitated regulation scaffolding and role assignment to develop cognitive presence in asynchronous online discussions. The Internet and Higher Education, 24, 53–65.

Gašević, D., Dawson, S., Rogers, T., & Gašević, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. Internet and Higher Education, 28, 68–84.

Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64–71.

Gašević, D., Kovanović, V., & Joksimović, S. (2017). Piecing the learning analytics puzzle: A consolidated model of a field of research and practice. Learning: Research and Practice, 3(1), 63–78.

Gašević, D., Kovanović, V., Joksimović, S., & Siemens, G. (2014). Where is research on massive open online courses headed? A data analysis of the MOOC Research Initiative. The International Review of Research in Open and Distributed Learning, 15(5).

Giesbers, B., Rienties, B., Tempelaar, D., & Gijselaers, W. (2013). Investigating the relations between motivation, tool use, participation, and performance in an e-learning course using web-videoconferencing. Computers in Human Behavior, 29(1), 285–292.

Goggins, S., & Xing, W. L. (2016). Building models explaining student participation behavior in asynchronous online discussion. Computers & Education, 94, 241–251.

Gunn, C., McDonald, J., Donald, L., Nichols, M., Milne, J., & Blumenstein, M. (2017). Building an evidence base for teaching and learning design using learning analytics (Report). Wellington, New Zealand: Ako Aotearoa — The National Centre for Tertiary Teaching Excellence. Retrieved from

Gunnarsson, B. L., & Alterman, R. (2012). Predicting failure: A case study in co-blogging. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ’12), 29 April–2 May 2012, Vancouver, BC, Canada (pp. 263–266). New York, NY: ACM.

Hattie, J. (2015). The applicability of visible learning to higher education. Scholarship of Teaching and Learning in Psychology, 1(1), 79–91. Retrieved from

Huang, T. C. K., Huang, C. H., & Chuang, Y. T. (2016). Change discovery of learning performance in dynamic educational environments. Telematics and Informatics, 33(3), 773–792.

Ifenthaler, D., Adcock, A. B., Erlandson, B. E., Gosper, M., Greiff, S., & Pirnay-Dummer, P. (2014). Challenges for education in a connected world: Digital learning, data rich environments, and computer-based assessment — Introduction to the inaugural special issue of Technology, Knowledge and Learning. Technology, Knowledge and Learning, 19(1-2), 121–126.

Jayaprakash, S. M., Moody, E. W., Lauría, E. J., Regan, J. R., & Baron, J. D. (2014). Early alert of academically at-risk students: An open source analytics initiative. Journal of Learning Analytics, 1(1), 6–47.

Jo, I.-H., Park, Y., Yoon, M., & Sung, H. (2016). Evaluation of online log variables that estimate learners’ time management in a Korean online learning context. International Review of Research in Open and Distance Learning, 17(1), 195–213.

Joksimović, S., Gašević, D., Kovanović, V., Riecke, B. E., & Hatala, M. (2015). Social presence in online discussions as a process predictor of academic performance. Journal of Computer Assisted Learning, 31(6), 638–654.

Joksimović, S., Gašević, D., Loughin, T., Kovanović, V., & Hatala, M. (2015). Learning at distance: Effects of interaction traces on academic achievement. Computers & Education, 87, 204–217.

Joksimović, S., Poquet, O., Kovanović, V., Dowell, N., Mills, C., Gašević, D., & Brooks, C. (2018). How do we model learning at scale? A systematic review of research on MOOCs. Review of Educational Research, 88(1), 43–86.

Jovanović, J., Gašević, D., Pardo, A., Dawson, S., & Whitelock-Wainwright, A. (2019). Introducing meaning to clicks: Towards traced-measures of self-efficacy and cognitive load. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK ’19), 4–8 March 2019, Tempe, AZ (pp. 511–520). New York: ACM.

Kim, D., Yoon, M., Jo, I. H., & Branch, R. M. (2018). Learning analytics to support self-regulated learning in asynchronous online courses: A case study at a women’s university in South Korea. Computers and Education, 127, 233–251.

Kitchenham, B., & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. EBSE Technical Report v2.3. Keele, Staffs and Durham, UK: Keele University and University of Durham [Joint Report]. Retrieved from

Laurillard, D., Charlton, P., Craft, B., Dimakopoulos, D., Ljubojevic, D., Magoulas, G., & Whittlestone, K. (2013). A constructionist learning environment for teachers to model learning designs. Journal of Computer Assisted Learning, 29(1), 15–30.

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press.

Li, K. C., Lam, H. K., & Lam, S. S. (2015). A review of learning analytics in educational research. In J. Lam, K. Ng, S. Cheung, T. Wong, K. Li, & F. Wang (Eds.), Technology in education. Technology-mediated proactive learning. ICTE 2015. Communications in Computer and Information Science, vol. 559 (pp. 173–184). New York, NY: Springer.

Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439–1459.

Macfadyen, L., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588–599.

Mangaroska, K., & Giannakos, M. (2019). Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning. IEEE Transactions on Learning Technologies, 12(4), 516–534.

Martin, F., & Whitmer, J. C. (2016). Applying learning analytics to investigate timed release in online learning. Technology, Knowledge and Learning, 21(1), 59–74.

Martínez-Muñoz, G., & Pulido, E. (2015). Using a SPOC to flip the classroom. Paper presented at the IEEE Global Engineering Education Conference, EDUCON, 18–20 March 2015, Tallinn, Estonia. Retrieved from

McConnell, M. (2019). Using OnTask in a large first year Commercial Law course on a Bachelor of Commerce degree. In A. Pardo, D. Y. T. Liu, L. Vigentini, & M. Blumenstein (Eds.), Scaling personalised student communication — Current initiatives and future directions (Workshop). Australian Learning Analytics Summer Institute (ALASI), University of Wollongong, NSW. Retrieved from

Milliron, M. D., Malcolm, L., & Kil, D. (2014). Insight and action analytics: Three case studies to consider. Research & Practice in Assessment, 9(Winter), 72–82. Retrieved from

Mor, Y., Ferguson, R., & Wasson, B. (2015). Learning design, teacher inquiry into student learning and learning analytics: A call for action. British Journal of Educational Technology, 46(2), 221–229.

Muuro, M. E., Oboko, R., & Wagacha, P. (2016). Evaluation of intelligent grouping based on learners’ collaboration competence level in online collaborative learning environment. International Review of Research in Open and Distance Learning, 17(2).

Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, R., & Whitelock, D. (2017). Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behavior, 76, 703–714.

Ott, C., Robins, A., Haden, P., & Shephard, K. (2015). Illustrating performance indicators and course characteristics to support students’ self-regulated learning in CS1. Computer Science Education, 25(2), 174–198.

Papamitsiou, Z., & Economides, A. A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Journal of Educational Technology & Society, 17(4), 49–64. Retrieved from

Pardo, A., Bartimote-Aufflick, K., Buckingham Shum, S., Dawson, S., Gao, J., Gašević, D., …, Vigentini, L. (2018). OnTask: Delivering data-informed personalized learning support actions. Journal of Learning Analytics, 5(3), 235–249.

Pardo, A., Han, F., & Ellis, R. A. (2017). Combining university student self-regulated learning indicators and engagement with online learning events to predict academic performance. IEEE Transactions on Learning Technologies, 10(1), 82–92.

Pardo, A., Jovanović, J., Dawson, S., Gašević, D., & Mirriahi, N. (2019). Using learning analytics to scale the provision of personalised feedback. British Journal of Educational Technology, 50(1), 128–138.

Redecker, C., Leis, M., Leendertse, M., Punie, Y., Gijsbers, G., Kirschner, P. A., …, Hoogveld, B. (2011). The future of learning: Preparing for change. Seville, Spain: JRC IPTS. Retrieved from

Reimann, P. (2016). Connecting learning analytics with learning research: The role of design-based research. Learning: Research and Practice, 2(2), 130–142.

Reschly, A. L., & Christenson, S. L. (2012). Jingle, jangle, and conceptual haziness: Evolution and future directions of the engagement construct. In S. Christenson, A. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 3–19). Boston, MA: Springer.

Rienties, B., Boroowa, A., Cross, S., Kubiak, C., Mayles, K., & Murphy, S. (2016). Analytics4Action Evaluation Framework: A review of evidence-based learning analytics interventions at the Open University UK. Journal of Interactive Media in Education, 2016(1).

Rienties, B., & Alden, B. (2014). Emotions used in learning analytics: A state-of-the-art review. Measuring and Understanding Learner Emotions: Evidence and Prospects (2). LACE Project. Retrieved from

Rienties, B., & Toetenel, L. (2016). The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules. Computers in Human Behavior, 60, 333–341.

Rienties, B., Toetenel, L., & Bryan, A. (2015). “Scaling up” learning design: Impact of learning design activities on LMS behavior and performance. In Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ’15), 16–20 March 2015, Poughkeepsie, NY (pp. 315–319). New York, NY: ACM.

Romero, C., López, M. I., Luna, J. M., & Ventura, S. (2013). Predicting students’ final performance from participation in on-line discussion forums. Computers and Education, 68, 458–472.

Romero-Zaldivar, V. A., Pardo, A., Burgos, D., & Delgado Kloos, C. (2012). Monitoring student progress using virtual appliances: A case study. Computers and Education, 58(4), 1058–1067.

Rosenthal, R., & DiMatteo, M. R. (2001). Meta-analysis: Recent developments in quantitative methods for literature reviews. Annual Review of Psychology, 52(1), 59–82.

Rubio, F., Thomas, J. M., & Li, Q. (2018). The role of teaching presence and student participation in Spanish blended courses. Computer Assisted Language Learning, 31(3), 226–250.

Schneider, M., & Preckel, F. (2017). Variables associated with achievement in higher education: A systematic review of meta-analyses. Psychological Bulletin, 143(6), 565–600. Retrieved from

Schumacher, C., & Ifenthaler, D. (2018). The importance of students’ motivational dispositions for designing learning analytics. Journal of Computing in Higher Education, 30(3), 599–619.

Sønderlund, A. L., Hughes, E., & Smith, J. (2018). The efficacy of learning analytics interventions in higher education: A systematic review. British Journal of Educational Technology, 50(5), 2594–2618.

Tabuenca, B., Kalz, M., Drachsler, H., & Specht, M. (2015). Time will tell: The role of mobile learning analytics in self-regulated learning. Computers and Education, 89, 53–74.

Tempelaar, D., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: Learning analytics in a data-rich context. Computers in Human Behavior, 47, 157–167.

Tempelaar, D., Rienties, B., Mittelmeier, J., & Nguyen, Q. (2018). Student profiling in a dispositional learning analytics application using formative assessment. Computers in Human Behavior, 78, 408–420.

Vermunt, J. D. (1996). Metacognitive, cognitive and affective aspects of learning styles and strategies: A phenomenographic analysis. Higher Education, 31(1), 25–50.

Vermunt, J. D., Ilie, S., & Vignoles, A. (2018). Building the foundations for measuring learning gain in higher education: A conceptual framework and measurement instrument. Higher Education Pedagogies, 3(1), 266–301.

Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98–110.

West, D., Huijser, H., & Heath, D. (2018). Blurring the boundaries: Developing leadership in learning analytics. In J. Lodge, J. C. Horvath, & L. Corrin (Eds.), Learning analytics in the classroom: Translating learning analytics research for teachers (pp. 267–283). London, UK: Routledge.

You, J. W. (2015). Examining the effect of academic procrastination on achievement using LMS data in e-learning. Journal of Educational Technology & Society, 18(3), 64–74. Retrieved from

Yu, T., & Jo, I. H. (2014). Educational technology approach toward learning analytics: Relationship between student online behavior and learning performance in higher education. In Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ’14), 24–28 March 2014, Indianapolis, IN (pp. 269–270). New York, NY: ACM

Zhang, J.-H., Zou, L.-c., Miao, J.-j., Zhang, Y.-X., Hwang, G.-J., & Zhu, Y. (2019). An individualized intervention approach to improving university students’ learning performance and interactive behaviors in a blended learning environment. Interactive Learning Environments, 28(2), 231–245.

Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory into Practice, 41(2), 64–70.




How to Cite

Blumenstein, M. (2020). Synergies of Learning Analytics and Learning Design: A Systematic Review of Student Outcomes. Journal of Learning Analytics, 7(3), 13-32.



Special Section: Learning Design and Learning Analytics