Predictive Learning Analytics 'At Scale': Guidelines to Successful Implementation in Higher Education
Predictive Learning Analytics (PLA) aim to improve learning by identifying students at risk of failing their studies. Yet, little is known about how best to integrate and scaffold PLA initiatives into higher education institutions. Towards this end, it becomes essential to capture and analyze the perceptions of relevant educational stakeholders (i.e., managers, teachers, students) about PLA. This paper presents an “at scale” implementation of PLA at a distance learning higher education institution and details, in particular, the perspectives of 20 educational managers involved in the implementation. It concludes with a set of recommendations about how best to adopt and apply large-scale PLA initiatives in higher education.
Arbaugh, J. B. (2014). System, scholar or students? Which most influences online MBA course effectiveness? Journal of Computer Assisted Learning, 30(4), 349–362. https://doi.org/10.1111/jcal.12048
Calvert, C. (2014). Developing a model and applications for probabilities of student success: A case study of predictive analytics. Open Learning: The Journal of Open, Distance and e-Learning, 29(2), 160–173. http://doi.org/10.1080/02680513.2014.931805
Calvert, S. L., Strong, B., & Gallagher, L. (2005). Control as an engagement feature for young children’s attention to and learning of computer content. American Behavior Scientist, 48, 578–589. https://doi.org/10.1177/0002764204271507
Chandler, N. (2013). Braced for turbulence: Understanding and managing resistance to change in the higher education sector. Management, 3(5), 243–251. https://doi.org/10.5923/j.mm.20130305.01
Coetsee, L. (1993). A practical model for the management of resistance to change: An analysis of political resistance in South Africa. International Journal of Public Administration, 16(11), 1815–1837. https://doi.org/10.1080/01900699308524874
Dawson, S., Poquet, O., Colvin, C., Rogers, T., Pardo, A., & Gašević, D. (2018). Rethinking learning analytics adoption through complexity leadership theory. Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, NSW, Australia (pp. 236–244). New York: ACM. http://dx.doi.org/10.1145/3170358.3170375
Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5), 304–317. https://doi.org/10.1504/ijtel.2012.051816
Ferguson, R., & Clow, D. (2017). Where is the evidence? Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 56–65). New York: ACM. https://doi.org/10.1145/3027385.3027396
Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68–84. https://doi.org/10.1016/j.iheduc.2015.10.002
Herodotou, C., Rienties, B., Boroowa, A., Zdrahal, Z., Hlosta, M. (under review). A large-scale implementation of predictive learning analytics in higher education: The teachers’ role and perspective.
Herodotou, C., Rienties, B., Boroowa, A., Zdrahal, Z., Hlosta, M., & Naydenova, G. (2017). Implementing predictive learning analytics on a large scale: The teacher’s perspective. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 267–271). New York: ACM. https://doi.org/10.1145/3027385.3027397
Hlosta, M., Zdrahal, Z., & Zendulka, J. (2017). Ouroboros: Early identification of at-risk students without models based on legacy data. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 6–15). New York: ACM. https://doi.org/10.1145/3027385.3027449
Huptych, M., Bohuslavek, M., Hlosta, M., & Zdrahal, Z. (2017). Measures for recommendations based on past students’ activity. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 404–408). New York: ACM. https://doi.org/10.1145/3027385.3027426
Jippes, E., Steinert, Y., Pols, J., Achterkamp, M. C., van Engelen, J. M. L., & Brand, P. L. P. (2013). How do social networks and faculty development courses affect clinical supervisors’ adoption of a medical education innovation? An exploratory study. Academic Medicine, 88(3), 398–404. https://doi.org/10.1097/acm.0b013e318280d9db
Kovanović, V., Gašević, D., Dawson, S., Joksimović, S., Baker, R. S., & Hatala, M. (2015). Penetrating the black box of time-on-task estimation. Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 184–193). New York: ACM. https://doi.org/10.1145/2723576.2723623
Kvale, S. (1996). Interviews: An introduction to qualitative research interviewing. London: SAGE Publications.
Lane, N. D., Miluzzo, E., Lu, H., Peebles, D., Choudhury, T., Campbell, A. T., & College, D. (2010). Ad hoc and sensor networks: A survey of mobile phone sensing. IEEE Communications Magazine (September 2010), 140–150. Retrieved from https://www.cs.dartmouth.edu/~campbell/papers/mobile-phone-survey.pdf
Li, N., Marsh, V., Rienties, B., & Whitelock, D. (2017). Online learning experiences of new versus continuing learners: A large-scale replication study. Assessment & Evaluation in Higher Education, 42(4), 657–672. http://dx.doi.org/10.1080/02602938.2016.1176989
Macrì, D. M., Tagliaventi, M. R., & Bertolotti, F. (2002). A grounded theory for resistance to change in a small organization. Journal of Organizational Change Management, 15(3), 292–310. https://doi.org/10.1108/09534810210429327
Moolenaar, N. M., Daly, A. J., & Sleegers, P. J. C. (2010). Occupying the principal position: Examining relationships between transformational leadership, social network position, and schools’ innovative climate. Educational Administration Quarterly, 46(5), 623–670. https://doi.org/10.1177/0013161x10378689
Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, R., & Whitelock, D. (2017). Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behavior, 76, 703–714. https://doi.org/10.1016/j.chb.2017.03.028
Nyce, C. (2007). Predictive analytics. White paper. Retrieved from https://www.the-digital-insurer.com/wp-content/uploads/2013/12/78-Predictive-Modeling-White-Paper.pdf%0A
Piderit, S. K. (2000). Rethinking resistance and recognizing ambivalence: A multidimensional view of attitudes toward an organizational change. Academy of Management Review, 25(4), 783–794. https://doi.org/10.5465/amr.2000.3707722
Rienties, B. (2014). Understanding academics’ resistance towards (online) student evaluation. Assessment & Evaluation in Higher Education, 39(8), 987–1001. https://doi.org/10.1080/02602938.2014.880777
Rienties, B., Boroowa, A., Cross, S., Kubiak, C., Mayles, K., & Murphy, S. (2016). Analytics4Action evaluation framework: A review of evidence-based learning analytics interventions at the Open University UK. Journal of Interactive Media in Education, 2016(1), p. 2. http://doi.org/10.5334/jime.394
Rienties, B., Cross, S., & Zdrahal, Z. (2016). Implementing a learning analytics intervention and evaluation framework: What works? Big Data and Learning Analytics in Higher Education. Springer International Publishing. https://doi.org/10.1007/978-3-319-06520-5_10
Rienties, B., Herodotou, C., Olney, T., Schencks, M., & Boroowa, A. (2018). Making sense of learning analytics dashboards: A technology acceptance perspective of 95 teachers. The International Review of Research in Open and Distributed Learning, 19(5). https://doi.org/10.19173/irrodl.v19i5.3493
Shea, P., Sau, C., & Pickett, A. (2006). A study of teaching presence and student sense of learning community in fully online and web-enhanced college courses. The Internet and Higher Education, 9(3), 175–190. https://doi.org/10.1016/j.iheduc.2006.06.005
Sillince, J. A. A. (1999). The role of political language forms and language coherence in the organizational change process. Organization Studies, 20(3), 485–518.
Tempelaar, D. T., Rienties, B., Mittelmeier, J., & Nguyen, Q. (2017). Student profiling in a dispositional learning analytics application using formative assessment. Computers in Human Behavior, 78, 408–420. https://doi.org/10.1177/0170840699203005
Wolff, A., Zdrahal, Z., Nikolov, A., & Pantucek, M. (2013). Improving retention: Predicting at-risk students by analysing clicking behaviour in a virtual learning environment. Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), 8–12 April 2013, Leuven, Belgium (pp. 145–149). New York: ACM. https://doi.org/10.1145/2460296.2460324
Wolfram Cox, J. R. (1997). Manufacturing the past: Loss and absence in organizational change. Organization Studies, 18(4), 623–625. https://doi.org/10.1177/017084069701800404
Share this article:
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.