Predictive Learning Analytics 'At Scale': Guidelines to Successful Implementation in Higher Education

Christothea Herodotou
Bart Rienties
Barry Verdin
Avinash Boroowa

Abstract


Predictive Learning Analytics (PLA) aim to improve learning by identifying students at risk of failing their studies. Yet, little is known about how best to integrate and scaffold PLA initiatives into higher education institutions. Towards this end, it becomes essential to capture and analyze the perceptions of relevant educational stakeholders (i.e., managers, teachers, students) about PLA. This paper presents an “at scale” implementation of PLA at a distance learning higher education institution and details, in particular, the perspectives of 20 educational managers involved in the implementation. It concludes with a set of recommendations about how best to adopt and apply large-scale PLA initiatives in higher education.


Full Text:

PDF

References

Arbaugh, J. B. (2014). System, scholar or students? Which most influences online MBA course effectiveness? Journal of Computer Assisted Learning, 30(4), 349–362. https://doi.org/10.1111/jcal.12048

Calvert, C. (2014). Developing a model and applications for probabilities of student success: A case study of predictive analytics. Open Learning: The Journal of Open, Distance and e-Learning, 29(2), 160–173. http://doi.org/10.1080/02680513.2014.931805

Calvert, S. L., Strong, B., & Gallagher, L. (2005). Control as an engagement feature for young children’s attention to and learning of computer content. American Behavior Scientist, 48, 578–589. https://doi.org/10.1177/0002764204271507

Chandler, N. (2013). Braced for turbulence: Understanding and managing resistance to change in the higher education sector. Management, 3(5), 243–251. https://doi.org/10.5923/j.mm.20130305.01

Coetsee, L. (1993). A practical model for the management of resistance to change: An analysis of political resistance in South Africa. International Journal of Public Administration, 16(11), 1815–1837. https://doi.org/10.1080/01900699308524874

Dawson, S., Poquet, O., Colvin, C., Rogers, T., Pardo, A., & Gašević, D. (2018). Rethinking learning analytics adoption through complexity leadership theory. Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, NSW, Australia (pp. 236–244). New York: ACM. http://dx.doi.org/10.1145/3170358.3170375

Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5), 304–317. https://doi.org/10.1504/ijtel.2012.051816

Ferguson, R., & Clow, D. (2017). Where is the evidence? Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 56–65). New York: ACM. https://doi.org/10.1145/3027385.3027396

Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68–84. https://doi.org/10.1016/j.iheduc.2015.10.002

Herodotou, C., Rienties, B., Boroowa, A., Zdrahal, Z., Hlosta, M. (under review). A large-scale implementation of predictive learning analytics in higher education: The teachers’ role and perspective.

Herodotou, C., Rienties, B., Boroowa, A., Zdrahal, Z., Hlosta, M., & Naydenova, G. (2017). Implementing predictive learning analytics on a large scale: The teacher’s perspective. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 267–271). New York: ACM. https://doi.org/10.1145/3027385.3027397

Hlosta, M., Zdrahal, Z., & Zendulka, J. (2017). Ouroboros: Early identification of at-risk students without models based on legacy data. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 6–15). New York: ACM. https://doi.org/10.1145/3027385.3027449

Huptych, M., Bohuslavek, M., Hlosta, M., & Zdrahal, Z. (2017). Measures for recommendations based on past students’ activity. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 404–408). New York: ACM. https://doi.org/10.1145/3027385.3027426

Jippes, E., Steinert, Y., Pols, J., Achterkamp, M. C., van Engelen, J. M. L., & Brand, P. L. P. (2013). How do social networks and faculty development courses affect clinical supervisors’ adoption of a medical education innovation? An exploratory study. Academic Medicine, 88(3), 398–404. https://doi.org/10.1097/acm.0b013e318280d9db

Kovanović, V., Gašević, D., Dawson, S., Joksimović, S., Baker, R. S., & Hatala, M. (2015). Penetrating the black box of time-on-task estimation. Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 184–193). New York: ACM. https://doi.org/10.1145/2723576.2723623

Kvale, S. (1996). Interviews: An introduction to qualitative research interviewing. London: SAGE Publications.

Lane, N. D., Miluzzo, E., Lu, H., Peebles, D., Choudhury, T., Campbell, A. T., & College, D. (2010). Ad hoc and sensor networks: A survey of mobile phone sensing. IEEE Communications Magazine (September 2010), 140–150. Retrieved from https://www.cs.dartmouth.edu/~campbell/papers/mobile-phone-survey.pdf

Li, N., Marsh, V., Rienties, B., & Whitelock, D. (2017). Online learning experiences of new versus continuing learners: A large-scale replication study. Assessment & Evaluation in Higher Education, 42(4), 657–672. http://dx.doi.org/10.1080/02602938.2016.1176989

Macrì, D. M., Tagliaventi, M. R., & Bertolotti, F. (2002). A grounded theory for resistance to change in a small organization. Journal of Organizational Change Management, 15(3), 292–310. https://doi.org/10.1108/09534810210429327

Moolenaar, N. M., Daly, A. J., & Sleegers, P. J. C. (2010). Occupying the principal position: Examining relationships between transformational leadership, social network position, and schools’ innovative climate. Educational Administration Quarterly, 46(5), 623–670. https://doi.org/10.1177/0013161x10378689

Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, R., & Whitelock, D. (2017). Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behavior, 76, 703–714. https://doi.org/10.1016/j.chb.2017.03.028

Nyce, C. (2007). Predictive analytics. White paper. Retrieved from https://www.the-digital-insurer.com/wp-content/uploads/2013/12/78-Predictive-Modeling-White-Paper.pdf%0A

Piderit, S. K. (2000). Rethinking resistance and recognizing ambivalence: A multidimensional view of attitudes toward an organizational change. Academy of Management Review, 25(4), 783–794. https://doi.org/10.5465/amr.2000.3707722

Rienties, B. (2014). Understanding academics’ resistance towards (online) student evaluation. Assessment & Evaluation in Higher Education, 39(8), 987–1001. https://doi.org/10.1080/02602938.2014.880777

Rienties, B., Boroowa, A., Cross, S., Kubiak, C., Mayles, K., & Murphy, S. (2016). Analytics4Action evaluation framework: A review of evidence-based learning analytics interventions at the Open University UK. Journal of Interactive Media in Education, 2016(1), p. 2. http://doi.org/10.5334/jime.394

Rienties, B., Cross, S., & Zdrahal, Z. (2016). Implementing a learning analytics intervention and evaluation framework: What works? Big Data and Learning Analytics in Higher Education. Springer International Publishing. https://doi.org/10.1007/978-3-319-06520-5_10

Rienties, B., Herodotou, C., Olney, T., Schencks, M., & Boroowa, A. (2018). Making sense of learning analytics dashboards: A technology acceptance perspective of 95 teachers. The International Review of Research in Open and Distributed Learning, 19(5). https://doi.org/10.19173/irrodl.v19i5.3493

Shea, P., Sau, C., & Pickett, A. (2006). A study of teaching presence and student sense of learning community in fully online and web-enhanced college courses. The Internet and Higher Education, 9(3), 175–190. https://doi.org/10.1016/j.iheduc.2006.06.005

Sillince, J. A. A. (1999). The role of political language forms and language coherence in the organizational change process. Organization Studies, 20(3), 485–518.

Tempelaar, D. T., Rienties, B., Mittelmeier, J., & Nguyen, Q. (2017). Student profiling in a dispositional learning analytics application using formative assessment. Computers in Human Behavior, 78, 408–420. https://doi.org/10.1177/0170840699203005

Wolff, A., Zdrahal, Z., Nikolov, A., & Pantucek, M. (2013). Improving retention: Predicting at-risk students by analysing clicking behaviour in a virtual learning environment. Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), 8–12 April 2013, Leuven, Belgium (pp. 145–149). New York: ACM. https://doi.org/10.1145/2460296.2460324

Wolfram Cox, J. R. (1997). Manufacturing the past: Loss and absence in organizational change. Organization Studies, 18(4), 623–625. https://doi.org/10.1177/017084069701800404



PDF


DOI: https://doi.org/10.18608/jla.2019.61.5

Share this article: