Are We Living In LA (P)LA Land?

Reporting on the Practice of 30 STEM Tutors in their Use of a Learning Analytics Implementation at the Open University




tutor practice, learning analytics implementation, social informatics, shadow practice


Most higher education institutions view their increasing use of learning analytics as having significant potential to improve student academic achievement, retention outcomes, and learning and teaching practice but the realization of this potential remains stubbornly elusive. While there is an abundance of published research on the creation of visualizations, dashboards, and predictive models, there has been little work done to explore the impact of learning analytics on the actual practice of teachers. Through the lens of social informatics (an approach that views the users of technologies as active social actors whose technological practices constitute a wider socio-technical system) this qualitative study reports on an investigation into the practice of 30 tutors in the STEM faculty at Europe’s largest distance learning organization, The Open University UK (OU). When asked to incorporate learning analytics (including predictive learning analytics) contained in the Early Alert Indicator (EAI) dashboard during the 2017–2018 academic year into their practice, we found that tutors interacted with this dashboard in certain unanticipated ways and developed three identifiable “shadow practices”.


Agnihotri, L., & Ott, A. (2014). Building a student at-risk model: An end-to-end perspective from user to data scientist. In J. Stamper et al. (Eds.), Proceedings of the 7th International Conference on Educational Data Mining (EDM2014), 4–7 July 2014, London, UK (pp. 209–212). International Educational Data Mining Society. Retrieved from

Arthars, N., Dollinger, M., Vigentini, L., Liu, Y.-T., Kondo, E., & King, D. (2019). Empowering teachers to personalise learning support. In D. Ifenthaler, J. Yau, & D. Mah (Eds), Utilising learning analytics to support study success. Cham, Switzerland: Springer.

Bartimote, K., Pardo, A., & Reimann, P. (2019). The perspective realism brings to learning analytics in the classroom. In J. M. Lodge, J. C. Horvath & L. Corrin (Eds.), Learning analytics in the classroom: Translating learning analytics research for teachers. Abingdon-on-Thames, UK: Routledge.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.

Calvert, C. (2014). Developing a model and applications for probabilities of student success: A case study of predictive analytics. Open Learning: The Journal of Open, Distance and e-Learning, 29(2), 160–173.

Castaneda, L., & Selwyn, N. (2018). More than tool? Making sense of the ongoing digitizations of higher education. International Journal of Education Technology in Higher Education, 15(22), 1–10.

Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education, 18(6), 683–695.

Conole, G. (2012). Designing for learning in an open world. Dordrecht, Netherlands: Springer.

Creanor, L., & Walker, S. (2012). Learning technology in context: A case for the sociotechnical interaction framework as an analytical lens for networked learning research. In L. Dirckinck-Holmfeld, V. Hodgson, & D. McConnell (Eds.), Exploring the pedagogy and practice of networked learning (pp. 173–187). London, UK: Springer.

Dalziel, J., Conole, G., Wills, S., Walker, S., Bennett, S., Dobozy, E., Cameron, L., Badilescu-Buga, E., & Bower, M. (2016). The Larnaca declaration on learning design. Journal of Interactive Media in Education, 1(7), 1–24.

Daniel, B. (2015). Big data and analytics in higher education: Opportunities and challenges. British Journal of Educational Technology, 46(5), 904–920.

Davis, F. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340.

Edwards, P. (2018). We have been assimilated: Some principles for thinking about algorithmic systems. In U. Schultze, M. Aanestad, M. Mähring, C. Østerlund, & K. Riemer (Eds.), Living with monsters? Social implications of algorithmic phenomena, hybrid agency, and the performativity of technology. Proceedings of the Working Conference on Information Systems and Organizations (IS&O 2018), 11–12 December 2018, San Francisco, CA, USA (pp. 19–27). IFIP Advances in Information and Communication Technology, vol 543. Cham, Switzerland: Springer.

Ferguson, R., Clow, D., Griffiths, D., & Brasher, A. (2019). Moving forward with learning analytics: Expert views. Journal of Learning Analytics, 6(3), 43–59.

Foster, E. (2015). What have we learnt from implementing learning analytics at NTU? Retrieved from

Gašević, D., Dawson, S., Rogers, T., & Gašević, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting learning success. Internet and Higher Education, 28, 68–84.

Herodotou, C., Hlosta, M., Boroowa, A., Rienties, B., Zdrahal, Z., & Mangafa, C. (2019c). Empowering online teachers through predictive learning analytics. British Journal of Educational Technology, 50(6), 3064–3079.

Herodotou, C., Rienties, B., Boroowa, A., Zdrahal, Z., & Hlosta, M. (2019a). A large-scale implementation of PLA in higher education: The teacher’s role and perspective. Educational Technology Research and Development, 67, 1273–1306.

Herodotou, C., Rienties, B., Hlosta, M., Boroowa, A., & Mangafa, C. (2020). The scalable implementation of predictive learning analytics at a distance learning university: Insights from a longitudinal case study. The Internet and Higher Education, 45, 1–3.

Herodotou, C., Rienties, B., Verdin, B., & Boroowa, A. (2019b). Predictive Learning Analytics “at scale”: Towards guidelines to successful implementation in higher education based on the case of The Open University UK. Journal of Learning Analytics, 6(1), 85–95.

Hidalgo, R. (2018). Analytics for action: Using data analytics to support students in improving their learning outcomes. In G. Ubachs & L. Konings (Eds.), The envisioning report for empowering universities (2nd ed.), (pp. 6–8). Maastricht: EADTU. Retrieved from

Hu, Y., Lo, C., & Shih, S. (2014). Developing early warning systems to predict students’ online learning performance. Computers in Human Behaviour, 36, 469–478.

Jivet, I., Scheffel, M., Spect, M., & Drachsler, H. (2018). License to evaluate: Preparing learning analytics dashboards for educational practice. Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, NSW, Australia (pp. 31–40). New York: ACM.

Klein, C., Lester, J., Rangwala, H., & Johri, A. (2019). Learning analytics tools in higher education: Adoption at the Intersection of institutional commitment and individual action. The Review of Higher Education, 42(2), 565–593. Baltimore, MD: John Hopkins University Press.

Kling, R. (2007). What is social informatics and why does it matter? The Information Society, 23(4), 205–220.

Knight, S., & Buckingham Shum, S. (2017). Theory and learning analytics. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), The handbook of learning analytics, (pp. 17–22). Beaumont, AB: Society for Learning Analytics Research (SoLAR).

Kop, R., Fournier, H., & Guillaume, D. (2017). A critical perspective on learning analytics and educational data mining. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), The handbook of learning analytics, (pp. 319–326). Beaumont, AB: Society for Learning Analytics Research (SoLAR).

Kuzilek, J., Hlosta, M., Herrmannova, D., Zdrahal, Z., & Wolff, A. (2015). OU analyse: Analysing at-risk students at The Open University. Learning Analytics Review, LAK15-1, 1–16. Retrieved from

Lamb, R., & Kling, R. (2003). Reconceptualising users as social actors in information systems research. MIS Quarterly, 27, 197–235.

Lodge, J., Horvath, J., & Corrin, L. (2019). Introduction. In J. M. Lodge, J. C. Horvath & L. Corrin (Eds.), Learning analytics in the classroom: Translating learning analytics research for teachers. Abingdon-on-Thames, UK: Routledge.

Long, P., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. Educause Review, September/October, 31–40. Retrieved from

Maguire, M., & Delahunt, B. (2017). Doing a thematic analysis: A practical, step-by-step guide for learning and teaching scholars. All Ireland Journal of Higher Education 9(3), 33501–33514. Retrieved from

McCoy, C., & Rosenbaum, H. (2019). Uncovering unintended and shadow practices of users of decision support systems decision support in higher education institutions, Journal of the American Association for Information Science and Technology, 70(4), 370–384.

Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, F., & Whitelock, D. (2017). Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behaviour, 76, 703–714.

Olney, T., Rienties, B., & Toetenel, L. (2019). Gathering, visualising and interpreting learning design analytics to inform classroom practice and curriculum design: Learning analytics in the classroom. In J. M. Lodge, J. C. Horvath & L. Corrin (Eds.), Learning analytics in the classroom: Translating learning analytics research for teachers, (pp. 71–92). Abingdon-on-Thames, UK: Routledge.

Orlikowski, W., & Gash, D. (1994). Technological frames: Making sense of information technology in organizations. ACM Transactions on Information Systems, 12(2) 174–207.

Piderit, S. (2000). Rethinking resistance and recognizing ambivalence: A multidimensional view of attitudes toward an organizational change. The Academy of Management Review, 25(4), 783–794.

Rienties, B., Boroowa, A., Cross, S., Kubiak, C., Mayles, K., & Murphy, S. (2016). Analytics4Action evaluation framework: A review of evidence-based learning analytics interventions at Open University UK. Journal of Interactive Media in Education, 1(2), 1–12.

Rienties, B., Herodotou, C., Olney, T., Schencks, M., & Boroowa, A. (2018). Making sense of learning analytics dashboards: A technology acceptance perspective of 95 teachers. The International Review of Research in Open and Distributed Learning, 19(5).

Sawyer, S., & Hartswood, M. (2014). Advancing social informatics. In P. Fichman & H. Rosenbaum (Eds.), Social informatics: Past, present and future (pp. 197–209). Newcastle upon Tyne, UK: Cambridge Scholars Publishing.

Schumacher, C., & Ifenthaler, D. (2018). Features students really expect from learning analytics. Computers in Human Behaviour, 78, 397–407.

Schwendimann, B., Rodríguez-Triana, M. J., Vozniuk, A., Prieto, L. P., Shirvani Boroujeni, M., Holzer, A., Gillet, D., & Dillenbourg, P. (2017). Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies, 10(1), 30–41.

Sclater, N., Peasgood, A., & Mullan, J. (2016, April 19). Learning analytics in higher education: A review of UK and international practice. Jisc. Retrieved from

Selwyn, N. (2019). What’s the problem with learning analytics? Journal of Learning Analytics, 6(3), 11–19.

Shacklock, X. (2016). From bricks to clicks: The potential of data and analytics in higher education. Policy Connect. Higher Education Commission. Retrieved from

SHEILA. (2018). Supporting higher education to integrate learning analytics framework. Retrieved from

Simpson, O. (2006). Predicting student success in open and distance learning. Open Learning: The Journal of Open, Distance and e-Learning, 21(2), 125–138.

Slade, S. & Boroowa, A. (2014). Policy on ethical use of student data for learning analytics. Milton Keynes, UK: The Open University. Retrieved from

Suchman, L. (1996). Supporting articulation work. In R. Kling (Ed.), Computerization and controversy: Value conflicts and social choices (2nd ed), (pp. 407–423). San Francisco, CA: Morgan Kaufmann.

Tanes, Z., Arnold, K., King, A., & Remnet, M. (2011). Using signals for appropriate feedback: Perceptions and practices. Computers & Education, 57(4), 2414–2422.

TEF. (2019). Teaching excellence and student outcomes framework. Retrieved from

Tsai, Y.-S., Moreno-Marcos, P., Jivet, I., Scheffel, M., Tammets, K., Kollom, K., & Gašević, D. (2018). The SHEILA framework: Informing institutional strategies and policy processes of learning analytics. Journal of Learning Analytics, 5(3), 5–20.

van Harmelen, M., & Workman, D. (2012). Analytics for learning and teaching. Technical Report in CETIS Analytics Series, vol. 1, no. 3. Bolton, UK: CETIS.

Walker, S., & Creanor, L. (2009). The STIN in the tale: A sociotechnical interaction perspective on networked learning. Journal of Educational Technology and Society, 12(4), 305–316. Retrieved from

Walker, S., Olney, T., Wood, C., Clarke, A., & Dunworth, M. (2019). How do tutors use data to support their students? Open Learning: The Journal of Open, Distance and e-Learning, 34, 118–133.

White, S., & White, S. (2016). Learning designers in the “third space”: The socio-technical construction of MOOCs and their relationship to educator and learning designer roles in HE. Journal of Interactive Media in Education, 2016(1) 1–12.

Williamson, B. (2016). Digital education governance: Data visualization, predictive analytics, and real-time policy instruments. Journal of Education Policy, 31(2), 123–141.

Wise, A., & Vytasek, J. (2017). Learning analytics implementation design. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), The handbook of learning analytics, (pp. 151–160). Beaumont, AB: Society for Learning Analytics Research (SoLAR).




How to Cite

Olney, T., Walker, S. ., Wood, C., & Clarke, A. . (2021). Are We Living In LA (P)LA Land? Reporting on the Practice of 30 STEM Tutors in their Use of a Learning Analytics Implementation at the Open University. Journal of Learning Analytics, 1-15.



Research Papers