Learning Analytics in Physics Education

Equity-Focused Decision-Making Lacks Guidance!





equity, principles, ethics, bias, critical consciousness, learning progression, responsible learning analytics, research paper


Learning Analytics are an academic field with promising usage scenarios for many educational domains. At the same time, learning analytics come with threats such as the amplification of historically grown inequalities. A range of general guidelines for more equity-focused learning analytics have been proposed but fail to provide sufficiently clear guidance for practitioners. With this paper, we attempt to address this theory–practice gap through domain-specific (physics education) refinement of the general guidelines We propose a process as a starting point for this domain-specific refinement that can be applied to other domains as well. Our point of departure is a domain-specific analysis of historically grown inequalities in order to identify the most relevant diversity categories and evaluation criteria. Through two focal points for normative decision-making, namely equity and bias, we analyze two edge cases and highlight where domain-specific refinement of general guidance is necessary. Our synthesis reveals a necessity to work towards domain-specific standards and regulations for bias analyses and to develop counter-measures against (intersectional) discrimination. Ultimately, this should lead to a stronger equity-focused practice in future.


Aikins, M. A., Bremberger, T., Aikins, J. K., Gyamerah, D., & Yıldırım-Caliman, D. (2021). Afrozensus 2020: Perspektiven, Anti-Schwarze Rassismuserfahrungen und Engagement Schwarzer, afrikanischer und afrodiasporischer Menschen in Deutschland. https://afrozensus.de

Artificial intelligence, platform work and gender equality. (2022). European Institute for Gender Equality. https://doi.org/10.2839/372863

Avraamidou, L. (2019). “I am a young immigrant woman doing physics and on top of that I am Muslim”: Identities, intersections, and negotiations. Journal of Research in Science Teaching, 57, 311–341. https://doi.org/10.1002/tea.21593

Baggett, H. C. (2020). Relevance, Representation, and Responsibility: Exploring World Language Teachers’ Critical Consciousness and Pedagogies. L2 Journal, 12(2), 34–54. https://doi.org/10.5070/L212246037

Baker, R., & Hawn, A. (2021). Algorithmic Bias in Education. https://doi.org/10.1007/s40593-021-00285-9

Bergner, Y. (2017). Chapter 3: Measurement and its Uses in Learning Analytics. In Handbook of Learning Analytics (1st ed., pp. 35–48). SoLAR. https://doi.org/10.18608/hla17.003

Broughan, C., & Prinsloo, P. (2020). (Re)centring students in learning analytics: In conversation with Paulo Freire. Assessment & Evaluation in Higher Education, 45(4), 617–628. https://doi.org/10.1080/02602938.2019.1679716

Carlone, H. B., & Johnson, A. (2007). Understanding the Science Experiences of Succesful Women of Color: Science Identity as an Analytic Lens. Journal of Research in Science Teaching, 44(8), 1187–1218. https://doi.org/10.1002/tea.20237

Cerratto Pargman, T., & McGrath, C. (2021). Mapping the Ethics of Learning Analytics in Higher Education: A Systematic Literature Review of Empirical Research. Journal of Learning Analytics, 8(2), 123–139. https://doi.org/10.18608/jla.2021.1

Cerratto Pargman, T., McGrath, C., Viberg, O., Kitto, K., Knight, S., & Ferguson, R. (2021). Responsible Learning Analytics: Creating just, ethical, and caring LA systems. Companion Proceedings. LAK21. https://www.solaresearch.org/wp-content/uploads/2021/04/LAK21_CompanionProceedings.pdf

Cheuk, T. (2021). Can AI be racist? Color‐evasiveness in the application of machine learning to science assessments. Science Education, 1–12. https://doi.org/10.1002/sce.21671

Costanza-Chock, S. (2020). Design justice: Community-led practices to build the worlds we need. The MIT Press.

D’Ignazio, C., & Klein, L. (2020). Introduction: Why Data Science Needs Feminism. In Data Feminism. https://data-feminism.mitpress.mit.edu/pub/frfa9szd/release/6

Dou, R., Hazari, Z., Dabney, K., Sonnert, G., & Sadler, P. (2019). Early informal STEM experiences and STEMidentity: The importance of talking science. Science Education, 103, 623–637. https://doi.org/10.1002/sce.21499

Drachsler, H., & Greller, W. (2016). Privacy and Analytics – it’s a DELICATE Issue. LAK 16, Edinburgh. http://dx.doi.org/10.1145/2883851.2883893

Düchs, G., & Ingold, G.-L. (2018). Frauenanteil bleibt stabil. Physik Journal, 17(8/9), 32–37.

Duncan, R. G., & Rivet, A. E. (2018). Learning progressions. In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, & P. Reimann (Eds.), International Handbook of the Learning Sciences (pp. 422–432). Routledge.

El-Mafaalani, A. (2021). Mythos Bildung (2nd ed.). Kiepenheuer & Witsch (KiWi).

Erden, D. (2020). KI und Beschäftigung: Das Ende menschlicher Vorurteile oder der Beginn von Diskriminierung 2.0? In Wenn KI, dann feministisch (pp. 77–90). netzforma* eV. https://netzforma.org/publikation-wenn-ki-dann-feministisch-impulse-aus-wissenschaft-und-aktivismus

Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., Schafer, B., Valcke, P., & Vayena, E. (2018). AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations. Minds & Machines, 28(4), 689–707. https://doi.org/10.1007/s11023-018-9482-5

Francis, P., Broughan, C., Foster, C., & Wilson, C. (2020). Thinking critically about learning analytics, student outcomes, and equity of attainment. Assessment & Evaluation in Higher Education, 45(6), 811–821. https://doi.org/10.1080/02602938.2019.1691975

Freire, P. (1970). Pedagogy of the Oppressed. Penguin Random House UK.

Gardner, J., Brooks, C., & Baker, R. (2019). Evaluating the Fairness of Predictive Student Models Through Slicing Analysis. LAK19: Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 225–234. https://doi.org/10.1145/3303772.3303791

Godwin, A. (2016). The Development of a Measure of Engineering Identity. 2016 ASEE Annual Conference & Exposition, New Orleans, Louisiana. https://doi.org/10.18260/p.26122

Gutachten der Datenethikkommission der Bundesregierung (pp. 1–32). (2019). [Kurzfassung]. DEK. https://www.bmi.bund.de/SharedDocs/downloads/DE/publikationen/themen/it-digitalpolitik/gutachten-datenethikkommission-kurzfassung.pdf?__blob=publicationFile&v=4

Khalil, M., Prinsloo, P., & Slade, S. (2022). A Comparison of Learning Analytics Frameworks: A Systematic Review. LAK22: LAK22: 12th International Learning Analytics and Knowledge Conference, 152–163. https://doi.org/10.1145/3506860.3506878

Kitto, K., & Knight, S. (2019). Practical ethics for building learning analytics. British Journal of Educational Technolog, 50(6), 2855–2870. https://doi.org/10.1111/bjet.12868

Kracke, N., Buck, D., & Middendorff, E. (2018). Beteiligung an Hochschulbildung, Chancen(un)gleichheit in Deutschland. DZHW Brief, 3. https://doi.org/10.34878/2018.03.dzhw_brief

Lohaus, M., Perrot, M., & von Luxburg, U. (2020). Too Relaxed to Be Fair. Proceedings of Machine Learning Research, 119, 6360--6369. https://proceedings.mlr.press/v119/lohaus20a.html

Mitchell, S., Potash, E., D´Amour, A., & Lum, K. (2021). Algorithmic Fairness: Choices, Assumptions, and Definitions. Annual Review of Statistics and Its Application, 8, 141–163. https://doi.org/10.1146/annurev-statistics-042720-125902

Mujtaba, T., & Reiss, M. J. (2013). Inequality in Experiences of Physics Education: Secondary School Girls’ and Boys’ Perceptions of their Physics Education and Intentions to Continue with Physics After the Age of 16. International Journal of Science Education, 35(11), 1824–1845. https://doi.org/10.1080/09500693.2012.762699

Neue deutsche Medienmacher (NdM)—Glossar. (2021). https://glossar.neuemedienmacher.de/

Odoi, N. (2021). Die Farbe der Gerechtigkeit ist weiß | bpb. bpb.de. https://www.bpb.de/gesellschaft/migration/afrikanische-diaspora/59470/rassismus-im-strafrechtssystem

OECD. (2016). Excellence and equity in education (Volume I; PISA 2015 Results). OECD.

OECD. (2018). Where All Students Can Succeed (Volume II; PISA 2018 Results). OECD.

Ogette, T. (2019). Exit racism (5th ed.). unrast-Verlag.

Phillips, P. J., Hahn, C. A., Fontana, P. C., Broniatowski, D. A., & Przybocki, M. A. (2020). Four Principles of Explainable Artificial Intelligence. National Institute of Standards and Technology. https://doi.org/10.6028/NIST.IR.8312-draft

Prinsloo, P., & Slade, S. (2018). Mapping responsible learning analytics: A critical proposal. In Responsible Analytics & Data Mining in Education: Global Perspectives on Quality, Support, and Decision-Making. Routledge.

Rainey, K., Dancy, M., Mickelson, R., Stearns, E., & Moller, S. (2018). Race and gender differences in how senseof belonging influences decisions to majorin STEM. International Journal of STEM Education, 5. https://doi.org/10.1186/s40594-018-0115-6

Rosa, K., & Moore Mensah, F. (2016). Educational pathways of Black women physicists: Stories of experiencing and overcoming obstacles in life. Physical Review Physics Education Research, 12(2), Article 2. https://doi.org/10.1103/PhysRevPhysEducRes.12.020113

Sclater, N. (2014). Code of practice for learning analytics (pp. 1–64) [Literature review]. Jisc. https://repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf

Slack, D., Hilgard, H., Jia, E., Singh, S., & Lakkaraju, H. (2020). Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods. Proceedings of the 2020 AAAI/ACM Conference on AI, Ethics, and Society (AIES ’20), 180–186. https://doi.org/10.1145/3375627.3375830

Slade, S. (2016). The Open University Ethical use of Student Data for Learning Analytics Policy. The Open University. https://doi.org/10.13140/RG.2.1.1317.4164

Smith, T., Avraamidou, L., & Adams, J. D. (2022). Culturally relevant/responsive and sustaining pedagogies in science education: Theoretical perspectives and curriculum implications. https://doi.org/10.1007/s11422-021-10082-4

SoLAR. (2022). What is Learning Analytics? Society for Learning Analytics Research (SoLAR). https://www.solaresearch.org/about/what-is-learning-analytics/

Steegh, A. M., Höffler, T. N., Keller, M. M., & Parchmann, I. (2019). Gender differences in mathematics and science competitions: A systematic review. Journal of Research in Science Teaching, 56(10), Article 10. https://doi.org/10.1002/tea.21580

Suresh, H., & Guttag, J. (2021). A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle. EAAMO ’21: Equity and Access in Algorithms, Mechanisms, and Optimization, 1–9. https://doi.org/10.1145/3465416.3483305

Traag, V. A., & Waltman, L. (2022). Causal foundations of bias, disparity and fairness. ArXiv. https://doi.org/10.48550/arXiv.2207.13665

Wise, A. F., Sarmiento, J. P., & Boothe Jr., M. (2021). Subversive Learning Analytics. 639–645. https://doi.org/10.1145/3448139.3448210

Yeung, K. (2019). Responsibility and AI (DGI(2019)05; Issue DGI(2019)05). Council of Europe. https://rm.coe.int/responsability-and-ai-en/168097d9c5

Zhai, X., Haudek, K. C., Shi, L., Nehm, R. H., & Urban-Lurain, M. (2019). From substitution to redefinition: A framework of machine learning-based science assessment. Journal of Research in Science Teaching, 57, 1430–1459. https://doi.org/10.1002/tea.21658




How to Cite

Grimm, A., Steegh, A., Kubsch, M., & Neumann, K. (2023). Learning Analytics in Physics Education: Equity-Focused Decision-Making Lacks Guidance!. Journal of Learning Analytics, 10(1), 71-84. https://doi.org/10.18608/jla.2023.7793



Special Section on Fairness, Equity, and Responsibility in Learning Analytics