Students’ Perceptions of Generative AI-Powered Learning Analytics in the Feedback Process
A Feedback Literacy Perspective
DOI:
https://doi.org/10.18608/jla.2025.8609Keywords:
generative AI, learning analytics, feedback engagement, feedback interaction, feedback literacy, research paperAbstract
This paper explores the integration of generative AI (GenAI) in the feedback process in higher education through a learning analytics (LA) tool, examined from a feedback literacy perspective. Feedback literacy refers to students’ ability to understand, evaluate, and apply feedback effectively to improve their learning, which is crucial for fostering self-regulated learning and academic growth. GenAI has the potential to open new avenues of research and design in augmenting feedback practices by providing innovative, personalized, and scalable feedback solutions. The study investigates how GenAI functionalities, specifically ChatGPT explanation features and GenAI-powered dashboard visualizations, can support students in engaging with feedback. Using feedback literacy theory, a thematic analysis was conducted and triangulated with usage trace data to assess students’ perceptions of these functionalities. The study involved three key activities: introductory lab sessions, in-semester use of the GenAI-powered LA feedback tool, and post hoc interviews, with data collected from 18 students from various disciplines (information technology, education, business and economics, and engineering) throughout all phases. Initial findings from the lab sessions showed positive perceptions of the GenAI functionalities. However, trace data from in-semester use indicated modest engagement with GenAI. Post hoc interviews revealed that reduced engagement was due to a mismatch between the GenAI outputs and student expectations. While some students appreciated the GenAI functionalities, others found them redundant when they perceived the feedback as clear and easy to understand. This study highlights the potential of GenAI in the feedback process and underscores the challenges of aligning AI tools with diverse student needs. Future developments should focus on creating adaptive and discipline-specific GenAI solutions.
References
Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: The challenge of design. Assessment & Evaluation in Higher Education, 38(6), 698–712. https://doi.org/10.1080/02602938.2012.691462
Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315–1325. https://doi.org/10.1080/02602938.2018.1463354
Conijn, R., Martinez-Maldonado, R., Knight, S., Buckingham Shum, S., Van Waes, L., & van Zaanen, M. (2022). How to provide automated feedback on the writing process? a participatory approach to design writing analytics tools. Computer Assisted Language Learning, 35(8), 1838–1868.
Crain, P., Lee, J., Yen, Y.-C., Kim, J., Aiello, A., & Bailey, B. (2023). Visualizing topics and opinions helps students interpret large collections of peer feedback for creative projects. ACM Transactions on Computer-Human Interaction, 30(3), 1–30.
Dai, W., Lin, J., Jin, H., Li, T., Tsai, Y.-S., Gasevic, D., & Chen, G. (2023). Can large language models provide feedback to students? A case study on ChatGPT. In M. Chang, N.-S. Chen, R. Kuo, G. Rudolph, D. G. Sampson, & A. Tlili (Eds.), Proceedings of the 2023 IEEE International Conference on Advanced Learning Technologies (ICALT 2023), 10–13 July 2023, Orem, Utah, USA (pp. 323–325). IEEE. https://doi.org/10.1109/ICALT58122.2023.00100
Dai, W., Tsai, Y.- S., Lin, J., Aldino, A., Jin, H., Li, T., Gasevic, D., & Chen, G. (2024). Assessing the proficiency of large language models in automatic feedback generation: An evaluation study. Computers and Education: Artificial Intelligence, 7, 100299. https://doi.org/10.1016/j.caeai.2024.100299
Darvishi, A., Khosravi, H., Sadiq, S., Gasevic, D., & Siemens, G. (2024). Impact of AI assistance on student agency. Computers & Education, 210, 104967. https://doi.org/10.1016/j.compedu.2023.104967
Escalante, J., Pack, A., & Barrett, A. (2023). Ai-generated feedback on writing: Insights into efficacy and enl student preference. International Journal of Educational Technology in Higher Education, 20(1), 57. https://doi.org/10.1186/s41239-023-00425-2
Filius, R. M., de Kleijn, R. A., Uijl, S. G., Prins, F. J., van Rijen, H. V., & Grobbee, D. E. (2018). Strengthening dialogic peer feedback aiming for deep learning in SPOCs. Computers & Education, 125, 86–100. https://doi.org/10.1016/j.compedu.2018.06.004
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
Henderson, M., Ajjawi, R., Boud, D., & Molloy, E. (2019). Identifying feedback that has impact. In M. Henderson, R. Ajjawi, D. Boud, & E. Molloy (Eds.), The impact of feedback in higher education (pp. 15–34). Springer International Publishing. https://doi.org/10.1007/978-3-030-25112-3_2
Jin, F., Maheshi, B., Martinez-Maldonado, R., Gasevic, D., & Tsai, Y.-S. (2024). Scaffolding feedback literacy: Designing a feedback analytics tool with students. Journal of Learning Analytics, 1–15. https://doi.org/10.18608/jla.2024.8339
Jin, H., Martinez-Maldonado, R., Li, T., Chan, P. W. K., & Tsai, Y.-S. (2022). Towards Supporting Dialogic Feedback Processes Using Learning Analytics: the Educators’ Views on Effective Feedback. In S. Wilson, N. Arthars, D. Wardak, P. Yeoman, E. Kalman, & D. Y. Liu (Eds.), ASCILITE 2022 Conference Proceedings: Reconnecting Relationships through Technology, 4–7 December 2022, Sydney, Australia (e22054). https://doi.org/10.14742/apubs.2022.54
Jurgensmeier, L., & Skiera, B. (2024). Generative ai for scalable feedback to multimodal exercises. International journal of research in marketing. https://doi.org/10.1016/j.ijresmar.2024.05.005
Keane, T., Linden, T., Hernandez-Martinez, P., Molnar, A., & Blicblau, A. (2023). Digital technologies: Students’ expectations and experiences during their transition from high school to university. Education and Information Technologies, 28(1), 857–877. https://doi.org/10.1007/s10639-022-11184-4
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gasevic, ́D. (2022). Explainable artificial intelligence in education. Computers and Education: Artificial Intelligence, 3, 100074. https://doi.org/10.1016/j.caeai.2022.100074
Kim, H.-S., Cha, Y., & Kim, N. Y. (2021). Effects of AI chatbots on EFL students’ communication skills. The Korean Association for the Study of English Language and Linguistics, 21, 712–734. https://doi.org/10.15738/kjell.21..202108.712
Knight, S., Shibani, A., Abel, S., Gibson, A., Ryan, P., Sutton, N., Wight, R., Lucas, C., Sandor, A., Kitto, K., Liu, M., Mogarkar, R. V., & Buckingham Shum, S. (2020). AcaWriter: A learning analytics tool for formative feedback on academic writing. Journal of Writing Research, 12(1), 141–186. https://doi.org/10.17239/JOWR-2020.12.01.06
Lazar, J., Feng, J. H., & Hochheiser, H. (2017). Analyzing qualitative data. In Research methods in human computer interaction (pp. 299–327). Elsevier. https://doi.org/10.1016/B978-0-12-805390-4.00011-X
Li, Q., Jung, Y., & Friend Wise, A. (2021). Beyond first encounters with analytics: Questions, techniques and challenges in instructors’ sensemaking. LAK21: 11th international learning analytics and knowledge conference, 344–353. https://doi.org/10.1145/3448139.3448172
Li, T., Nath, D., Cheng, Y., Fan, Y., Li, X., Rakovic, M., Khosravi, H., Swiecki, Z., Tsai, Y.- S., & Gasevic, D. (2025). Turning real-time analytics into adaptive scaffolds for self-regulated learning using generative artificial intelligence. In Proceedings of the 15th International Conference on Learning Analytics and Knowledge (LAK 2025), 3–7 March 2025, Dublin, Ireland (pp. 667–679). ACM. https://doi.org/10.1145/3706468.3706559
Lim, L.- A., Dawson, S., Gasevic, D., Joksimovic, S., Pardo, A., Fudge, A., & Gentili, S. (2021). Students’ perceptions of, and emotional responses to, personalised learning analytics-based feedback: An exploratory study of four courses. Assessment and Evaluation in Higher Education, 46(3), 339–359. https://doi.org/10.1080/02602938.2020.1782831
Molloy, E., Boud, D., & Henderson, M. (2020). Developing a learning-centred framework for feedback literacy. Assessment & Evaluation in Higher Education, 45(4), 527–540. https://doi.org/10.1080/02602938.2019.1667955
Pardo, A., Jovanovic, J., Dawson, S., Gasevic, D., & Mirriahi, N. (2019). Using learning analytics to scale the provision of personalised feedback. British Journal of Educational Technology, 50(1), 128–138. https://doi.org/10.1111/BJET.12592
Rad, H. S., Alipour, R., & Jafarpour, A. (2023). Using artificial intelligence to foster students’ writing feedback literacy, engagement, and outcome: A case of wordtune application. Interactive Learning Environments, 1–21.
Sedrakyan, G., Malmberg, J., Verbert, K., Ja ̈rvela ̈, S., & Kirschner, P. A. (2020). Linking learning behavior analytics and learning science concepts: Designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior, 107, 105512. https://doi.org/10.1016/j.chb.2018.05.004_13
Sutton, P. (2009). Towards dialogic feedback. Critical and Reflective Practice in Education, 1(1), 1–10. https://marjon.repository. guildhe.ac.uk/id/eprint/17582/1/Towards%20dialogic%20feedback.pdf
Tsai, Y. - S. (2022). Why feedback literacy matters for learning analytics. In C. Chinn, E. Tan, C. Chan, & Y. Kali (Eds.), Proceedings of the 16th International Conference of the Learning Sciences (ICLS 2022), 6–10 June 2022, Hiroshima, Japan (pp. 27–34). International Society of the Learning Sciences. https://doi.org/10.48550/arXiv.2209.00879
Tubino, L., & Adachi, C. (2022). Developing feedback literacy capabilities through an AI automated feedback tool. In S. Wilson, N. Arthars, D. Wardak, P. Yeoman, E. Kalman, & D. Y. Liu (Eds.), ASCILITE 2022 Conference Proceedings: Reconnecting Relationships through Technology, 4–7 December 2022, Sydney, Australia (e22039-1–e22039-5). https://doi.org/10.14742/apubs.2022.39
Whitelock, D., Twiner, A., Richardson, J. T., Field, D., & Pulman, S. (2015). OpenEssayist: A supply and demand learning analytics tool for drafting academic essays. In Proceedings of the Fifth International Conference on Learning Analytics and Knowledge (LAK 2015), 16–20 March 2015, Poughkeepsie, New York, USA (pp. 208–212). ACM. https://doi.org/10.1145/2723576.2723599
Winstone, N. (2019). Facilitating students’ use of feedback: Capturing and tracking impact using digital tools. In M. Henderson, R. Ajjawi, D. Boud, & E. Molloy (Eds.), The impact of feedback in higher education (pp. 225–242). Springer International Publishing. https://doi.org/10.1007/978-3-030-25112-3_13
Yan, L., Greiff, S., Teuber, Z., & Gasevic, D. (2024). Promises and challenges of generative artificial intelligence for human learning. Nature Human Behaviour, 8(10), 1839–1850.
Yan, L., Martinez-Maldonado, R., Echeverria, V., Jin, Y., Alfredo, R., Milesi, M., Zhao, L., Fan, J., & Gasevic, D. (2024). The effects of generative AI agents and scaffolding on enhancing students’ comprehension of visual learning analytics. arXiv preprint arXiv:2409.11645. https://doi.org/10.48550/arXiv.2409.11645
Yan, L., Martinez-Maldonado, R., & Gasevic, D. (2024). Generative artificial intelligence in learning analytics: Contextualising opportunities and challenges through the learning analytics cycle. In Proceedings of the 14th International Conference on Learning Analytics and Knowledge (LAK 2024), 18–22 March 2024, Tokyo, Japan (pp. 101–111). ACM. https://doi.org/10.1145/3636555.3636856
Yan, L., Sha, L., Zhao, L., Li, Y., Martinez-Maldonado, R., Chen, G., Li, X., Jin, Y., & Gasevic, D. (2024). Practical and ethical challenges of large language models in education: A systematic scoping review. British Journal of Educational Technology, 55(1), 90–112. https://doi.org/10.1111/bjet.13370
Yan, L., Zhao, L., Echeverria, V., Jin, Y., Alfredo, R., Li, X., Gasevic, D., & Martinez-Maldonado, R. (2024). izChat: Enhancing learning analytics dashboards with contextualised explanations using multimodal generative AI chatbots. In A. Olney, I. Chounta, Z. Liu, O. Santos, & I. Bittencourt (Eds.), Artificial intelligence in education. AIED 2024. Lecture notes in computer science (pp. 180–193, Vol. 14830). Springer. https://doi.org/10.1007/978-3-031-64299-9_13
Yang, M., & Carless, D. (2013). The feedback triangle and the enhancement of dialogic feedback processes. Teaching in Higher Education, 18(3), 285–297. https://doi.org/10.1080/13562517.2012.719154
Zheng, L., Niu, J., & Zhong, L. (2022). Effects of a learning analytics-based real-time feedback approach on knowledge elaboration, knowledge convergence, interactive relationships and group performance in CSCL. British Journal of Educational Technology, 53(1), 130–149. https://doi.org/10.1111/bjet.13156
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution 4.0 International License.
