Transparency and Trustworthiness in User Intentions to Follow Career Recommendations from a Learning Analytics Tool




artificial intelligence, AI, learning analytics, career guidance, transparency, trustworthiness, research paper


Transparency and trustworthiness are among the key requirements for the ethical use of learning analytics (LA) and artificial intelligence (AI) in the context of social inclusion and equity. However, research on these issues pertaining to users is lacking, leaving it unclear as to how transparent and trustworthy current LA tools are for their users and how perceptions of these variables relate to user behaviour. In this study, we investigate user experiences of an LA tool in the context of career guidance, which plays a crucial role in supporting nonlinear career pathways for individuals. We review the ethical challenges of big data, AI, and LA in connection to career guidance and analyze the user experiences (N = 106) of the LA career guidance tool, which recommends study programs and institutions to users. Results indicate that the LA career guidance tool was evaluated as trustworthy but not transparent. Accuracy was found to be a stronger predictor for the intention to follow on the recommendations of the LA guidance tool than was understanding the origins of the recommendation. The user’s age emerged as an important factor in their assessment of transparency. We discuss the implications of these findings and suggest emphasizing accuracy in the development of LA tools for career guidance.


Albert, E. T. (2019). AI in talent acquisition: A review of AI-applications used in recruitment and selection. Strategic HR Review, 18(5), 215–221.

Alkan, O., Daly, E. M., Botea, A., Valente, A. N., & Pedemonte, P. (2019). Where can my career take me? Harnessing dialogue for interactive career goal recommendations. Proceedings of the 24th International Conference on Intelligent User Interfaces (IUI ’19), 17–20 March 2019, Marina del Ray, CA, USA (pp. 603–613). ACM Press.

Ananny, M., & Crawford, K. (2018). Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society, 20(3), 973–989.

Araujo, T., Helberg, N., Kruikemeier, S., & de Vreese, C. H. (2020). In AI we trust? Perceptions about automated decision-making by artificial intelligence. AI & Society, 35, 611–623.

Aroyo, A. M., de Bruyne, J., Dheu, O., Fosch-Villaronga, E., Gudkov, A., Hoch, H., Jones, S., Lutz, C., Sætra, H., Solberg, M., & Tamò-Larrieux, A. (2021). Overtrusting robots: Setting a research agenda to mitigate overtrust in automation. Paladyn: Journal of Behavioral Robotics, 12(1), 423–436.

Ashfaq, M., Yun, J., Yu, S., & Loureiro, S. M. C. (2020). I, Chatbot: Modeling the determinants of users’ satisfaction and continuance intention of AI-powered service agents. Telematics and Informatics, 54, 101473.

Baker, T., & Smith, L. (2019). Educ-AI-tion rebooted? Exploring the future of artificial intelligence in schools and colleges. Nesta.

Baneres, D., & Conesa, J. (2017). A life-long learning recommender system to promote employability. International Journal of Emerging Technologies in Learning (iJET) 12(6), 77–93.

Bengtsson, M. (2016). How to plan and perform a qualitative study using content analysis. NursingPlus Open, 2, 8–14.

Betz, N. E., & Borgen, F. H. (2009). Comparative effectiveness of CAPA and FOCUS online: Career assessment systems with undecided college students. Journal of Career Assessment, 17(4), 351–366.

Bøegh, J. (2014). Trust and trustworthiness in human behavior and IT services: Concepts, definitions and relations. In Y. Yuan, X. Wu, & Y. Lu (Eds.), Proceedings of the International Conference on Trustworthy Computing and Services (ISCTCS 2013). Communications in Computer and Information Science, vol. 426, Beijing, China (pp. 235–251). Springer.

Cambridge Dictionary. (n.d.). Transparency.

Chen, S., Chen, H., Ling, H., & Gu, X. (2022). An online career intervention for promoting Chinese high school students’ career readiness. Frontiers in Psychology, 12, 815076.

Copeland, L. Y., Dik, B. J., McLaren, M. R., Onder, C., Wolfson, N. E., & Kraiger, K. (2011). Recommendations for using computer-assisted career guidance systems (CACGS) in career counseling practice. Journal of Psychological Issues in Organizational Culture, 2(3), 86–94.

Council of the European Union. (2008). Council resolution on better integrating lifelong guidance into lifelong learning strategies.

Cramer, H., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., & Wielinga, B. (2008). The effects of transparency on trust in and acceptance of a content-based art recommender. User Modeling and User-Adapted Interaction, 18, 455–496.

De Laet, T., Millecamp, M., Ortiz-Rojas, M., Jimenez, A., Maya, R., & Verbert, K. (2020). Adoption and impact of a learning analytics dashboard supporting the advisor–student dialogue in a higher education institute in Latin America. British Journal of Educational Technology, 51(4), 1002–1018.

Dindar, M., Järvelä, S., & Haataja, E. (2020). What does physiological synchrony reveal about metacognitive experiences and group performance? British Journal of Educational Technology, 51(5), 1577–1596.

Drachsler, H., & Greller, W. (2016). Privacy and analytics: It’s a DELICATE issue a checklist for trusted learning analytics. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 89–98). ACM Press.

Drachsler, H., Verbert, K., Santos, O. C., & Manouselis, N. (2015). Panorama of recommender systems to support learning. In F. Ricci, L. Rokach, & B. Shapira (Eds.), Recommender systems handbook (pp. 421–451). Springer.

Eubanks, B. (2022). Artificial intelligence for HR: Use AI to support and develop a successful workforce (2nd ed.). Kogan Page.

European Commission. (2021, April 21). Proposal for a regulation of the European Parliament and of the council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain union legislative acts.

Ez-Zaouia, M., & Lavoué, E. (2017). EMODA: A tutor oriented multimodal and contextual emotional dashboard. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 429–438). ACM Press.

Felzmann, H., Villaronga, E. F., Lutz, C., & Tamò-Larrieux, A. (2019). Transparency you can trust: Transparency requirements for artificial intelligence between legal norms and contextual concerns. Big Data & Society, 6(1).

Fumagalli, E., Rezaei, S., & Salomons, A. (2022). OK computer: Worker perceptions of algorithmic recruitment. Research Policy, 51(2), 104420.

Gati, I., & Asher, I. (2001). The PIC model for career decision making: Prescreening, in-depth exploration, and choice. In F. T. L. Leong & A. Barak (Eds.), Contemporary models in vocational psychology: A volume in honor of Samuel H. Osipow (pp. 7–54). Lawrence Erlbaum Associates.

Gati, I., & Kulcsár, V. (2021). Making better career decisions: From challenges to opportunities. Journal of Vocational Behavior, 126, 103545.

Gedrimiene, E., Celik, I., Kaasila, A., Mäkitalo, K., Muukkonen, H. (2023). Artificial intelligence (AI)-enhanced learning analytics (LA) for supporting career decisions: Advantages and challenges from user perspective. [Manuscript submitted for publication]. Faculty of Education and Psychology, University of Oulu.

General Data Protection Regulation (2016, May 4). Legislation. Official Journal of the European Union, 59, L 119.

Guerra, J., Ortiz-Rojas, M., Zúñiga‐Prieto, M. A., Scheihing, E., Jiménez, A., Broos, T., De Laet, T., & Verbert, K. (2020). Adaptation and evaluation of a learning analytics dashboard to improve academic support at three Latin American universities. British Journal of Educational Technology, 51(4), 973–1001.

Gutiérrez, F., Charleer, S., De Croon, R., Htun, N. N., Goetschalckx, G., & Verbert, K. (2019). Explaining and exploring job recommendations: A user-driven approach for interacting with knowledge-based job recommender systems. Proceedings of the 13th ACM Conference on Recommender Systems (RecSys ’19), 16–20 September 2019, New York, NY, USA (pp. 60–68). ACM Press.

Gutiérrez, F., Seipp, K., Ochoa, X., Chiluiza, K., De Laet, T., & Verbert, K. (2020). LADA: A learning analytics dashboard for academic advising. Computers in Human Behavior, 107, 105826.

Hakami, E., & Hernández-Leo, D. (2020). How are learning analytics considering the societal values of fairness, accountability, transparency and human well-being? Proceedings of the Learning Analytics Summer Institute (LASI 2020), 15–16 June 2020, Valladolid, Spain (pp. 121–141). CEUR-WS.

Harman, J. L., O’Donovan, J., Abdelzaher, T., & Gonzalez, C. (2014). Dynamics of human trust in recommender systems. Proceedings of the 8th ACM Conference on Recommender Systems (RecSys ’14), 6–10 October 2014, Foster City, CA, USA (pp. 305–308). ACM Press.

Harris-Bowlsbey, J. (2013). Computer-assisted career guidance systems: A part of NCDA history. The Career Development Quarterly, 61(2), 181–185.

Harris-Bowlsbey, J., & Samson, J. P. (2011). Computer-based career planning systems: Dreams and reality. The Career Development Quarterly, 49(3), 250–260.

Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000). Explaining collaborative filtering recommendations. Proceedings of the 2000 Conference on Computer Supported Cooperative Work (CSCW ’00), 2–6 December 2000, Philadelphia, PA, USA (pp. 241–250). ACM Press.

High-Level Expert Group on AI. (2019). Ethics guidelines for trustworthy AI. European Commission.

Hong, I. B., & Cho, H. (2011). The impact of consumer trust on attitudinal loyalty and purchase intentions in B2C e-marketplaces: Intermediary trust vs. seller trust. International Journal of Information Management, 31(5), 469–479.

Hu, L-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.

Kamal, N., Sarker, F., & Mamun, K. A. (2020). A comparative study of machine learning approaches for recommending university faculty. Proceedings of the 2nd International Conference on Sustainable Technologies for Industry 4.0 (STI 2020), 19–20 December 2020, Dhaka, Bangladesh (pp. 1–6). IEEE.

Karaoglan Yilmaz, F. G., & Yilmaz, R. (2021). Learning analytics as a metacognitive tool to influence learner transactional distance and motivation in online learning environments. Innovations in Education and Teaching International, 58(5), 575–585.

Kettunen, J. (2017). Career practitioners’ conceptions of social media and competency for social media in career services. Doctoral dissertation, University of Jyväskylä.

Kew, S. N., & Tasir, Z. (2022). Developing a learning analytics intervention in e-learning to enhance students’ learning performance: A case study. Education and Information Technologies, 27, 7099–7134.

Khare, R., Stewart, B., & Khare, A. (2018). Artificial intelligence and the student experience: An institutional perspective. IAFOR Journal of Education, 6(3).

Lee, M. K. (2018). Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management. Big Data & Society, 5(1).

Mahmud, H., Islam, A. K. M. N., Ahmed, S. I., & Smolander, K. (2022). What influences algorithmic decision-making? A systematic literature review on algorithm aversion. Technological Forecasting and Social Change, 175, 121390.

Maples, M. R., & Luzzo, D. A. (2005). Evaluating DISCOVER’s effectiveness in enhancing college students’ social cognitive career development. The Career Development Quarterly, 53(3), 274–285.

Mau, W.-C. (1999). Effects of computer-assisted career decision making on vocational identity and career exploratory behaviors. Journal of Career Development, 25(4), 261–274.

Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An integrative model of organizational trust. The Academy of Management Review, 20(3), 709–734.

Merriam-Webster. (n.d.). Transparency.

Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space.

Namoun, A., & Alshanqiti, A. (2020). Predicting student performance using data mining and learning analytics techniques: A systematic literature review. Applied Sciences, 11(1), 237–265.

Ochirbat, A., Shih, T. K., Chootong, C., Sommool, W., Gunarathne, W. K. T. M., Hai-Hui, W., & Zhao-Heng, M. (2018). Hybrid occupation recommendation for adolescents on interest, profile, and behavior. Telematics and Informatics, 35(3), 534–550.

Offer, M., & Sampson, J. P. (1999). Quality in the content and use of information and communications technology in guidance. British Journal of Guidance & Counselling, 27(4), 501–516.

Osborn, D. S., Brown, C. A., & Morgan, M. J. (2021). Expectations, experiences, and career-related outcomes of computer-assisted career guidance systems. Journal of Employment Counseling, 58(2), 74–90.

Osborn, D. S., Kronholz, J. F., Finklea, J. T., & Cantonis, A. M. (2014). Technology-savvy career counselling. Canadian Psychology, 55(4), 258–265.

Pardo, A., & Siemens, G. (2014). Ethical and privacy principles for learning analytics. British Journal of Educational Technology, 45(3), 438–450.

Ribes, D., Henchoz, N., Portier, H., Defayes, L., Phan, T.-T., Gatica-Perez, D., & Sonderegger, A. (2021). Trust indicators and explainable AI: A study on user perceptions. Proceedings of the 18th IFIP International Conference on Human–Computer Interaction (INTERACT 2021), August 30–September 3, 2021, Bari, Italy (pp. 662–671). Springer.

Rivera, A. C., Tapia-Leon, M., & Lujan-Mora, S. (2018). Recommendation systems in education: A systematic mapping study. Proceedings of the International Conference on Information Technology & Systems (ICITS 2018), 10–12 January 2018, Libertad, Ecuador (pp. 937–947). Springer.

Sampson, J. P., & Makela, J. P. (2014). Ethical issues associated with information and communication technology in counseling and guidance. International Journal for Educational and Vocational Guidance, 14(1), 135–148.

Sampson, J. P., Kettunen, J., & Vuorinen, R. (2020). The role of practitioners in helping persons make effective use of information and communication technology in career interventions. International Journal for Educational and Vocational Guidance, 20, 191–208.

Sampson, J. P., Osborn, D. S., Kettunen, J., Hou, P.-C., Miller, A. K., & Makela, J. P. (2018). The validity of social media-based career information. The Career Development Quarterly, 66(2), 121–134.

Schnackenberg, A. K., & Tomlinson, E. C. (2016). Organizational transparency: A new perspective on managing trust in organization–stakeholder relationships. Journal of Management 42(7), 1784–1810.

Schnackenberg, A. K., Tomlinson, E., & Coen, C. (2021). The dimensional structure of transparency: A construct validation of transparency as disclosure, clarity, and accuracy in organizations. Human Relations, 74(10), 1628–1660.

Sclater, N. (2016). Developing a code of practice for learning analytics. Journal of Learning Analytics, 3(1), 16–42.

Seeger, B. A. (1988). The effect of using a computer assisted career guidance system on career development attitudes, knowledge, and behaviors in students. Doctoral dissertation, Iowa State University.

Shin, D. (2020). User perceptions of algorithmic decisions in the personalized AI system: Perceptual evaluation of fairness, accountability, transparency, and explainability. Journal of Broadcasting & Electronic Media, 64(4), 541–565.

Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400.

Silvola, A., Näykki, P., Kaveri, A., & Muukkonen, H. (2021). Expectations for supporting student engagement with learning analytics: An academic path perspective. Computers & Education, 168, 104192.

Sinha, R., & Swearingen, K. (2002). The role of transparency in recommender systems. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’02), 20–25 April 2002, Minneapolis, MN, USA (pp. 830–831). ACM Press.

Staunton, T. (2020). Icarus, grannies, black holes and the death of privacy: Exploring the use of digital networks for career enactment. British Journal of Guidance & Counselling, 48(5), 611–622.

Stephen, A. (2010). The effect of the Kuder career planning system used in a classroom setting on perceived career barriers, coping self-efficacy, career decidedness, and retention. Doctoral dissertation, Iowa State University.

Tirpak, D. M., & Schlosser, L. Z. (2013). Evaluating FOCUS‐2’s effectiveness in enhancing first‐year college students’ social cognitive career development. The Career Development Quarterly, 61(2), 110–123.

Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98–110.

Westman, S., Kauttonen, J., Klemetti, A., Korhonen, N., Manninen, M., Mononen, A., Niittymäki, S., & Paananen, H. (2021). Artificial intelligence for career guidance: Current requirements and prospects for the future. IAFOR Journal of Education, 9(4), 43–62.

Winfield, A. F. T., & Jirotka, M. (2017). The case for an ethical black box. In Y. Gao, S. Fallah, Y. Jin, C. Lekakou (Eds.), Towards autonomous robotic systems (TAROS 2017). Lecture Notes in Computer Science, vol. 10454. (pp. 262–273). Springer.

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education: Where are the educators? International Journal of Educational Technology in Higher Education, 16.

Zheng, Y., & Toribio, J. R. (2021). The role of transparency in multi-stakeholder educational recommendations. User Modeling and User-Adapted Interaction, 31, 513–540.




How to Cite

Gedrimiene, E., Celik, I., Mäkitalo, K., & Muukkonen, H. (2023). Transparency and Trustworthiness in User Intentions to Follow Career Recommendations from a Learning Analytics Tool. Journal of Learning Analytics, 10(1), 54-70.



Special Section on Fairness, Equity, and Responsibility in Learning Analytics