Course Load Analytics Interventions on Higher Education Course Selection
Experimental Evidence
DOI:
https://doi.org/10.18608/jla.2025.8473Keywords:
higher education, course selection, learning analytics, course load analytics, experiments, interventions, pathways, workloadAbstract
Inadequate consideration of course workload in undergraduate students' course selections has contributed to adverse academic outcomes. At the same time, credit hours, the default institutional metric to convey time-based course workload to students, has been shown to capture students' experienced workload insufficiently. Recent research documents the accuracy of analytics learned from enrolment and learning management system data that promise to convey course facets other than time to students. To investigate the feasibility of scalable interventions of such analytics on course selection, we conducted a preregistered and randomized forced-choice experiment of two courses satisfying equal requirements across 61 undergraduate students at a large public university. The control condition mimicked a university's standard course catalogue, while the intervention condition augmented this catalogue with analytics on time load, mental effort, and psychological stress alongside institutional credit hours. We find that course load analytics (CLA) significantly influenced students' course selection decisions, leading them to more deliberately optimize their expected workload relative to credit hours. Specifically, we document a strategy regarding students being more likely to take a higher credit hour course that minimizes the expected workload associated with each credit hour. Among all three dimensions of workload, students' course selection decisions were most guided by predicted mental effort, followed by psychological stress, highlighting the limitations of a time-based credit hour metric. The intervention effect of CLA was constant across first-year students and comparatively senior students. We further discuss the implications of our findings for CLA interventions at scale and the institutional stakeholders that could be involved.
References
Babad, E., Darley, J. M., & Kaplowitz, H. (1999). Developmental aspects in students’ course selection. Journal of Educational Psychology, 91(1), 157–168. https://doi.org/10.1037/0022-0663.91.1.157
Babad, E., & Tayeb, A. (2003). Experimental analysis of students’ course selection. British Journal of Educational Psychology, 73(3), 373–393. https://doi.org/10.1348/000709903322275894
Bahari, M., Arpaci, I., Azmi, N. F. M., & Shuib, L. (2023). Predicting the intention to use learning analytics for academic advising in higher education. Sustainability, 15(21), 15190. https://doi.org/10.3390/su152115190
Bolker, B. M. (2015). Linear and generalized linear mixed models. In G. A. Fox, S. Negrete-Yankelevich, & V. J. Sosa (Eds.), Ecological statistics: Contemporary theory and application (pp. 309–333). Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199672547.003.0014
Borchers, C., & Pardos, Z. A. (2023). Insights into undergraduate pathways using course load analytics. In Proceedings of the 13th International Conference on Learning Analytics and Knowledge (LAK 2023), 13–17 March 2023, Arlington, Texas, USA (pp. 219–229). ACM. https://doi.org/10.1145/3576050.3576081
Borchers, C., Xu, Y., & Pardos, Z. A. (2025). Workload overload? Late enrollment leads to course dropout. Journal of Educational Data Mining, 17(1), 126–156. https://doi.org/10.5281/zenodo.14907388
Bound, J., Lovenheim, M., & Turner, S. (2007). Understanding the increased time to the baccalaureate degree [working paper]. Stanford Institute for Economic Policy Research. https://siepr.stanford.edu/publications/working-paper/understanding-increased-time-baccalaureate-degree
Brown, A. (2016). Item response models for forced-choice questionnaires: A common framework. Psychometrika, 81, 135–160. https://doi.org/10.1007/s11336-014-9434-9
Chaturapruek, S., Dalberg, T., Thompson, M. E., Giebel, S., Harrison, M. H., Johari, R., Stevens, M. L., & Kizilcec, R. F. (2021). Studying undergraduate course consideration at scale. AERA Open, 7, 2332858421991148. https://doi.org/10.1177/2332858421991148
Chaturapruek, S., Dee, T. S., Johari, R., Kizilcec, R. F., & Stevens, M. L. (2018). How a data-driven course planning tool affects college students’ GPA: Evidence from two field experiments. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale (L@S 2018), 26–28 June 2018, London, UK (63:1–63:10). ACM. https://doi.org/10.1145/3231644.3231668
Chen, Y., Fu, A., Lee, J. J.-L., Tomasik, I. W., & Kizilcec, R. F. (2022). Pathways: Exploring academic interests with historical course enrollment records. In Proceedings of the Ninth ACM Conference on Learning at Scale (L@S 2022), 1–3 June 2022, New York, New York, USA (pp. 222–233). ACM. https://doi.org/10.1145/3491140.3528270
Cheung, K., Yip, T. L., Wan, C. J., Tsang, H., Zhang, L. W., & Parpala, A. (2020). Differences in study workload stress and its associated factors between transfer students and freshmen entrants in an Asian higher education context. PLOS One, 15(5), e0233022. https://doi.org/10.1371/journal.pone.0233022
Chockkalingam, S., Yu, R., & Pardos, Z. A. (2021). Which one’s more work? Predicting effective credit hours between courses. In Proceedings of the 11th International Conference on Learning Analytics and Knowledge (LAK 2021), 12–16 April 2021, Irvine, California, USA (pp. 599–605). ACM. https://doi.org/10.1145/3448139.3448204
Cooke, M. L. (1910). Academic and industrial efficiency: A report to the Carnegie Foundation for the Advancement of Teaching. The Merrymount Press.
Cummings, M., & Knott, T. (2001). How many credit hours should first semester engineering freshmen take? In 2001 Annual Conference of the American Society for Engineering Education, 24–27 June 2001, Albuquerque, New Mexico, USA (pp. 6.544.1–6.544.7). ASEE Peer. https://doi.org/10.18260/1-2--9334
De Laet, T., Millecamp, M., Broos, T., De Croon, R., Verbert, K., & Duorado, R. (2020). Explainable learning analytics: Challenges and opportunities. In Companion Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK 2020), 23–27 March 2020, Frankfurt, Germany (pp. 500–510). SoLAR. https://www.solaresearch.org/wp-content/uploads/2020/06/LAK20_Companion_Proceedings.pdf
Dika, S. L., & D’Amico, M. M. (2016). Early experiences and integration in the persistence of first-generation college students in STEM and non-STEM majors. Journal of Research in Science Teaching, 53(3), 368–383. https://doi.org/10.1002/tea.21301
Evrard, A. E., Matz, R., Murray, E., Hayward, B., Mills, M., & Hayward, C. (2023). Innovating at campus scale: The case of Michigan’s atlas. In Proceedings of the 10th ACM Conference on Learning at Scale (L@S 2023), 20–22 July 2023, Copenhagen, Denmark (pp. 306–310). ACM. https://doi.org/10.1145/3573051.3596177
Fischer, C., Pardos, Z. A., Baker, R. S., Williams, J. J., Smyth, P., Yu, R., Slater, S., Baker, R., & Warschauer, M. (2020). Mining big data in education: Affordances and challenges. Review of Research in Education, 44(1), 130–160. https://doi.org/10.3102/0091732X20903304
Hagedorn, L. S., Maxwell, W. E., Cypers, S., Moon, H. S., & Lester, J. (2007). Course shopping in urban community colleges: An analysis of student drop and add activities. The Journal of Higher Education, 78(4), 464–485. https://doi.org/10.1080/00221546.2007.11772324
Heffernan, J. M. (1973). The credibility of the credit hour: The history, use, and shortcomings of the credit system. The Journal of Higher Education, 44(1), 61–72. https://doi.org/10.1080/00221546.1973.11776844
Huntington-Klein, N., & Gill, A. (2021). Semester course load and student performance. Research in Higher Education, 62(5), 623–650. https://doi.org/10.1007/s11162-020-09614-8
Ilies, R., Dimotakis, N., & De Pater, I. E. (2010). Psychological and physiological reactions to high workloads: Implications for well-being. Personnel Psychology, 63(2), 407–436. https://doi.org/10.1111/j.1744-6570.2010.01175.x
Jiang, W., Pardos, Z. A., & Wei, Q. (2019). Goal-based course recommendation. In Proceedings of the Ninth International Conference on Learning Analytics and Knowledge (LAK 2019), 4–8 March 2019, Tempe, Arizona, USA (pp. 36–45). ACM. https://doi.org/10.1145/3303772.3303814
Karimi-Haghighi, M., Castillo, C., & Hernandez-Leo, D. (2022). A causal inference study on the effects of first year workload on the dropout rate of undergraduates. In M. Rodrigo, N. Matsuda, A. Cristea, & V. Dimitrova (Eds.), Artificial intelligence in education. AIED 2022. Lecture notes in computer science (pp. 15–27, Vol. 13355). Springer. https://doi.org/10.1007/978-3-031-11644-5_2
Laitinen, A. (2012). Cracking the credit hour [policy paper]. New America Foundation. https://www.newamerica.org/education-policy/policy-papers/cracking-the-credit-hour/
Lang, D., Wang, A., Dalal, N., Paepcke, A., & Stevens, M. L. (2022). Forecasting undergraduate majors: A natural language approach. AERA Open, 8, 23328584221126516. https://doi.org/10.1177/23328584221126516
Larrabee Sønderlund, A., Hughes, E., & Smith, J. (2019). The efficacy of learning analytics interventions in higher education: A systematic review. British Journal of Educational Technology, 50(5), 2594–2618. https://doi.org/10.1111/bjet.12720
Liu, R., & Koedinger, K. R. (2017). Closing the loop: Automated data-driven cognitive model discoveries lead to improved instruction and learning gains. Journal of Educational Data Mining, 9(1), 25–41. https://doi.org/10.5281/zenodo.3554625
Othman, M. H., Mohamad, N., & Barom, M. N. (2019). Students’ decision making in class selection and enrolment. International Journal of Educational Management, 33(4), 587–603. https://doi.org/10.1108/IJEM-06-2017-0143
Paas, F., Van Gog, T., & Sweller, J. (2010). Cognitive load theory: New conceptualizations, specifications, and integrated research perspectives. Educational Psychology Review, 22(2), 115–121. https://doi.org/10.1007/s10648-010-9133-8
Pardos, Z. A., Borchers, C., & Yu, R. (2023). Credit hours is not enough: Explaining undergraduate perceptions of course workload using LMS records. The Internet and Higher Education, 56, 100882. https://doi.org/10.1016/j.iheduc.2022.100882
Pardos, Z. A., Fan, Z., & Jiang, W. (2019). Connectionist recommendation in the wild: On the utility and scrutability of neural networks for personalized course guidance. User Modeling and User-Adapted Interaction, 29, 487–525. https://doi.org/10.1007/s11257-019-09218-7
Parker, J. D., Hogan, M. J., Eastabrook, J. M., Oke, A., & Wood, L. M. (2006). Emotional intelligence and student retention: Predicting the successful transition from high school to university. Personality and Individual Differences, 41(7), 1329–1336. https://doi.org/10.1016/j.paid.2006.04.022
Paulsen, M. B., & Wells, C. T. (1998). Domain differences in the epistemological beliefs of college students. Research in Higher Education, 39, 365–384. https://doi.org/10.1023/A:1018785219220
Reid, G. B., & Nygren, T. E. (1988). The subjective workload assessment technique: A scaling procedure for measuring mental workload. Advances in Psychology, 52, 185–218. https://doi.org/10.1016/S0166-4115(08)62387-0
Ruiz-Gallardo, J.-R., Castano, S., Gomez-Alday, J. J., & Vald ´es, A. (2011). Assessing student workload in problem based learning: Relationships among teaching method, student workload and achievement. A case study in natural sciences. Teaching and Teacher Education, 27(3), 619–627. https://doi.org/10.1016/j.tate.2010.11.001
Shedd, J. M. (2003). The history of the student credit hour. New Directions for Higher Education, 2003(122), 5–12. https://doi.org/10.1002/he.106
Silva, E., White, T., & Toch, T. (2015). The Carnegie unit: A century-old standard in a changing education landscape. Carnegie Foundation for the Advancement of Teaching. https://www.carnegiefoundation.org/resources/publications/carnegie-unit/
Smith, A. P. (2019). Student workload, wellbeing and academic attainment. In L. Longo & M. Leva (Eds.), Human mental workload: Models and applications. H-WORKLOAD 2019. Communications in computer and information science (pp. 35–47, Vol. 1107). Springer. https://doi.org/10.1007/978-3-030-32423-0_3
Sogunro, O. A. (2015). Motivating factors for adult learners in higher education. International Journal of Higher Education, 4(1), 1–22. https://doi.org/10.5430/ijhe.v4n1p22
Srivastava, N., Nawaz, S., Tsai, Y. - S., & Gasevic, D. (2024). Curriculum analytics of course choices: Links with academic performance. Journal of Learning Analytics, 11(1), 116–131. https://doi.org/10.18608/jla.2024.8095
Towers, A., & Towers, N. (2020). Re-evaluating the postgraduate students’ course selection decision making process in the digital era. Studies in Higher Education, 45(6), 1133–1148. https://doi.org/10.1080/03075079.2018.1545757
van Haastrecht, M., Brinkhuis, M., Peichl, J., Remmele, B., & Spruit, M. (2023). Embracing trustworthiness and authenticity in the validation of learning analytics systems. In Proceedings of the 13th International Conference on Learning Analytics and Knowledge (LAK 2023), 13–17 March 2023, Arlington, Texas, USA (pp. 552–558). ACM. https://doi.org/10.1145/3576050.3576060
Vemula, S. R., & Moraes, M. (2024). Learning analytics dashboards for advisors—A systematic literature review. International Journal on Cybernetics & Informatics (IJCI), 13(1). https://doi.org/10.5121/ijci.2024.130101
Waschle, K., Allgaier, A., Lachner, A., Fink, S., & Nuckles, M. (2014). Procrastination and self-efficacy: Tracing vicious and virtuous circles in self-regulated learning. Learning and Instruction, 29, 103–114. https://doi.org/10.1016/j.learninstruc.2013.09.005
Wheeler, E. L. (2019). Extending “guided pathways” beyond the community college: Lessons for university transfer orientation. Community College Journal of Research and Practice, 43(4), 275–279. https://doi.org/10.1080/10668926.2018.1460283
Wise, A. F., Brackett, A., & Maddox, B. (2023). Flexible coupling of learning analytics research and practice in the university: A collective strengths approach. In Companion Proceedings of the 13th International Conference on Learning Analytics and Knowledge (LAK 2023), 13–17 March 2023, Arlington, Texas, USA (p. 25). SoLAR. https://www.solaresearch.org/wp-content/uploads/2023/03/LAK23_CompanionProceedings.pdf
Yang, C., Chen, A., & Chen, Y. (2021). College students’ stress and health in the COVID-19 pandemic: The role of academic workload, separation from school, and fears of contagion. PLOS one, 16(2), e0246676. https://doi.org/10.1371/journal.pone.0246676
Yu, R., Lee, H., & Kizilcec, R. F. (2021). Should college dropout prediction models include protected attributes? In Proceedings of the Eighth ACM Conference on Learning at Scale (L@S 2021), 22–25 June 2021, online (pp. 91–100). ACM. https://doi.org/10.1145/3430895.3460139
Zhang, Y. L. (2022). Early academic momentum: Factors contributing to community college transfer students’ STEM degree attainment. Journal of College Student Retention: Research, Theory & Practice, 23(4), 873–902. https://doi.org/10.1177/1521025119881130
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution 4.0 International License.
