A Mixed-Method Study of How Instructors Design for Learning in Online and Distance Education





Learning design, Learning analytics, Mixed-method


The use of analytical methods from learning analytics (LA) research combined with visualizations of learning activities using learning design (LD) tools and frameworks has provided important insight into how instructors design for learning. Nonetheless, there are many subtle nuances in instructors’ design decisions that might not easily be captured using LA tools. Therefore, this study sets out to explore how and why instructors design for learning in an online and distance higher education setting by employing a mixed-method approach, which combined semi-structured interviews of 12 instructors with network analyses of their LDs. Our findings uncovered several underlying factors that influenced how instructors designed their modules and highlighted some discrepancies between instructors’ pedagogical beliefs and their actual LD as captured by the Open University Learning Design Initiative (OULDI). This study showcases the potential of combining LA with qualitative insights for a better understanding of the complex design process in online distance higher education.


AUTCLearningDesign. (2002). Predict–observe–explain: Designer’s voice — Context. Retrieved from http://www.learningdesigns.uow.edu.au/exemplars/info/LD44/more/03Context.html

Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gašević, D., Mulder, R., …, Lockyer, L. (2016). A conceptual framework linking learning design with learning analytics. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK, (pp. 329–338). New York, NY: ACM. https://dx.doi.org/10.1145/2883851.2883944

Bennett, S., Agostinho, S., & Lockyer, L. (2015). Technology tools to support learning design: Implications derived from an investigation of university teachers’ design practices. Computers & Education, 81, 211–220. https://dx.doi.org/10.1016/j.compedu.2014.10.016

Borgatti, S. P., Everett, M. G., & Freeman, L. C. (2002). Ucinet for Windows: Software for social network analysis. Harvard, MA: Analytic Technologies. Retrieved from https://sites.google.com/site/ucinetsoftware/home

Braun, V., & Clarke, V. (2012). Thematic analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. (pp. 57–71). Washington, D.C.: American Psychological Association. https://dx.doi.org/10.1037/13620-000

Chambers, E. (1992). Work-load and the quality of student learning. Studies in Higher Education, 17(2), 141–153. https://dx.doi.org/10.1080/03075079212331382627

Cherney, M. R., Fetherston, M., & Johnsen, L. J. (2017). Online course student collaboration literature: A review and critique. Small Group Research, 49(1), 98–128. https://dx.doi.org/10.1177/1046496417721627

Conole, G. (2012). Designing for learning in an open world (Vol. 4). New York, NY: Springer Science & Business Media. https://dx.doi.org/10.1007/978-1-4419-8517-0

Creswell, J. W., & Poth, C. N. (2017). Qualitative inquiry and research design: Choosing among five approaches. London, UK: Sage Publications. https://us.sagepub.com/en-us/nam/qualitative-inquiry-and-research-design/book246896

Cross, S., Galley, R., Brasher, A., & Weller, M. (2012). Final project report of the OULDI-JISC Project: Challenge and change in curriculum design process, communities, visualisation and practice. York, UK: JISC. Retrieved from http://www.jisc.ac.uk/media/documents/programmes/curriculumdesign/OULDI_Final_Report_instit%20story.pdf

Dalziel, J. (2003). Implementing learning design: The learning activity management system (LAMS). In G. Crisp, D. Thiele, I. Scholten, S. Barker, & J. Baron (Eds)., Proceedings of the 20th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE), 7–10 December 2003, Adelaide, Australia (pp. 593–596). Wollongong, N.S.W.: Australasian Society for Computers in Learning in Tertiary Education.

Dalziel, J. (2015). Learning design: Conceptualizing a framework for teaching and learning online. New York, NY: Routledge.

Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5-6), 304–317. https://dx.doi.org/10.1504/IJTEL.2012.051816

Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68–84. https://dx.doi.org/10.1016/j.iheduc.2015.10.002

Goodyear, P. (2015). Teaching as design. HERDSA Review of Higher Education, 2, 27–50.

Griffiths, D. (2017). The use of models in learning design and learning analytics. Interaction Design and Architecture(s) Journal-IxD&A, 33, 113–133. Retrieved from http://www.mifav.uniroma2.it/inevent/events/idea2010/doc/33_6.pdf

Hernández-Leo, D., Asensio-Pérez, J. I., Derntl, M., Pozzi, F., Chacón, J., Prieto, L. P., & Persico, D. (2018). An integrated environment for learning design. Frontiers in ICT, 5(9). https://dx.doi.org/10.3389/fict.2018.00009

Ifenthaler, D., Gibson, D., & Dobozy, E. (2018). Informing learning design through analytics: Applying network graph analysis. Australasian Journal of Educational Technology, 34(2). https://dx.doi.org/10.14742/ajet.3767

Koper, R., & Manderveld, J. (2004). Educational modelling language: Modelling reusable, interoperable, rich and personalised units of learning. British Journal of Educational Technology, 35(5), 537–551. https://dx.doi.org/10.1111/j.0007-1013.2004.00412.x

Kreijns, K., Kirschner, P. A., & Jochems, W. (2003). Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: A review of the research. Computers in Human Behavior, 19(3), 335–353. https://dx.doi.org/10.1016/S0747-5632(02)00057-2

Kruskal, J. B. (1964). Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 29(1), 1–27. https://dx.doi.org/10.1007/BF02289565

Laurillard, D., Kennedy, E., Charlton, P., Wild, J., & Dimakopoulos, D. (2018). Using technology to develop teachers as designers of TEL: Evaluating the learning designer. British Journal of Educational Technology, 49(6), 1044–1058. https://dx.doi.org/10.1111/bjet.12697

Law, N., Li, L., Herrera, L. F., Chan, A., & Pong, T.-C. (2017). A pattern language based learning design studio for an analytics informed inter-professional design community. Interaction Design and Architecture(s), 33, 92–112. Retrieved from http://www.mifav.uniroma2.it/inevent/events/idea2010/doc/33_5.pdf

Lockyer, L., & Dawson, S. (2011). Learning designs and learning analytics. Proceedings of the 1st International Conference on Learning Analytics and Knowledge (LAK ’11), 27 February–1 March 2011, Banff, AB, Canada (pp. 153–156). New York, NY: ACM. https://dx.doi.org/10.1145/2090116.2090140

Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588–599. https://dx.doi.org/10.1016/j.compedu.2009.09.008

Mangaroska, K., & Giannakos, M. N. (2018). Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning. IEEE Transactions on Learning Technologies, 12(4), 516–534. https://dx.doi.org/10.1109/TLT.2018.2868673

Mittelmeier, J., Rienties, B., Tempelaar, D., & Whitelock, D. (2018). Overcoming cross-cultural group work tensions: Mixed student perspectives on the role of social relationships. Higher Education, 75(1), 149–166. https://dx.doi.org/10.1007/s10734-017-0131-3

Mor, Y., Ferguson, R., & Wasson, B. (2015). Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action. British Journal of Educational Technology, 46(2), 221–229. https://dx.doi.org/10.1111/bjet.12273

Newman, M. E. (2001). Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. Physical Review E, 64(1), 016132. https://dx.doi.org/10.1103/physreve.64.016132

Nguyen, Q., Huptych, M., & Rienties, B. (2018). Linking students’ timing of engagement to learning design and academic performance. Proceedings of the 8th International Conference on Learning Analytics & Knowledge (LAK ’18), 5–9 March 2018, Sydney, Australia (pp. 141–150). New York, NY: ACM. https://dx.doi.org/10.1145/3170358.3170398

Nguyen, Q., Rienties, B., & Toetenel, L. (2017). Unravelling the dynamics of instructional practice: A longitudinal study on learning design and VLE activities. Proceedings of the Seventh International Learning Analytics & Knowledge Conference (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 168–177). New York, NY: ACM. https://dx.doi.org/10.1145/3027385.3027409

Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, R., & Whitelock, D. (2017). Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behavior, 76, 703–714. https://dx.doi.org/10.1016/j.chb.2017.03.028

Nguyen, Q., Thorne, S., & Rienties, B. (2018). How do students engage with computer-based assessments: Impact of study breaks on intertemporal engagement and pass rates. Behaviormetrika, 45(2), 597–614. https://dx.doi.org/10.1007/s41237-018-0060-1

Persico, D., & Pozzi, F. (2015). Informing learning design with learning analytics to improve teacher inquiry. British Journal of Educational Technology, 46(2), 230–248. https://dx.doi.org/10.1111/bjet.12207

Persico, D., Pozzi, F., & Goodyear, P. (2018). Teachers as designers of TEL interventions. British Journal of Educational Technology, 49(6), 975–980. https://dx.doi.org/10.1111/bjet.12706

Rienties, B., & Toetenel, L. (2016). The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules. Computers in Human Behavior, 60, 333–341. https://dx.doi.org/10.1016/j.chb.2016.02.074

Rienties, B., Toetenel, L., & Bryan, A. (2015). Scaling up learning design: Impact of learning design activities on LMS behavior and performance. Proceedings of the Fifth International Conference on Learning Analytics and Knowledge (LAK ’15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 315–319). New York, NY: ACM. https://dx.doi.org/10.1145/2723576.2723600

Thorpe, M. (2002). Rethinking learner support: The challenge of collaborative online learning. Open Learning: The Journal of Open, Distance and e-Learning, 17(2), 105–119. https://dx.doi.org/10.1080/02680510220146887a

Toetenel, L., & Rienties, B. (2016a). Analysing 157 learning designs using learning analytic approaches as a means to evaluate the impact of pedagogical decision making. British Journal of Educational Technology, 47(5), 981–992. https://dx.doi.org/10.1111/bjet.12423

Toetenel, L., & Rienties, B. (2016b). Learning design — Creative design to visualise learning activities. Open Learning: The Journal of Open, Distance and e-Learning, 31(3), 233–244. https://dx.doi.org/10.1080/02680513.2016.1213626

Ullmann, T. D. (2019). Automated analysis of reflection in writing: Validating machine learning approaches. International Journal of Artificial Intelligence in Education, 29(2), 217–257. https://dx.doi.org/10.1007/s40593-019-00174-2

Whitelock, D., Thorpe, M., & Galley, R. (2015). Student workload: A case study of its significance, evaluation and management at the Open University. Distance Education, 36(2), 161–176. https://dx.doi.org/10.1080/01587919.2015.1055059

Wise, A. F., Cui, Y., Jin, W., & Vytasek, J. (2017). Mining for gold: Identifying content-related MOOC discussion threads across domains through linguistic modeling. The Internet and Higher Education, 32, 11–28. https://dx.doi.org/10.1016/j.iheduc.2016.08.001




How to Cite

Nguyen, Q., Rienties, B., & Whitelock, D. (2020). A Mixed-Method Study of How Instructors Design for Learning in Online and Distance Education. Journal of Learning Analytics, 7(3), 64-78. https://doi.org/10.18608/jla.2020.73.6



Special Section: Learning Design and Learning Analytics