A Teacher-facing Dashboard for Online Video Lectures




learning analytics, teacher dashboard, student feedback, video leacture


The use of online video lectures in universities, primarily for content delivery and learning, is on the rise. Instructors’ ability to recognize and understand student learning experiences with online video lectures, identify particularly difficult or disengaging content and thereby assess overall lecture quality can inform their instructional practice related to such lectures. This paper introduces Tcherly, a teacher-facing dashboard that presents class-level aggregated time series data on students’ self-reported cognitive-affective states they experienced during a lecture. Instructors can use the dashboard to evaluate and improve their instructional practice related to video lectures. We report the detailed iterative prototyping design process of the Tcherly Dashboard involving two stakeholders (instructors and designers) that informed various design decisions of the dashboard, and also provide usability and usefulness data. We demonstrate, with real-life examples of Tcherly Dashboard use generated by the researchers based on data collected from six courses and 11 lectures, how the dashboard can assist instructors in understanding their students’ learning experiences and evaluating the associated instructional materials.


Aldridge, J., Bell, L., & Fraser, B. (2010). Using students’ perceptions of the learning environment to guide teacher action research to improve senior secondary classrooms. Paper presented at the AARE Annual Conference, Canberra 2009. Australian Association for Research in Education.

Aldridge, J. M., & Fraser, B. J. (2008). Outcomes-focused learning environments: Determinants and effects. Brill Sense.

Atapattu, T., & Falkner, K. (2018). Impact of lecturer’s discourse for students’ video engagement: Video learning analytics case study of MOOCs. Journal of Learning Analytics, 5(3), 182–197.

Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. International Journal of Human–Computer Interaction, 24(6), 574–594.

Bargeron, D., Grudin, J., Gupta, A., Sanocki, E., Li, F., & Leetiernan, S. (2002). Asynchronous collaboration around multimedia applied to on-demand education. Journal of Management Information Systems, 18(4), 117–145.

Bell, L. M., & Aldridge, J. M. (2014). Investigating the use of student perception data for teacher reflection and classroom improvement. Learning Environments Research, 17(3), 371–388.

Boyd, D., & Crawford, K. (2012). Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon. Information, Communication & Society, 15(5), 662–679.

Brooke, J. (1996). SUS: A ʻquick and dirty’ usability scale. Usability Evaluation in Industry, 189(194), 4–7.

Brooks, C., Amundson, K., & Greer, J. (2009). Detecting significant events in lecture video using supervised machine learning. In V. Dimitrova, R. Mizoguchi, B. du Boulay, & A. Graesser (Eds.), Frontiers in Artificial Intelligence and Applications (pp. 483–490). Artificial Intelligence in Education, vol. 200. IOS Press.

Brooks, C., Demmans Epp, C., Logan, G., & Greer, J. (2011). The who, what, when, and why of lecture capture. In P. Long, G. Siemens, G. Conole, & D. Gašević (Eds.), Proceedings of the 1st International Conference on Learning Analytics and Knowledge: Connecting the technical, pedagogical, and social dimensions of learning analytics (LAK ʼ11), 27 February–1 March 2011, Banff, AB, Canada (pp. 86–92). ACM Press.

Brooks, C., Greer, J., & Gutwin, C. (2014). The data-assisted approach to building intelligent technology-enhanced learning environments. In J. A. Larusson & B. White (Eds.), Learning analytics: From research to practice (pp. 123–156). Springer.

Brooks, C., Thompson, C., & Greer, J. (2013). Visualizing lecture capture usage: A learning analytics case study. WAVe 2013 Workshop at LAK ’13, 8 April 2013, Leuven, Belgium, (pp. 9–14).

Bustingorry, S. O. (2008). Towards teachers’ professional autonomy through action research. Educational Action Research, 16(3), 407–420.

Chavan, P., Gupta, S., & Mitra, R. (2018). A novel feedback system for pedagogy refinement in large lecture classrooms. In J. C. Yang, M. Chang, L.-H. Wong, & M. T. Rodrigo (Eds.), Proceedings of the 26th International Conference on Computers in Education (ICCE 2018), 26–30 November 2018, Manila, Philippines (pp. 464–469). Asia-Pacific Society for Computers in Education.

Chavan, P., & Mitra, R. (2019). Developing a student feedback system using a design-based research approach. Proceedings of the 10th International Conference on Technology for Education (T4E), 9–11 December 2019, Goa, India (pp. 1–8). IEEE Computer Society.

Chen, Y., & Hoshower, L. B. (2003). Student evaluation of teaching effectiveness: An assessment of student perception and motivation. Assessment & Evaluation in Higher Education, 28(1), 71–88.

Corrin, L., & De Barba, P. (2015). How do students interpret feedback delivered via dashboards? Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 430–431). ACM Press.

Cross, A., Bayyapunedi, M., Cutrell, E., Agarwal, A., & Thies, W. (2013). TypeRighting: Combining the benefits of handwriting and typeface in online educational videos. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ʼ13), 27 April–2 May 2013, Paris, France (pp. 793–796). ACM Press.

Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. HarperPerennial.

de Barba, P. G., Kennedy, G. E., & Ainley, M. D. (2016). The role of students’ motivation and participation in predicting performance in a MOOC. Journal of Computer Assisted Learning, 32(3), 218–231.

Demmans Epp, C., Perez, R., Phirangee, K., Hewitt, J., & Toope, K. (2019). User-centered dashboard design: Iterative design to support teacher informational needs in online learning contexts. Paper presented at the American Educational Research Association (AERA) Annual Meeting, Toronto, Canada, April, 5–9.

Ellis, R. A., Han, F., & Pardo, A. (2017). Improving learning analytics: Combining observational and self-report data on student learning. Journal of Educational Technology & Society, 20(3), 158–169.

Fiorini, S., Sewell, A., Bumbalough, M., Chauhan, P., Shepard, L., Rehrey, G., & Groth, D. (2018). An application of participatory action research in advising-focused learning analytics. Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, NSW, Australia (pp. 89–96). ACM Press.

Fraser, B. J. (2012). Classroom learning environments: Retrospect, context and prospect. In B. J. Fraser, K. Tobin, & C. J. McRobbie (Eds.), Second international handbook of science education (pp. 1191–1239). Springer.

Gajos, K. Z., Kim, J., Li, S.-W., Cai, C. J., & Miller, R. C. (2014). Leveraging video interaction data and content analysis to improve video learning. In Proceedings of the CHI 2014 Learning Innovation at Scale workshop, 27 April 2014, Toronto, Canada.

Gašević, D., Mirriahi, N., Dawson, S., & Joksimović, S. (2017). Effects of instructional conditions and experience on the adoption of a learning tool. Computers in Human Behavior, 67, 207–220.

Graesser, A. C., & D’Mello, S. (2012). Emotions during the learning of difficult material. In B. H. Ross (Ed.), The Psychology of Learning and Motivation (pp. 183–225).

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.

Hoban, G., & Hastings, G. (2006). Developing different forms of student feedback to promote teacher reflection: A 10-year collaboration. Teaching and Teacher Education, 22(8), 1006–1019.

Holstein, K., Hong, G., Tegene, M., McLaren, B. M., & Aleven, V. (2018). The classroom as a dashboard: Co-designing wearable cognitive augmentation for K–12 teachers. Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, NSW, Australia (pp. 79–88). ACM Press.

Jacobs, L. C. (1987). University faculty and students’ opinions of student ratings. Indiana Studies in Higher Education No. 55.

Kay, R. H. (2012). Exploring the use of video podcasts in education: A comprehensive review of the literature. Computers in Human Behavior, 28(3), 820–831.

Kim, J., Guo, P. J., Seaton, D. T., Mitros, P., Gajos, K. Z., & Miller, R. C. (2014). Understanding in-video dropouts and interaction peaks in online lecture videos. Proceedings of the 1st ACM Conference on Learning @ Scale (L@S 2014), 4–5 March 2014, Atlanta, GA, USA (pp. 31–40). ACM Press.

Kizilcec, R. F., Bailenson, J. N., & Gomez, C. J. (2015). The instructor’s face in video instruction: Evidence from two large-scale field studies. Journal of Educational Psychology, 107(3), 724.

Kizilcec, R. F., Papadopoulos, K., & Sritanyaratana, L. (2014). Showing face in video instruction: Effects on information retention, visual attention, and affect. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ʼ14), 26 April–1 May 2014, Toronto, ON, Canada (pp. 2095–2102). ACM Press.

Larson, R., & Csikszentmihalyi, M. (2014). The experience sampling method. In Flow and the foundations of positive psychology: The collected works of Mihaly Csikszentmihalyi (pp. 21–34). Springer.

Li, N., Kidziński, Ł., Jermann, P., & Dillenbourg, P. (2015). MOOC video interaction patterns: What do they tell us? Proceedings of the 10th European Conference on Technology Enhanced Learning: Design for Teaching and Learning in a Networked World (EC-TEL 2015), 15–17 September 2015, Toledo, Spain (pp. 197–210). Lecture Notes in Computer Science, vol. 9307. Springer.

Marsh, H. W. (1987). Students’ evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11(3), 253–388.

Marsh, H. W., & Roche, L. (1993). The use of students’ evaluations and an individually structured intervention to enhance university teaching effectiveness. American Educational Research Journal, 30(1), 217–251.

Michos, K., Lang, C., Hernández-Leo, D., & Price-Dennis, D. (2020). Involving teachers in learning analytics design: Lessons learned from two case studies. Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 94–99). ACM Press.

Mirriahi, N., & Vigentini, L. (2017). Analytics of learner video use. C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.). (2017). The handbook of learning analytics, 1st ed. (pp. 251–267). Society for Learning Analytics Research (SoLAR).

Mitra, R., & Chavan, P. (2019). DEBE feedback for large lecture classroom analytics. Proceedings of the 9th International Conference on Learning Analytics and Knowledge (LAK ’19), 4–8 March 2019, Tempe, AZ, USA (pp. 426–430). ACM Press. ACM.

Montrieux, H., Vangestel, S., Raes, A., Matthys, P., & Schellens, T. (2015). Blending face-to-face higher education with web-based lectures: Comparing different didactical application scenarios. Educational Technology and Society, 18(1), 170–182.

Morris, L. V. (2009). Little lectures? Innovative Higher Education, 34(2), 67–68.

Namuddu, J., & Watts, P. N. (2020). Choice of device to view video lectures: An analysis of two independent cohorts of first-year university students. Research in Learning Technology, 28.

Newtson, D. (1973). Attribution and the unit of perception of ongoing behavior. Journal of Personality and Social Psychology, 28(1), 28.

Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4), 315–341.

Pham, P., & Wang, J. (2015). AttentiveLearner: Improving mobile MOOC learning via implicit heart rate tracking. In C. Conati, N. Heffernan, A. Mitrovic, & M. Verdejo (Eds.), Proceedings of the 17th International Conference on Artificial Intelligence in Education (AIED ʼ15), 22–26 June 2015, Madrid, Spain (pp. 367–376). Springer.

Pham, P., & Wang, J. (2017). AttentiveLearner2: A multimodal approach for improving MOOC learning on mobile devices. In E. André, R. S. Baker, X. Hu, M. M. T. Rodrigo, & B. du Boulay (Eds.), Proceedings of the 18th International Conference on Artificial Intelligence in Education (AIED 2017), 28 June–1 July 2017, Wuhan, China (pp. 561–564). Springer.

Pham, P., & Wang, J. (2018a). Adaptive review for mobile MOOC learning via multimodal physiological signal sensing: A longitudinal study. Proceedings of the 20th ACM International Conference on Multimodal Interaction (ICMI ’18) 14–18 October 2019, Bolder, CO, USA (pp. 63–72). ACM Press.

Pham, P., & Wang, J. (2018b). Predicting learners’ emotions in mobile MOOC learning via a multimodal intelligent tutor. In R. Nkambou, R. Azevedo, & J. Vassileva (Eds.), Proceedings of the 14th International Conference on Intelligent Tutoring Systems (ITS 2018), 11–15 June 2018, Montreal, QC, Canada (Lecture Notes in Computer Science, vol. 10858, pp. 150–159). Springer.

Preston, G., Phillips, R., Gosper, M., McNeill, M., Woo, K., & Green, D. (2010). Web-based lecture technologies: Highlighting the changing nature of teaching and learning. Australasian Journal of Educational Technology, 26(6).

Rhine, S. (1998). The role of research and teachers’ knowledge base in professional development. Educational Researcher, 27(5), 27–31.

Richardson, J. T. E. (2005). Instruments for obtaining student feedback: A review of the literature. Assessment & Evaluation in Higher Education, 30(4), 387–415.

Risko, E. F., Foulsham, T., Dawson, S., & Kingstone, A. (2012). The collaborative lecture annotation system (CLAS): A new TOOL for distributed learning. IEEE Transactions on Learning Technologies, 6(1), 4–13.

Rivera-Pelayo, V., Munk, J., Zacharias, V., & Braun, S. (2013). Live interest meter: Learning from quantified feedback in mass lectures. Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), 8–12 April 2013, Leuven, Belgium (pp. 23–27). ACM Press.

Roche, L. A., & Marsh, H. W. (2002). Teaching self-concept in higher education. In N. Hativa & P. Goodyear (Eds.), Teacher thinking, beliefs and knowledge in higher education (pp. 179–218). Springer.

Santagata, R., Zannoni, C., & Stigler, J. W. (2007). The role of lesson analysis in pre-service teacher education: An empirical investigation of teacher learning from a virtual video-based field experience. Journal of Mathematics Teacher Education, 10(2), 123–140.

Scagnoli, N. I., Choo, J., & Tian, J. (2017). Students’ insights on the use of video lectures in online classes. British Journal of Educational Technology, 50(1), 399–414.

Scheffel, M. (2017). The evaluation framework for learning analytics. Open Universiteit in the Netherlands, Welten Institute, Research Centre for Learning, Teaching and Technology.

Scheffel, M., Drachsler, H., Stoyanov, S., & Specht, M. (2014). Quality indicators for learning analytics. Journal of Educational Technology & Society, 17(4), 117–132.

Schön, D. A. (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. Jossey-Bass.

Schwendimann, B. A., Rodriguez-Triana, M. J., Vozniuk, A., Prieto, L. P., Boroujeni, M. S., Holzer, A., Gillet, D., & Dillenbourg, P. (2016). Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies, 10(1), 30–41.

Sharma, K., Alavi, H. S., Jermann, P., & Dillenbourg, P. (2016). A gaze-based learning analytics model: In-video visual feedback to improve learner’s attention in MOOCs. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 417–421). ACM Press.

Speer, N. K., Swallow, K. M., & Zacks, J. M. (2003). Activation of human motion processing areas during event perception. Cognitive, Affective, & Behavioral Neuroscience, 3(4), 335–345.

Srivastava, N., Nawaz, S., Newn, J., Lodge, J., Velloso, E., M. Erfani, S., Gašević, D., & Bailey, J. (2021). Are you with me? Measurement of learners’ video-watching attention with eye tracking. Proceedings of the 11th International Conference on Learning Analytics and Knowledge (LAK ’21), 12–16 April 2021, Irvine, CA, USA (pp. 88–98). ACM Press.

Srivastava, N., Velloso, E., Lodge, J. M., Erfani, S., & Bailey, J. (2019). Continuous evaluation of video lectures from real-time difficulty self-report. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19), 4–9 May 2019, Glasgow, Scotland, UK (Paper No. 586). ACM Press.

Stöhr, C., Demazière, C., & Adawi, T. (2020). The polarizing effect of the online flipped classroom. Computers & Education, 147, 103789.

Tze, V. M., Daniels, L. M., & Klassen, R. M. (2016). Evaluating the relationship between boredom and academic outcomes: A meta-analysis. Educational Psychology Review, 28(1), 119–144.

van Es, E. A., Stockero, S. L., Sherin, M. G., Van Zoest, L. R., & Dyer, E. (2015). Making the most of teacher self-captured video. Mathematics Teacher Educator, 4(1), 6–19.

Vatrapu, R. K. (2012). Towards semiology of teaching analytics. Workshop Towards Theory and Practice of Teaching Analytics at the European Conference on Technology Enhanced Learning (TaPTA-2012) 18 September 2012, Saarbrücken, Germany. CEUR Workshop Proceedings.

Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186–204.

Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10), 1500–1509.

Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Assche, F., Parra, G., & Klerkx, J. (2014). Learning dashboards: An overview and future research opportunities. Personal and Ubiquitous Computing, 18(6), 1499–1514.

Wise, A. F., & Jung, Y. (2019). Teaching with analytics: Towards a situated model of instructional decision-making. Journal of Learning Analytics, 6(2), 53–69.

Xhakaj, F., Aleven, V., & McLaren, B. M. (2016). How teachers use data to help students learn: Contextual inquiry for the design of a dashboard. Proceedings of the 11th European Conference on Technology Enhanced Learning (EC-TEL 2016), 13–16 September 2016, Lyon, France (pp. 340–354). Lecture Notes in Computer Science, vol. 9891. Springer.

Xiao, X., Pham, P., & Wang, J. (2017). Dynamics of affective states during MOOC learning. In E. André, R. S. Baker, X. Hu, M. M. T. Rodrigo, & B. du Boulay (Eds.), Proceedings of the 18th International Conference on Artificial Intelligence in Education (AIED 2017), 28 June–1 July 2017, Wuhan, China (pp. 586–589). Springer.

Xiao, X., & Wang, J. (2015). Towards attentive, bi-directional MOOC learning on mobile devices. Proceedings of the 17th ACM International Conference on Multimodal Interaction (ICMI ’15), 9–13 November 2015, Seattle, WA, USA (pp. 163–170). ACM Press.

Xiao, X., & Wang, J. (2016). Context and cognitive state triggered interventions for mobile MOOC learning. Proceedings of the 18th ACM International Conference on Multimodal Interaction (ICMI ’16), 12–16 November 2016, Tokyo, Japan (pp. 378–385). ACM Press..

Yoon, M., Lee, J., & Jo, I.-H. (2021). Video learning analytics: Investigating behavioral patterns and learner clusters in video-based online learning. The Internet and Higher Education, 50, 100806.




How to Cite

Chavan, P., & Mitra, R. (2022). Tcherly: : A Teacher-facing Dashboard for Online Video Lectures. Journal of Learning Analytics, 1-27.



Research Papers