Is Seeing the Instructor’s Face or Gaze in Online Videos Helpful for Learning?
DOI:
https://doi.org/10.18608/jla.2024.8235Keywords:
online learning, instructor presence, shared gaze visualizations, multimodal learning analytics, extended conference paperAbstract
Over the last decade, the prevalence of online learning has dramatically increased. As part of their curriculum, students are expected to spend more and more time watching videos. These videos tend to follow a widespread format: a screen recording of slides with a picture-in-picture (PiP) image of the instructor’s face. While this format is ubiquitous, there is mixed evidence that it supports student learning. In this paper, we explore alternative formats for designing educational videos. Based on prior work showing the significance of joint attention for social learning, we create instructional videos augmented with the instructor’s gaze and/or face. Testing these formats in a semester-long online course using a 2x2 experimental design, we found that showing the instructor’s face had no significant effect on learning, while adding the instructor’s eye-tracking data to the video promoted conceptual understanding of the material. Mediation analysis showed that joint visual attention played a significant mediatory role for learning. We conclude by discussing the implications of these findings and formulate recommendations for designing learning videos.
References
Adnan, M., & Anwar, K. (2020). Online learning amid the COVID-19 pandemic: Students’ perspectives. Journal of Pedagogical Sociology and Psychology, 2(1), 45–51. https://doi.org/10.33902/jpsp.2020261309
Alemdag, E. (2022). Effects of instructor-present videos on learning, cognitive load, motivation, and social presence: A meta-analysis. Education and Information Technologies, 27(9), 12713–12742. https://doi.org/10.1007/s10639-022-11154-w
Al-Mawee, W., Kwayu, K. M., & Gharaibeh, T. (2021). Student’s perspective on distance learning during COVID-19 pandemic: A case study of Western Michigan University, United States. International Journal of Educational Research Open, 2, 100080. https://doi.org/10.1016/j.ijedro.2021.100080
Akkil, D., Thankachan, B., & Isokoski, P. (2018). I see what you see: Gaze awareness in mobile video collaboration. Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications (ETRA ’18), 14–17 June 2018, Warsaw, Poland (Article 32). ACM Press. https://doi.org/10.1145/3204493.3204542
Barron, B. (2003). When smart groups fail. Journal of the Learning Sciences, 12(3), 307–359. https://doi.org/10.1207/s15327809jls1203_1
Blikstein, P., & Worsley, M. (2016). Multimodal learning analytics and education data mining: Using computational technologies to measure complex learning tasks. Journal of Learning Analytics, 3(2), 220–238. https://doi.org/10.18608/jla.2016.32.11
Baltrušaitis, T., Robinson, P., & Morency, L.-P. (2016). OpenFace: An open source facial behavior analysis toolkit. 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), 7–10 March 2016, Lake Placid, NY, USA. IEEE. https://doi.org/10.1109/wacv.2016.7477553
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017). Realtime multi-person 2D pose estimation using part affinity fields. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 21–26 July 2017, Honolulu, HI, USA (pp. 1302–1310). https://doi.org/10.1109/cvpr.2017.143
Chen, C.-M., & Wu, C.-H. (2015). Effects of different video lecture types on sustained attention, emotion, cognitive load, and learning performance. Computers & Education, 80, 108–121. https://doi.org/10.1016/j.compedu.2014.08.015
Chen, Y., Zhou, J., Gao, J., Gao, G., Wang, S., & Zhang, W. (2021). Joint gaze estimation and facial expression for student engagement prediction in collaborative learning. 2021 IEEE International Conference on Engineering, Technology & Education (TALE), 5–8 December 2021, Wuhan, China (pp. 703–707). IEEE. https://doi.org/10.1109/tale52509.2021.9678844
Clark, H. H., & Brennan S. E. (1991). Grounding in communication. In L. Resnick, J. Levine, & S. Teasley (Eds.), Perspectives on socially shared cognition (pp. 127–149). American Psychological Association. https://doi.org/10.1037/10096-006
D’Angelo, S., & Schneider, B. (2021). Shared gaze visualizations in collaborative interactions: Past, present and future. Interacting With Computers, 33(2), 115–133. https://doi.org/10.1093/iwcomp/iwab015
D’Angelo, S., Brewer, J., & Gergle, D. (2019). Iris: A tool for designing contextually relevant gaze visualizations. Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications (ETRA ’19), 25–28 June 2019, Denver, CO, USA (Article 79). https://doi.org/10.1145/3314111.3318228
D’Angelo, S., & Gergle, D. (2016). Gazed and confused: Understanding and designing shared gaze for remote collaboration. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16), 7–12 May 2016, San Jose, CA, USA (pp. 2492–2496). ACM Press. https://doi.org/10.1145/2858036.2858499
Denisova, O. A., Lekhanova, O. L., & Gudina, T. V. (2020). Problems of distance learning for students with disabilities in a pandemic. SHS Web of Conferences, 87, 00044. https://doi.org/10.1051/shsconf/20208700044
Djamasbi, S., Siegel, M., & Tullis, T. S. (2012). Faces and viewing behavior: An exploratory investigation. AIS Transactions on Human–Computer Interaction, 4(3), 190–211. https://doi.org/10.17705/1thci.00046
Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. Chapman & Hall/CRC. https://doi.org/10.1201/9780429246593
Fyfield, M., Henderson, M., & Phillips, M. (2022). Improving instructional video design: A systematic review. Australasian Journal of Educational Technology, 38(3), 155–183. https://doi.org/10.14742/ajet.7296.
Garbarino, M., Lai, M., Bender, D., Picard, R. W., & Tognetti, S. (2014). Empatica E3: A wearable wireless multi-sensor device for real-time computerized biofeedback and data acquisition. Proceedings of the 4th International Conference on Wireless Mobile Communication and Healthcare: Transforming Healthcare Through Innovations in Mobile and Wireless Technologies (MOBIHEALTH 2014), 3–5 November 2014, Athens, Greece (pp. 39–42). IEEE. https://doi.org/10.4108/icst.mobihealth.2014.257418
Guo, Z., & Barmaki, R. (2020). Deep neural networks for collaborative learning analytics: Evaluating team collaborations using student gaze point prediction. Australasian Journal of Educational Technology, 36(6), 53–71. https://doi.org/10.14742/ajet.6436
Hartshorn, K. J., & McMurry, B. L. (2020). The effects of the COVID-19 pandemic on ESL learners and TESOL practitioners in the United States. International Journal of TESOL Studies, 2(2), 140–156. https://doi.org/10.46451/ijts.2020.09.11
Henderson, M. L., & Schroeder, N. L. (2021). A systematic review of instructor presence in instructional videos: Effects on learning and affect. Computers and Education Open, 2, 100059. https://doi.org/10.1016/j.caeo.2021.100059
Jarodzka, H., Van Gog, T., Dorr, M., Scheiter, K., & Gerjets, P. (2013). Learning to see: Guiding students’ attention via a model’s eye movements fosters learning. Learning and Instruction, 25, 62–70. https://doi.org/10.1016/j.learninstruc.2012.11.004
Kizilcec, R. F., Bailenson, J. N., & Gomez, C. J. (2015). The instructor’s face in video instruction: Evidence from two large-scale field studies. Journal of Educational Psychology, 107(3), 724–739. https://doi.org/10.1037/edu0000013
Kizilcec, R. F., Papadopoulos, K., & Sritanyaratana, L. (2014). Showing face in video instruction: Effects on information retention, visual attention, and affect. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ʼ14), 26 April–1 May 2014, Toronto, ON, Canada (pp. 2095–2102). ACM Press. https://doi.org/10.1145/2556288.2557207
Kokoç, M., IIgaz, H., & Altun, A. (2020). Effects of sustained attention and video lecture types on learning performances. Educational Technology Research and Development, 68(6), 3015–3039. https://doi.org/10.1007/s11423-020-09829-7
Li, W., Wang, F., Mayer, R. E., & Liu, H. (2019). Getting the point: Which kinds of gestures by pedagogical agents improve multimedia learning? Journal of Educational Psychology, 111(8), 1382–1395. https://doi.org/10.1037/edu0000352
McAlpin, E., Levine, M., & Plass, J. L. (2023). Comparing two whole task patient simulations for two different dental education topics. Learning and Instruction, 83, 101690. https://doi.org/10.1016/j.learninstruc.2022.101690
Mason, L., Pluchino, P., & Tornatora, M. C. (2015). Eye-movement modeling of integrative reading of an illustrated text: Effects on processing and learning. Contemporary Educational Psychology, 41, 172–187. https://doi.org/10.1016/j.cedpsych.2015.01.004
Mautone P. D., & Mayer R. E. (2001). Signaling as a cognitive guide in multimedia learning. Journal of Educational Psychology, 93(2), 377–389. https://doi.org/10.1037//0022-0663.93.2.377
Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 31–48). Cambridge University Press. https://doi.org/10.1017/cbo9780511816819.004
Mayer, R. E. (2021). Evidence-based principles for how to design effective instructional videos. Journal of Applied Research in Memory and Cognition, 10(2), 229–240. https://doi.org/10.1016/j.jarmac.2021.03.007
Mayer, R. E., Fiorella, L., & Stull, A. (2020). Five ways to increase the effectiveness of instructional video. Educational Technology Research and Development, 68(3), 837–852. https://doi.org/10.1007/s11423-020-09749-6
Melnyk, R., Campbell, T., Holler, T., Cameron, K., Saba, P., Witthaus, M. W., Joseph, J., & Ghazi, A. (2021). See like an expert: Gaze-augmented training enhances skill acquisition in a virtual reality robotic suturing task. Journal of Endourology, 35(3), 376–382. https://doi.org/10.1089/end.2020.0445
Mundy, P., Sigman, M., & Kasari, C. (1990). A longitudinal study of joint attention and language development in autistic children. Journal of Autism and Developmental Disorders, 20(1), 115–128. https://doi.org/10.1007/bf02206861
Ng, Y. Y., & Przybyłek, A. (2021). Instructor presence in video lectures: Preliminary findings from an online experiment. IEEE Access, 9, 36485–36499. https://doi.org/10.1109/access.2021.3058735
Paciej-Woodruff, A. (2021). The case for your face: Teacher presence in asynchronous education courses. Stop feeling uncomfortable and start recording your face to humanize the online experience. Society for Information Technology & Teacher Education International Conference (SITE), 29 March 2021, Online (pp. 535–539). Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/219181/
Papoutsaki, A., Sangkloy, P., Laskey, J., Daskalova, N., Huang, J., & Hays, J. (2016). WebGazer: Scalable webcam eye tracking using user interactions. In S. Kambhampati (Ed.), Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI ’16), 9–15 July 2016, New York, NY, USA (pp. 3839–3845). AAAI Press/International Joint Conferences on Artificial Intelligence. https://www.ijcai.org/Proceedings/16/Papers/540.pdf
Pi, Z., Xu, K., Liu, C., & Yang, J. (2020). Instructor presence in video lectures: Eye gaze matters, but not body orientation. Computers & Education, 144, 103713. https://doi.org/10.1016/j.compedu.2019.103713
Sauter, M., Wagner, T., & Huckauf, A. (2022). Distance between gaze and laser pointer predicts performance in video-based e-learning independent of the presence of an on-screen instructor. 2022 Symposium on Eye Tracking Research and Applications (ETRA ’22), 8–11 June 2022, Seattle, WA, USA (Article 26). ACM Press. https://doi.org/10.1145/3517031.3529620
Serdar, C. C., Cihan, M., Yücel, D., & Serdar, M. A. (2021). Sample size, power and effect size revisited: Simplified and practical approaches in pre-clinical, clinical and laboratory studies. Biochemia Medica, 31(1), 010502. https://doi.org/10.11613/bm.2021.010502
Schneider, B. (2020). A methodology for capturing joint visual attention using mobile eye-trackers. JoVE, (155), e60670. https://doi.org/10.3791/60670-v
Schneider, B. (2023). Three Challenges in Implementing Multimodal Learning Analytics in Real-World Learning Environments. Learning: Research and Practice, 1-10. https://doi.org/10.1080/23735082.2023.2270611
Schneider, B., & Bryant, T. (2024). Using mobile dual eye-tracking to capture cycles of collaboration and cooperation in co-located dyads. Cognition and Instruction, 42(1), 26–55. https://doi.org/10.1080/07370008.2022.2157418
Schneider, B., Davis, R., Martinez-Maldonado, R., Biswas, G., Worsley, M., & Rummel, N. (2024). Stepping outside the ivory tower: How can we implement multimodal learning analytics in ecological settings, and turn complex temporal data sources into actionable insights? Proceedings of the 17th International Conference on Computer-Supported Collaborative Learning (CSCL 2024), 10–14 June 2024, Buffalo, NY, USA (pp. 323–330). International Society of the Learning Sciences. https://doi.org/10.22318/cscl2024.259119
Schneider, B., Dich, Y., & Radu, I. (2020). Unpacking the relationship between existing and new measures of physiological synchrony and collaborative learning: A mixed methods study. International Journal of Computer-Supported Collaborative Learning, 15(1), 89–113. https://doi.org/10.1007/s11412-020-09318-2
Schneider, B., Feng, D., & Sung, G. (2023). Joint visual attention predicts learning in 1-on-1 remote teaching: A dual eye-tracking study. Proceedings of the 16th International Conference on Computer-Supported Collaborative Learning (CSCL 2023), 10–15 June 2023, Montréal, QC, Canada (pp. 83–90). International Society of the Learning Sciences. https://doi.org/10.22318/cscl2023.849100
Schneider, B., & Pea, R. (2013). Real-time mutual gaze perception enhances collaborative learning and collaboration quality. International Journal of Computer-Supported Collaborative Learning, 8(4), 375–397. https://doi.org/10.1007/s11412-013-9181-4
Schneider, B., Sharma, K., Cuendet, S., Zufferey, G., Dillenbourg, P., & Pea, R. (2018). Leveraging mobile eye-trackers to capture joint visual attention in co-located collaborative learning groups. International Journal of Computer-Supported Collaborative Learning, 13(3), 241–261. https://doi.org/10.1007/s11412-018-9281-2
Schneider, B., Sung, G., Chng, E., & Yang, S. (2022). How Can High-Frequency Sensors Capture Collaboration? A Review of the Empirical Links Between Multimodal Metrics and Collaborative Constructs. Sensors, 21(24), 8185.
Sharma, K., Alavi, H. S., Jermann, P., & Dillenbourg, P. (2016). A gaze-based learning analytics model: In-video visual feedback to improve learner’s attention in MOOCs. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 417–421). ACM Press. https://doi.org/10.1145/2883851.2883902
Sharma, K., D’Angelo, S., Gergle, D., & Dillenbourg, P. (2016). Visual augmentation of deictic gestures in MOOC videos. In C. K. Looi, J. L. Polman, U. Cress, & P. Reimann (Eds.), Transforming learning, empowering learners: The international conference of the learning sciences (ICLS) 2016 (Vol. 1, pp. 202–209). International Society of the Learning Sciences. https://repository.isls.org/handle/1/117
Sharma, K., Jermann, P., & Dillenbourg, P. (2014). “With-me-ness”: A gaze-measure for students’ attention in MOOCs. Learning and Becoming in Practice: Proceedings of the International Conference of the Learning Sciences (ICLS ’14), 23–27 June 2014, Boulder, CO, USA (Vol. 2, pp. 1017–1022). International Society of the Learning Sciences. https://repository.isls.org//handle/1/924
Sharma, K., Jermann, P., & Dillenbourg, P. (2015). Displaying teacher’s gaze in a MOOC: Effects on students’ video navigation patterns. In G. Conole, T. Klobučar, C. Rensing, J. Konert, & E. Lavoué (Eds.), Design for teaching and learning in a networked world: 10th European conference on technology enhanced learning, EC-TEL 2015, Toledo, Spain, September 15–18, 2015, proceedings (pp. 325–338). Springer Cham. https://doi.org/10.1007/978-3-319-24258-3_24
Špakov, O., Niehorster, D., Istance, H., Räihä, K.-J., & Siirtola, H. (2019). Two-way gaze sharing in remote teaching. In D. Lamas, F. Loizides, L. Nacke, H. Petrie, M. Winckler, & P. Zaphiris (Eds.), Human–computer interaction – INTERACT 2019: 17th IFIP TC 13 international conference, Paphos, Cyprus, September 2–6, 2019, proceedings, part II (pp. 242–251). Springer Cham. https://doi.org/10.1007/978-3-030-29384-0_16
Stull, A. T., Fiorella, L., & Mayer, R. E. (2018). An eye-tracking analysis of instructor presence in video lectures. Computers in Human Behavior, 88, 263–272. https://doi.org/10.1016/j.chb.2018.07.019
Sung, G., Feng, T., & Schneider, B. (2021). Learners learn more and instructors track better with real-time gaze sharing. Proceedings of the ACM on Human–Computer Interaction (CSCW1, Vol. 5, Article 134). ACM Press. https://doi.org/10.1145/3449208
Tomasello, M. (1995). Joint attention as social cognition. In C. Moore & P. J. Dunham (Eds.), Joint attention: Its origins and role in development (pp. 103–130). Lawrence Erlbaum. https://doi.org/10.4324/9781315806617
van Wermeskerken, M., Ravensbergen, S., & van Gog, T. (2018). Effects of instructor presence in video modeling examples on attention and learning. Computers in Human Behavior, 89, 430–438. https://doi.org/10.1016/j.chb.2017.11.038
Westfall, J. (2015, May 26). Think about total N, not n per cell. Cookie Scientist. https://web.archive.org/web/20190216111000/http://jakewestfall.org/blog/index.php/2015/05/26/think-about-total-n-not-n-per-cell/#expand
Wilson, K. E., Martinez, M., Mills, C., D’Mello, S., Smilek, D., & Risko, E. F. (2018). Instructor presence effect: Liking does not always lead to learning. Computers & Education, 122, 205–220. https://doi.org/10.1016/j.compedu.2018.03.011
Xie, H., Zhao, T., Deng, S., Peng, J., Wang, F., & Zhou, Z. (2021). Using eye movement modelling examples to guide visual attention and foster cognitive performance: A meta‐analysis. Journal of Computer Assisted Learning, 37(4), 1194–1206. https://doi.org/10.1111/jcal.12568
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution 4.0 International License.
