Deep Networks for Collaboration Analytics

Promoting Automatic Analysis of Face-to-Face Interaction in the Context of Inquiry-Based Learning




collaboration analytics, computational models, computer-supported collaborative learning, CSCL, CSCIL, deep networks, inquiry-based learning, word embedding


Scholars have applied automatic content analysis to study computer-mediated communication in computer-supported collaborative learning (CSCL). Since CSCL also takes place in face-to-face interactions, we studied the automatic coding accuracy of manually transcribed face-to-face communication. We conducted our study in an authentic higher-education physics context where computer-supported collaborative inquiry-based learning (CSCIL) is a popular pedagogical approach. Since learners’ needs for support in CSCIL vary in the different inquiry phases (orientation, conceptualization, investigation, conclusion, and discussion), we studied, first, how the coding accuracy of five computational models (based on word embeddings and deep neural networks with attention layers) differed in the various inquiry-based learning (IBL) phases when compared to human coding. Second, we investigated how the different features of the best performing computational model improved the coding accuracy. The study indicated that the accuracy of the best performing computational model (differentiated attention with pre-trained static embeddings) was slightly better than that of the human coder (58.9% vs. 54.3%). We also found that considering the previous and following utterances, as well as the relative position of the utterance, improved the model’s accuracy. Our method illustrates how computational models can be trained for specific purposes (e.g., to code IBL phases) with small data sets by using pre-trained models.


Alfieri, L., Brooks, P. J., Aldrich, N. J., & Tenenbaum, H. R. (2011). Does discovery-based instruction enhance learning? Journal of Educational Psychology, 103(1), 1–18.

Bell, T., Urhahne, D., Schanze, S., & Ploetzner, R. (2010). Collaborative inquiry learning: Models, tools, and challenges. International Journal of Science Education, 32(3), 349–377.

Bybee, R. W., Taylor, J. A., Gardner, A., Van Scotter, P., Powell, J. C., Westbrook, A., & Landes, N. (2006). The BSCS 5E instructional model: Origins and effectiveness. Biological Sciences Curriculum Study.

de Jong, T., & Lazonder, A. W. (2014). The guided discovery learning principle in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 371–390). Cambridge, UK: Cambridge University Press.

Dönmez, P., Rosé, C., Stegmann, K., Weinberger, A., & Fischer, F. (2005). Supporting CSCL with automatic corpus analysis technology. In T. Koschmann, D. D. Suthers, & T.-W. Chan (Eds.), Learning 2005: The Next 10 Years! Proceedings of the 2005 Conference on Computer Support for Collaborative Learning (CSCL ’05), May 30–June 4 2005, Taipei, Taiwan (pp. 125–134). International Society of the Learning Sciences.

Donnelly, D. F., Linn, M. C., & Ludvigsen, S. (2014). Impacts and characteristics of computer-based science inquiry learning environments for precollege students. Review of Educational Research, 84(4), 572–608.

Espinoza, C., Lämsä, J., Araya, R., Hämäläinen, R., Jiménez, A. G., Gormaz, R., & Viiri, J. (2019). Automatic content analysis in collaborative inquiry-based learning. In O. Levrini & G. Tasquier (Eds.), The Beauty and Pleasure of Understanding: Engaging with Contemporary Challenges Through Science Education. Proceedings of the European Science Education Research Association Conference (ESERA 2019), 26–30 August, 2019, Bologna, Italy (Part 18, pp. 2041–2050). University of Bologna.

Hämäläinen, R., De Wever, B., Waaramaa, T., Laukkanen, A., & Lämsä, J. (2018). It’s not only what you say, but how you say it: Investigating the potential of prosodic analysis as a method to study teacher’s talk. Frontline Learning Research, 6(3), 204–227.

Henri, F. (1992). Computer conferencing and content analysis. In A. R. Kaye (Ed.), Collaborative learning through computer conferencing: The Najadan Papers (pp. 117–136). Springer-Verlag.

Howard, S. K., Thompson, K., Yang, J., Ma, J., Pardo, A., & Kanasa, H. (2017). Capturing and visualizing: Classroom analytics for physical and digital collaborative learning processes. In B. K. Smith, M. Borge, E. Mercier, & K. Y. Lim (Eds.), Making a Difference: Prioritizing Equity and Access in CSCL. Proceedings of the 12th International Conference on Computer Supported Collaborative Learning (CSCL 2017) 18–22 June 2017, Philadelphia, PA, USA (Vol. 2, pp. 801–802). International Society of the Learning Sciences.

Hu, D. (2020). An introductory survey on attention mechanisms in NLP problems. In Y. Bi, R. Bhatia, & S. Kapoor (Eds.), Intelligent systems and applications: Proceedings of the 2019 Intelligent Systems Conference (IntelliSys) (pp. 432–448). Springer.

Jeong, H., Hmelo-Silver, C. E., & Jo, K. (2019). Ten years of computer-supported collaborative learning: A meta-analysis of CSCL in STEM education during 2005–2014. Educational Research Review, 28, 100284.

Kim, Y., Denton, C., Hoang, L., & Rush, A. M. (2017). Structured attention networks.

Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization.

Knight, S., & Littleton, K. (2015). Discourse centric learning analytics: Mapping the terrain. Journal of Learning Analytics, 2(1), 185–209.

Koskinen, P., Lämsä, J., Maunuksela, J., Hämäläinen, R., & Viiri, J. (2018). Primetime learning: Collaborative and technology-enhanced studying with genuine teacher presence. International Journal of STEM Education, 5(20), 1–13.

Kotsiantis, S. B. (2007). Supervised machine learning: A review of classification techniques. Informatica (Ljubljana), 31(3), 249–268.

Kronholm, H., Caballero, D., Mansikkaniemi, A., Araya, R., Lehesvuori, S., Pertilä, P., Virtanen, T., Kurimo, M., & Viiri, J. (2017). The automatic analysis of classroom talk. Proceedings of the Annual FMSERA Symposium 2016, Joensuu, Finland (pp. 142–151).

Lämsä, J., Hämäläinen, R., Koskinen, P., Viiri, J., & Mannonen, J. (2020). The potential of temporal analysis: Combining log data and lag sequential analysis to investigate temporal differences between scaffolded and non-scaffolded group inquiry-based learning processes. Computers & Education, 143, 103674.

Lämsä, J., Hämäläinen, R., Koskinen, P., & Viiri, J. (2018). Visualising the temporal aspects of collaborative inquiry-based learning processes in technology-enhanced physics learning. International Journal of Science Education, 40(14), 1697–1717.

Lemke, J. L. (2000). Across the scales of time: Artifacts, activities, and meanings in ecosocial systems. Mind, Culture, and Activity, 7(4), 273–290.

Luotolahti, J., Kanerva, J., Laippala, V., Pyysalo, S., & Ginter, F. (2015). Towards universal web parsebanks. Proceedings of the Third International Conference on Dependency Linguistics (Depling 2015), 24–26 August 2015, Uppsala, Sweden (pp. 211–220).

Maldonado, R. M., Worsley, M., Schneider, B., & Kharrufa, A. (2019). International workshop on collaboration analytics: Making learning visible in collaborative settings. In K. Lund, G. Niccolai, E. Lavoué, C. Hmelo-Silver, G. Gweon, & M. Baker (Eds.), A Wide Lens: Combining Embodied, Enactive, Extended, and Embedded Learning in Collaborative Settings. Proceedings of the 13th International Conference on Computer Supported Collaborative Learning (CSCL 2019), 17–21 June 2019, Lyon, France (Vol. 2, p. 1044). International Society of the Learning Sciences.

Matuk, C., & Linn, M. C. (2018). Why and how do middle school students exchange ideas during science inquiry? International Journal of Computer-Supported Collaborative Learning, 13(3), 263–299.

Mercer, N. (2008). The seeds of time: Why classroom dialogue needs a temporal analysis. Journal of the Learning Sciences, 17(1), 33–59.

Mercer, N., Wegerif, R., & Dawes, L. (1999). Children’s talk and the development of reasoning in the classroom. British Educational Research Journal, 25(1), 95–111.

Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space.

Mu, J., Stegmann, K., Mayfield, E., Rosé, C., & Fischer, F. (2012). The ACODEA framework: Developing segmentation and classification schemes for fully automatic analysis of online discussions. International Journal of Computer-Supported Collaborative Learning, 7(2), 285–305.

Neuendorf, K. A. (2002). The content analysis: Guidebook (1st ed.). Thousand Oaks, CA: Sage Publications.

Noroozi, O., Alikhani, I., Järvelä, S., Kirschner, P. A., Juuso, I., & Seppänen, T. (2019). Multimodal data to design visual learning analytics for understanding regulation of learning. Computers in Human Behavior, 100, 298–304.

Olsen, J. K., Sharma, K., Rummel, N., & Aleven, V. (2020). Temporal analysis of multimodal data to predict collaborative learning outcomes. British Journal of Educational Technology, 51(5), 1527–1547.

Pedaste, M., Mäeots, M., Siiman, L. A., de Jong, T., van Riesen, S. A. N., Kamp, E. T., Manoli, C. C., Zacharia, Z. C., & Tsourlidaki, E. (2015). Phases of inquiry-based learning: Definitions and the inquiry cycle. Educational Research Review, 14, 47–61.

Pennebaker, J. W., Boyd, R. L., Jordan, K., & Blackburn, K. (2015). The development and psychometric properties of LIWC2015. University of Texas at Austin.

Rosé, C., Wang, Y., Cui, Y., Arguello, J., Stegmann, K., Weinberger, A., & Fischer, F. (2008). Analyzing collaborative learning processes automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning. International Journal of Computer-Supported Collaborative Learning, 3(3), 237–271.

Schneider, B., & Blikstein, P. (2014). Unraveling students’ interaction around a tangible interface using gesture recognition. In J. Stamper et al. (Eds.), Proceedings of the 7th International Conference on Educational Data Mining (EDM2014), 4–7 July 2014, London, UK (pp. 320–323). International Educational Data Mining Society.

Schneider, B., & Pea, R. (2013). Real-time mutual gaze perception enhances collaborative learning and collaboration quality. International Journal of Computer-Supported Collaborative Learning, 8(4), 375–397.

Schneider, B., & Pea, R. (2015). Does seeing one another’s gaze affect group dialogue? A computational approach. Journal of Learning Analytics, 2(2), 107–133.

Sins, P. H. M., Savelsbergh, E. R., van Joolingen, W. R., & van Hout-Wolters, B. H. A. M. (2011). Effects of face-to-face versus chat communication on performance in a collaborative inquiry modeling task. Computers & Education, 56(2), 379–387.

Smith, J., Bratt, H., Richey, C., Bassiou, N., Shriberg, E., Tsiartas, A., D’Angelo, C., & Alozie, N. (2016). Spoken interaction modeling for automatic assessment of collaborative learning. Proceedings of Speech Prosody 8, 31 May–3 June 2016, Boston, MA, USA (pp. 277–281).

Tan, E. (2018). Effects of two differently sequenced classroom scripts on common ground in collaborative inquiry learning. Instructional Science, 46(6), 893–919.

Thompson, K., Ashe, D., Carvalho, L., Goodyear, P., Kelly, N., & Parisio, M. (2013). Processing and visualizing data in complex learning environments. American Behavioral Scientist, 57(10), 1401–1420.

Tshitoyan, V., Dagdelen, J., Weston, L., Dunn, A., Rong, Z., Kononova, O., Persson, K. A., Ceder, G., & Jain, A. (2019). Unsupervised word embeddings capture latent knowledge from materials science literature. Nature, 571(7763), 95–98.

van Joolingen, W. R., de Jong, T., Lazonder, A. W., Savelsbergh, E. R., & Manlove, S. (2005). Co-Lab: Research and development of an online learning environment for collaborative scientific discovery learning. Computers in Human Behavior, 21(4), 671–688.

van Leeuwen, A. (2015). Learning analytics to support teachers during synchronous CSCL: Balancing between overview and overload. Journal of Learning Analytics, 2(2), 138–162.

White, B. Y., & Frederiksen, J. R. (1998). Inquiry, modeling, and metacognition: Making science accessible to all students. Cognition and Instruction, 16(1), 3–118.

Worsley, M., & Blikstein, P. (2015). Using learning analytics to study cognitive disequilibrium in a complex learning environment. Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 426–427). New York: ACM.

Xing, W., Popov, V., Zhu, G., Horwitz, P., & McIntyre, C. (2019). The effects of transformative and non-transformative discourse on individual performance in collaborative-inquiry learning. Computers in Human Behavior, 98, 267–276.

Zacharia, Z. C., Manoli, C., Xenofontos, N., de Jong, T., Pedaste, M., van Riesen, S. A. N., Kamp, E. T., Mäeots, M., Siiman, L., & Tsourlidaki, E. (2015). Identifying potential types of guidance for supporting student inquiry when using virtual and remote labs in science: A literature review. Educational Technology Research and Development, 63(2), 257–302.




How to Cite

Lämsä, J., Uribe, P., Jiménez, A., Caballero, D., Hämäläinen, R., & Araya, R. (2021). Deep Networks for Collaboration Analytics: Promoting Automatic Analysis of Face-to-Face Interaction in the Context of Inquiry-Based Learning. Journal of Learning Analytics, 8(1), 113-125.



Special Section: Collaboration Analytics