Collaboration Analytics Need More Comprehensive Models and Methods

An Opinion Paper




collaboration analytics, analytics framework, comprehensive perspective


As technology advances, learning analytics is expanding to include students’ collaboration settings. Despite their increasing application in practice, some types of analytics might not fully capture the comprehensive educational contexts in which students’ collaboration takes place (e.g., when data is collected and processed without predefined models, which forces users to make conclusions without sufficient contextual information). Furthermore, existing definitions and perspectives on collaboration analytics are incongruent. In light of these circumstances, this opinion paper takes a collaborative classroom setting as context and explores relevant comprehensive models for collaboration analytics. Specifically, this paper is based on Pei-Ling Tan and Koh’s ecological lens (2017, Situating learning analytics pedagogically: Towards an ecological lens. Learning: Research and Practice, 3(1), 1–11., which illustrates the co-emergence of three interactions among students, teachers, and content interwoven with time. Moreover, this paper suggests several factors to consider in each interaction when executing collaboration analytics. Agendas and recommendations for future research are also presented.


Avezado, R., & Cromley, J. G. (2004). Does training on self-regulated learning facilitate students’ learning with hypermedia? Journal of Educational Psychology, 96(3), 523–535.

Baker, R., Dee, T., Evans, B., & John, J. (2018, March). Bias in Online Classes: Evidence from a Field Experiment (Institute for Economic Policy Research Working paper No 18-055).

Bardram, J. E. (2005). Activity-based computing: Support for mobility and collaboration in ubiquitous computing. Personal and Ubiquitous Computing, 9, 312–322.

Bolsinova, M., de Boeck, P., & Tijmstra, J. (2017). Modelling conditional dependence between response time and accuracy. Psychometrika, 82, 1126–1148.

Bolsinova, M., & Molenaar, D. (2018). Modeling nonlinear conditional dependence between response time and accuracy. Frontiers in Psychology, 9(1), 1–12.

Bright, B. P. (1992). Reflection and knowledge: Aspects and problems. In P. Armstrong & N. Miller (Eds.), Knowledge and performance: The politics of adult/continuing education (pp. 42–47). Proceedings of the 21st Annual SCUTREA Conference.

Bronfenbrenner, U. (1979). The Ecology of Human Development: Experiments by Nature and Design. Cambridge, MA: Harvard University Press.

Brown, G., Bull, J., & Pendlebury, M. (1997). Assessing Student Learning in Higher Education. London: Routledge.

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42.

Buckingham Shum, S., & Ferguson, R. (2012). Social learning analytics. Educational Technology & Society, 15(3), 3–26.

Chen, B., Chang, Y-H., Ouyang, F., & Zhou, W. (2018). Fostering student engagement in online discussion through social learning analytics. The Internet and Higher Education, 37(1), 21–30.

Chiu, M. M., & Kuo, S. W. (2009). From metacognition to social metacognition: Similarities, differences, and learning. Journal of Education Research, 3(4), 1–19.

Choi, H., Dowell, N., Brooks, C., & Teasley, S. (2019). Social comparison in MOOCs: Perceived SES, opinion, and message formality. Proceedings of the Ninth International Conference on Learning Analytics and Knowledge (LAK ’19), 4–8 March 2019, Tempe, AZ, USA (pp. 160–169). New York: ACM.

Chounta, I.-A., & Avouris, N. (2016). Towards the real-time evaluation of collaborative activities: Integration of an automatic rater of collaboration quality in the classroom from the teacher’s perspective. Education and Information Technologies, 21, 815–835.

Clarke, A., & Erickson, G. (Eds.). (2003). Teacher Inquiry: Living the Research in Everyday Practice. London: RoutledgeFalmer.

Cloninger, C. R., Svrakic, D. M., & Przybeck, T. R. (1993). A psychobiological model of temperament and character. Archives of General Psychiatry, 50(12), 975–990.

Cukurova, M., Luckin, R., & Baines, E. (2018). The significance of context for the emergence and implementation of research evidence: The case of collaborative problem-solving. Oxford Review of Education, 44(3), 322–337.

Darling, N. (2007). Ecological systems theory: The person in the center of circles. Research in Human Development, 4(3-4), 203–217.

Dascalu, M., Trausan-Matu, S., McNamara, D. S., & Dessus, P. (2015). ReaderBench: Automated evaluation of collaboration based on cohesion and dialogism. International Journal of Computer-Supported Collaborative Learning, 10, 395–423.

De Boeck, P., & Scalise, K. (2019). Collaborative problem solving: Processing actions, time, and performance. Frontiers in Psychology, 10, 1-9.

De Laat, M., & Prinsen, F. R. (2014). Social learning analytics: Navigating the changing settings of higher education. Research & Practice in Assessment, 9, 51–60.

Dey, A. (2001). Understanding and using context. Personal and Ubiquitous Computing, 5, 4–7.

Dillenbourg, P., Järvelä, S., & Fischer, F. (2009). The evolution of research on computer-supported collaborative learning: From design to orchestration. In N. Balacheff, S. Ludvigsen, T. de Jong, A. Lazonder, & S. Barnes (Eds.), Technology-Enhanced Learning: Principles and Products (pp. 3–19). Amsterdam: Springer.

Dowell, N., Lin, Y., Godfrey, A., & Brooks, C. (2019). Promoting inclusivity through time-dynamic discourse analysis in digitally-mediated collaborative learning. In S. Isotani, E. Millán, A. Ogan, P. Hastings, B. McLaren, & R. Luckin (Eds.), Artificial Intelligence in Education (pp. 207–219). AIED 2019. Lecture Notes in Computer Science, vol. 11625. Cham: Springer International Publishing.

Dowell, N., Nixon, T. M., & Graesser, A. C. (2019). Group communication analysis: A computational linguistics approach for detecting sociocognitive roles in multiparty interactions. Behavior Research Methods, 51, 1007–1041.

Driskell, J. E., Salas, E., & Hughes, S. (2010). Collective orientation and team performance: Development of an individual differences measure. Human Factors, 52(2), 316–318.

Echeverria, V., Martinez-Maldonado, R., & Buckingham Shum, S. (2019). Towards collaboration translucence: Giving meaning to multimodal group data. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19), 4–9 May 2019, Glasgow, UK (pp. 1–16). New York: ACM.

Ferguson, R., & Buckingham Shum, S. (2012). Social learning analytics: Five approaches. Proceedings of the Second International Conference on Learning Analytics and Knowledge (LAK ’12), 29 April–2 May 2012, Vancouver, BC, Canada (pp. 23–33). New York: ACM.

Fiore, S. M., & Kapalo, K. A. (2017). Innovation in team interaction: New methods for assessing collaboration between brains and bodies using a multi-level framework. In A. A. von Davier, M. Zhu, & P. C. Kyllonen (Eds.), Innovative Assessment of Collaboration (pp. 51–64). Cham: Springer International Publishing.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34(10), 906–911.

Funke, J. (2010). Complex problem solving: A case for complex cognition?. Cognitive Processing, 11, 133–142.

Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68–84.

Gašević, D., Dawson, S., & Siemens. G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59, 64–71.

Gašević, D., Joksimović. S., Eagan, B. R., & Shaffer, D. W. (2019). SENS: Network analytics to combine social and cognitive perspectives of collaborative learning. Computers in Human Behavior, 92, 562–577.

Gillies, R. M., Ashman, A., & Terwel, J. (Eds.). (2008). The Teacher’s Role in Implementing Cooperative Learning in the Classroom. New York: Springer.

Goos, M., Galbraith, P., & Renshaw, P. (2002). Socially mediated metacognition: Creating collaborative zones of proximal development in small group problem solving. Educational Studies in Mathematics, 49, 193–223.

Graesser, A. C., Fiore, S. M., Greiff, S., Andrews-Todd, J., Foltz, P. W., & Hesse, F. W. (2018). Advancing the science of collaborative problem solving. Psychological Science in the Public Interest, 19(2), 59–92.

Haataja, E., Moreno-Esteva, E. G., Salonen, V., Laine, A., Toivanen, M., & Hannula, M. S. (2019). Teacher’s visual attention when scaffolding collaborative mathematical problem solving. Teaching and Teacher Education, 86, 1–15.

Hagemann, V., & Kluge, A. (2017). Complex problem solving in teams: The impact of collective orientation on team process demands. Frontiers in Psychology, 8, 1–17.

Heath, J. (2014). Contemporary privacy theory contributions to learning analytics. Journal of Learning Analytics, 1(1), 140–149.

Hernández-García, Á., Acquila-Natale, E., Chaparro-Peláez, J., & Conde, M. Á. (2018). Predicting teamwork group assessment using log data-based learning analytics. Computers in Human Behavior, 89, 373–384.

Hesse, F., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). A framework for teachable collaborative problem solving skills. In P. Griffin & E. Care (Eds.), Assessment and Teaching of 21st Century Skills: Methods and Approach (pp. 37–56). Dordrecht: Springer.

Isohätälä, J., Järvenoja, H., & Järvelä, S. (2017). Socially shared regulation of learning and participation in social interaction in collaborative learning. International Journal of Education Research, 81, 11–24.

Jang, H. W., & Park, S. W. (2016). Effects of personality traits on collaborative performance in problem-based learning tutorials. Saudi Medical Journal, 37(12), 1365–1371.

Järvelä, S., Malmberg, J., Haataja, E., Sobocinski, M., & Kirschner, P. (in press). What multimodal data can tell us about the students’ regulation of their learning process? Learning and Instruction.

Johnson, D. W., & Johnson. R. T. (2008). Social interdependence theory and cooperative learning: The teacher’s role. In R. M. Gillies, A. Ashman, & J. Terwel (Eds.), The Teacher’s Role in Implementing Cooperative Learning in the Classroom (pp. 9–37). New York: Springer.

Joksimović. S., Kovanović, V., & Dawson, S. (2019). The journey of learning analytics. HERDSA Review of Higher Education, 6, 37–63.

Kaendler, C., Wiedmann, M., Rummel, N., & Spada, H. (2015). Teacher competencies for the implementation of collaborative learning in the classroom: A framework and research review. Educational Psychology Review, 27, 505–536.

Kelly, J. R., & Barsade, S. G. (2001). Mood and emotions in small groups and work teams. Organizational Behavior and Human Decision Processes, 86(1), 99–130.

King, A. (2008). Structuring peer interaction to promote higher-order thinking and complex learning in cooperating groups. In R. M. Gillies, A. Ashman, & J. Terwel (Eds.), The Teacher’s Role in Implementing Cooperative Learning in the Classroom (pp. 73–91). New York: Springer.

Knight, S., Buckingham Shum, S., & Littleton, K. (2013). Epistemology, pedagogy, assessment and learning analytics. Proceedings of the Third International Conference on Learning Analytics and Knowledge (LAK ’13), 8–12 April 2013, Leuven, Belgium (pp. 75–84). New York: ACM.

Knox, J. (2017). Data power in education: Exploring critical awareness with the “learning analytics report card.” Television & New Media, 18(8), 734–752.

Knox, J., Williamson, B., & Bayne, S. (2020). Machine behaviourism: Future visions of “learnification” and “datafication” across humans and digital technologies. Learning, Media and Technology, 45(1), 31–45.

Kovanović, V., Joksimović. S., Gašević, D., Halata, M., & Siemens, G. (2017). Content analytics: The definition, scope, and an overview of published research. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), Handbook of Learning Analytics (1st ed., pp. 77–92). Society for Learning Analytics Research.

Kunter, M., Klusmann, U., Baumert, J., Richter, D., Voss, T., & Hachfeld, A. (2013). Professional competence of teachers: Effects on instructional quality and student development. Journal of Educational Psychology, 105(3), 805–820.

Le, H., Janssen, J., & Wubbels, T. (2018). Collaborative learning practices: Teacher and student perceived obstacles to effective student collaboration. Cambridge Journal of Education, 48(1), 103–122.

Leontyev, A. N. (1978). Activity, Consciousness, and Personality. Englewood Cliffs, NJ, USA: Prentice Hall.

Lin, Y., Dowell, N., Godfrey, A., Choi, H., & Brooks, C. (2019). Modeling gender dynamics in intra and interpersonal interactions during online collaborative learning. Proceedings of the Ninth International Conference on Learning Analytics and Knowledge (LAK ’19), 4–8 March 2019, Tempe, AZ, USA (pp. 431–435). New York: ACM.

Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439–1459.

Ludvigsen, S. (2016). CSCL: Connecting the social, emotional and cognitive dimensions. International Journal of Computer-Supported Collaborative Learning, 11, 115–121.

Malmberg, J., Järvelä, S., Holappa, J., Haataja, E., Huang, X., & Siipo, A. (2019). Going beyond what is visible: What multichannel data can reveal about interaction in the context of collaborative learning? Computers in Human Behavior, 96, 235–245.

Marks, M. A., Mathieu, J. E., & Zaccaro, S. J. (2001). A temporally based framework and taxonomy of team processes. The Academy of Management Review, 26(3), 356–376.

Martinez-Maldonado, R., Clayphan, A., & Kay, J. (2013). Towards the integration of collaboration analytics and interactive surfaces. Proceedings of the International Conference on Interactive Tabletops and Surfaces (ITS ’13), 6–9 October 2013, St. Andrews, UK (pp. 1–8). New York: ACM.

Martinez-Maldonado, R., Goodyear, P., Kay, J., Thompson, K., & Carvalho, L. (2016). An actionable approach to understand group experience in complex, multi-surface spaces. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16), 7–12 May 2016, San Jose, CA, USA (pp. 2062–2074). New York: ACM.

Martinez-Maldonado, R., Kay, J., Buckingham Shum, S., & Yacef, K. (2019). Collocated collaboration analytics: Principles and dilemmas for mining multimodal interaction data. Human-Computer Interaction, 34(1), 1–50.

McCoy, C., & Shih, P. (2016). Teachers as producers of data analytics: A case study of a teacher-focused educational data science program. Journal of Learning Analytics, 3(3), 193–214.

Morrison, R. L. (2009). Are women tending and befriending in the workplace? Gender differences in the relationship between workplace friendships and organizational outcomes. Sex Roles, 60(1-2), 1–13.

Ndukwe, I. G., & Daniel, B. K. (2020). Teaching analytics, value and tools for teacher data literacy: A systematic and tripartite approach. International Journal of Educational Technology in Higher Education, 17, 1–31.

Nguyen, Q., Rienties, B., & Toetenel, L. (2017). Mixing and matching learning design and learning analytics. In P. Zaphiris & A. Ioannou (Eds.), Learning and Collaboration Technologies. Technology in Education (pp. 302–316). Cham: Springer.

Nieswandt, M., McEneaney, E. H., & Affolter, R. (2020). A framework for exploring small group learning in high school science classrooms: The triple problem solving space. Instructional Science, 48, 243–290.

Nokes-Malach, T. J., Meade, M. L., & Morrow, D. G. (2012). The effect of expertise on collaborative problem solving. Thinking & Reasoning, 18(1), 32–58.

OECD (Organisation for Economic Co-operation and Development). (2017). PISA 2015 Collaborative Problem-Solving Framework.

OERservices. (n.d.). The Five Stages of Team Development.

Pei-Ling Tan, J., & Koh, E. (2017). Situating learning analytics pedagogically: Towards an ecological lens. Learning: Research and Practice, 3(1), 1–11.

Prieto, L. P., Sharma, K., Dillenbourg, P., & Jesús, M. (2016). Teaching analytics: Towards automatic extraction of orchestration graphs using wearable sensors. Proceedings of the Sixth International Conference on Learning Analytics and Knowledge (LAK ’16), 25–29 April 2016, Edinburgh, UK (pp. 148–157). New York: ACM.

Reilly, J. M., & Schneider, B. (2019). Predicting the quality of collaborative problem solving through linguistic analysis of discourse. Proceedings of the 12th International Conference on Educational Data Mining (EDM ’19), 2–5 July 2019, Montreal, QC, Canada (pp. 149–157).

Reimann, P. (2009). Time is precious: Variable- and event-centred approaches to process analysis in CSCL research. International Journal of Computer-Supported Collaborative Learning, 4, 239–257.

Rienties, B., & Toetenel, L. (2016). The impact of learning design on student behavior, satisfaction and performance: A cross-institutional comparison across 151 modules. Computers in Human Behavior, 60, 333–341.

Ron, S., Berka, C., & Sprang, M. (2009). Neurophysiologic collaboration patterns during team problem solving. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 53(12), 804–808.

Roschelle, J., Dimitriadis, Y., & Hoppe, U. (2013). Classroom orchestration: Synthesis. Computers & Education, 69, 523–526.

Ruparelia, N. B. (2010). Software development lifecycle models. ACM SIGSOFT Software Engineering Notes, 35(3), 8–13.

Santagata, R., Zannoni, C., & Stigler, J. W. (2007). The role of lesson analysis in pre-service teacher education: An empirical investigation of teacher learning from a virtual video-based field experience. Journal of Mathematics Teacher Education, 10, 123–140.

Scheffel, M., Tsai, Y-S., Gašević, D., & Drachsler, H. (2019). Policy matters: Expert recommendations for learning analytics policy. In M. Scheffel, J. Broisin, V. Pammer-Schindler, A. Ioannou, & J. Schneider (Eds.), Transforming Learning with Meaningful Technologies (pp. 510–524). Cham: Springer.

Schön, D. A. (1983). The Reflective Practitioner: How Professionals Think in Action. San Francisco: Jossey-Bass.

Scoular, C., & Care, E. (2020). Monitoring patterns of social and cognitive student behaviors in online collaborative problem solving assessments. Computers in Human Behavior, 104, 1–8.

Selwyn, N. (2019). What’s the problem with learning analytics? Journal of Learning Analytics, 6(3), 11–19.

Sergis, S., & Sampson, D. G. (2017). Teaching and learning analytics to teacher inquiry: A systematic literature review. In A. Peña-Ayala (Ed.), Learning Analytics: Fundaments, Applications, and Trends (pp. 25–63). Cham: Springer.

Shankar, S. K., Prieto, L. P., Rodríguez-Triana, M. J., & Ruiz-Calleja, A. (2018). A review of multimodal learning analytics architectures. Proceedings of the IEEE 18th International Conference on Advanced Learning Technologies (ICALT ’18), 9–13 July 2018, Mumbai, India (pp. 212–214). IEEE.

Siemens, G., & Gašević, D. (2012). Guest editorial — Learning and knowledge analytics. Educational Technology & Society, 15(3), 1–2.

Silverman, D. (2014). Interpreting Qualitative Data (5th ed.). London: Sage.

Smith, A. (30 August 2018). Franken-algorithms: The deadly consequences of unpredictable code. The Guardian. Retrieved from

Society for Learning Analytics Research. (n.d.). What Is Learning Analytics?

Stahl, G. (2011). Rediscovering CSCL. In T. Koschmann, R. Hall, & N. Miyaki (Eds.), CSCL2: Carrying forward the Conversation (pp. 169–184). Hillsdale, NJ: Lawrence Erlbaum Associates.

Stewart, A. E. B., Amon, M. J., Duran, N. D., & D’Mello, S. K. (2020). Beyond team makeup: Diversity in teams predicts valued outcomes in computer-mediated collaborations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20), 25–30 April 2020, Honolulu, HI, USA (pp. 1–13). New York: ACM.

Stewart, A. E. B., Vrzakova, H., Sun, C., Yonehiro, J., Stone, C. A., Duran, N. D., … D’Mello, S.K. (2019). I say, you say, we say: Using spoken language to model socio-cognitive processes during computer-supported collaborative problem solving. Proceedings of the ACM on Human-Computer Interaction, 3, 1–19.

Tsoni, R., & Verykios, V. S. (2019). Looking for the “more knowledgeable other” through learning analytics. Proceedings of the 10th International Conference in Open and Distance Learning (ICODL ’19), 22–24 November 2019, Athens, Greece (pp. 239–251).

Tuckman, B. W., & Jensen, M. A. C. (1977). Stages of small-group development revisited. Group & Organizational Studies, 2(4), 419–427.

Urhahne, D., Schanze, S., Bell, T., Mansfield, A., & Holmes, J. (2010). Role of the teacher in computer-supported collaborative inquiry learning. International Journal of Science Education, 32(2), 221–243.

Van Leeuwen, A., van Wermeskerken, M., Erkens, G., & Rummel, N. (2017). Measuring teacher sense making strategies of learning analytics: A case study. Learning: Research and Practice, 3(1), 42–58.

Verbert, K., Manouselis, N., Ochoa, X., Wolpers, M., Drachsler, H., Bosnic, I., & Duval, E. (2012). Context-aware recommender systems for learning: A survey and future challenges. IEEE Transactions on Learning Technologies, 5(4), 318–335.

Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. Cambridge, MA: Harvard University Press.

Webb, N. M. (2008). Teacher practices and small-group dynamics in cooperative learning classrooms. In R. M. Gillies, A. Ashman, & J. Terwel (Eds.), The Teacher’s Role in Implementing Cooperative Learning in the Classroom (pp. 201–221). New York: Springer.

Webber, S. S. (2008). Development of cognitive and affective trust in teams: A longitudinal study. Small Group Research, 39(6), 746–769.

Wiltshire, T. J., Steffensen, S. V., & Fiore, S. M. (2019). Multiscale movement coordination dynamics in collaboration team problem solving. Applied Ergonomics, 79, 143–151.

Winne, P. H., & Baker, R. S. J. D. (2013). The potentials of educational data mining for researching metacognition, motivation and self-regulated learning. Journal of Educational Data Mining, 5(1), 1–8.

Wise, A. F. (2014). Designing pedagogical interventions to support student use of learning analytics. Proceedings of the Fourth International Conference on Learning Analytics and Knowledge (LAK ’14), 24–28 March 2014, Indianapolis, IN, USA (pp. 203–211). New York: ACM.

Wise, A. F., & Jung, Y. (2019). Teaching with analytics: Towards a situated model of instructional decision-making. Journal of Learning Analytics, 6(2), 53–69.

Wise, A. F., & Shaffer, D. W. (2015). Why theory matters more than ever in the age of big data. Journal of Learning Analytics, 2(2), 5–13.

Yow, W. Q., & Lim, T. Z. M. (2019). Sharing the same languages helps us work better together. Palgrave Communications, 5, 1–11.

Zhang, X., Meng, Y., de Pablos, P. O., & Sun. Y. (2019). Learning analytics in collaborative learning supported by Slack: From the perspective of engagement. Computers in Human Behavior, 92, 625–633.

Zimmermann, A., Lorenz, A., & Oppermann, R. (2007). An operational definition of context. In B. Kokinov, D. C. Richardson, T. R. Roth-Berghofer, & L. Vieu (Eds.), CONTEXT 2007: Modeling and using context (pp. 558–571). Berlin, Heidelberg: Springer.




How to Cite

Han, A., Krieger, F., & Greiff, S. (2021). Collaboration Analytics Need More Comprehensive Models and Methods: An Opinion Paper. Journal of Learning Analytics, 8(1), 13-29.



Special Section: Collaboration Analytics