Designing Analytics for Collaboration Literacy and Student Empowerment
DOI:
https://doi.org/10.18608/jla.2021.7242Keywords:
collaboration, literacy, multimodal, analyticsAbstract
Collaboration has garnered global attention as an important skill for the 21st century. While researchers have been doing work on collaboration for nearly a century, many of the questions that the field is investigating overlook the need for students to learn how to read and respond to different collaborative settings. Existing research focuses on chronicling the various factors that predict the effectiveness of a collaborative experience, or on changing user behaviour in the moment. These are worthwhile research endeavours for developing our theoretical understanding of collaboration. However, there is also a need to centre student perceptions and experiences with collaboration as an important area of inquiry. Based on a survey of 131 university students, we find that student collaboration-related concerns can be represented across seven different categories or dimensions: Climate, Compatibility, Communication, Conflict, Context, Contribution, and Constructive. These categories extend prior research on collaboration and can help the field ensure that future collaboration analytics tools are designed to support the ways that students think about and utilize collaboration. Finally, we describe our instantiation of many of these dimensions in our collaborative analytics tool, BLINC, and suggest that these seven dimensions can be instructive for re-orienting the Multimodal Learning Analytics (MMLA) and collaboration analytics communities.
References
Alhadad, S. S. (2018). Visualizing data to support judgement, inference, and decision making in learning analytics: Insights from cognitive psychology and visualization science. Journal of Learning Analytics, 5(2), 60–85. https://doi.org/10.18608/jla.2018.52.5
Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences, 12(3), 307–359. https://doi.org/10.1207/S15327809JLS1203_1
Bassiou, N., Tsiartas, A., Smith, J., Bratt, H., Richey, C., Shriberg, E., & Alozie, N. (2016). Privacy-preserving speech analytics for automatic assessment of student collaboration. Proceedings of the 17th Annual Conference of the International Speech Communication Association (INTERSPEECH 2016), 8–12 September 2016, San Francisco, CA, USA (pp. 888–892). International Speech Communication Association. https://doi.org/10.21437/Interspeech.2016-1569
Berland, M., Davis, D., & Smith, C. P. (2015). AMOEBA: Designing for collaboration in computer science classrooms through live learning analytics. International Journal of Computer-Supported Collaborative Learning, 10(4), 425–447. https://doi.org/10.1007/s11412-015-9217-z
Binkley, M., Erstad, O., Herman, J., Raizen, S., Ripley, M., Miller-Ricci, M., & Rumble, M. (2012). Defining twenty-first century skills. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 17–66). Dordrecht: Springer Netherlands. https://doi.org/10.1007/978-94-007-2324-5_2
Blanchard, N., D’Mello, S., Olney, A. M., & Nystrand, M. (2015). Automatic classification of question & answer discourse segments from teacher’s speech in classrooms. In O. C. Santos et al. (Eds.), Proceedings of the 8th International Conference on Educational Data Mining (EDM2015), 26–29 June 2015, Madrid, Spain (pp. 282–288). International Educational Data Mining Society. Retrieved from https://files.eric.ed.gov/fulltext/ED560555.pdf
Boaler, J., Wiliam, D., & Brown, M. (2000). Students’ experiences of ability grouping-disaffection, polarisation and the construction of failure. British Educational Research Journal, 26(5), 631–648.
Borge, M., & Rosé, C. P. (2016). Automated feedback on group processes: An experience report. In T. Barnes et al. (Eds.), Proceedings of the 9th International Conference on Educational Data Mining (EDM2016), 29 June–2 July 2016, Raleigh, NC, USA (pp. 573–574). International Educational Data Mining Society.
Bos, M. C. (1937). Experimental study of productive collaboration. Acta Psychologica, 3, 315–426. https://doi.org/10.1016/S0001-6918(01)90007-1
Carbone, E. (1999). Students behaving badly in large classes. New Directions for Teaching and Learning, 1999(77), 35–43.
Cukurova, M., Luckin, R., Mavrikis, M., & Millan, E. (2017). Machine and human observable differences in groups collaborative problem-solving behaviours. Proceedings of the 12th European Conference on Technology Enhanced Learning (EC-TEL 2017), 12–15 September 2017, Tallinn, Estonia (pp. 17–29). Lecture Notes in Computer Science, Springer.
Cukurova, M., Luckin, R., Millan, E., & Mavrikis, M. (2018). The NISPI framework: Analysing collaborative problem-solving from students’ physical interactions. Computers & Education, 116, 93–109. https://doi.org/10.1016/j.compedu.2017.08.007
Cukurova, M., Zhou, Q., Spikol, D., & Landolfi, L. (2020). Modelling collaborative problem-solving competence with transparent learning analytics: Is video data enough? Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 270–275). New York: ACM. https://doi.org/10.1145/3375462.3375484
D’Angelo, C., Smith, J., Alozie, N., Tsiartas, A., Richey, C., & Bratt, H. (2019). Mapping individual to group level collaboration indicators using speech data. In K. Lund, G. Niccolai, E. Lavoué, C. Hmelo-Silver, G. Gweon, & M. Baker (Eds.), A Wide Lens: Combining Embodied, Enactive, Extended, and Embedded Learning in Collaborative Settings. Proceedings of the 13th International Conference on Computer Supported Collaborative Learning (CSCL 2019), 17–21 June 2019, Lyon, France (Vol. 2, pp. 628–631). International Society of the Learning Sciences.
Dillenbourg, P., & Bachour, K. (2010). An interactive table for supporting participation balance in face-to-face collaborative learning. IEEE Transactions on Learning Technologies, 3(3), 203–213. https://doi.org/10.1109/TLT.2010.18
Dillenbourg, P., Nussbaum, M., Dimitriadis, Y., & Roschelle, J. (2013). Design for classroom orchestration. Computers & Education, 69, 485–492.
Dillenbourg, P., Zufferey, G., Alavi, H., Jermann, P., Do-lenh, S., & Bonnard, Q. (2011). Classroom orchestration: The third circle of usability. Proceedings of the 9th International Conference on Computer-Supported Collaborative Learning (CSCL 2011), 4–8 July 2011, Hong Kong, China (Vol. 1, pp. 510–517). International Society of the Learning Sciences.
Domínguez, F., Chiluiza, K., Echeverria, V., & Ochoa, X. (2015). Multimodal selfies: Designing a multimodal recording device for students in traditional classrooms. Proceedings of the 17th ACM International Conference on Multimodal Interaction (ICMI ’15), 9–13 November 2015, Seattle, WA, USA (pp. 567–574). New York: ACM.
Echeverria, V., Martinez-Maldonado, R., Shum, S. B., Chiluiza, K., Granda, R., & Conati, C. (2018). Exploratory versus explanatory visual learning analytics: Driving teachers’ attention through educational data storytelling. Journal of Learning Analytics, 5(3), 72–97. https://doi.org/10.18608/jla.2018.53.6
Echeverria, V., Martinez-Maldonado, R., Shum, S. B., & Sydney, T. (2019). Towards collaboration translucence: Giving meaning to multimodal group data. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19), 4–9 May 2019, Glasgow, Scotland, UK (Paper No. 39, pp. 1–16). New York: ACM.
Edmondson, A. C., & Lei, Z. (2014). Psychological safety: The history, renaissance, and future of an interpersonal construct. Annual Review of Organizational Psychology and Organizational Behavior, 1(1), 23–43.
Furuichi, K., & Worsley, M. (2018). Using physiological responses to capture unique idea creation in team collaborations. Companion of the 2018 ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW ’18) 3–7 November 2018, Jersey City, NJ, USA (pp. 369–372). New York: ACM.
Gero, J., & Kannengiesser, U. (2004, 07). The situated function-behaviour-structure framework. Design Studies, 25(4), 373–391. https://doi.org/10.1016/j.destud.2003.10.010
Gibbs, G., Jenkins, A., & Alan, J. (Eds.). (1992). Teaching large classes in higher education: How to maintain quality with reduced resources. Hove, East Sussex, UK: Psychology Press.
Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory. International Journal of Qualitative Methods, 5, 1–10. Retrieved from http://www.sxf.uevora.pt/wp-content/uploads/2013/03/Glaser_1967.pdf
Grover, S., Bienkowski, M., Tamrakar, A., Siddiquie, B., Salter, D., & Divakaran, A. (2016). Multimodal analytics to study collaborative problem solving in pair programming. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 516–517). New York: ACM. https://doi.org/10.1145/2883851.2883877
Hmelo-Silver, C. E., & Barrows, H. S. (2008). Facilitating collaborative knowledge building. Cognition and Instruction, 26(1), 48–94. https://doi.org/10.1080/07370000701798495
Jung, Y. J., Toprani, D., Yan, S., & Borge, M. (2017). Children’s participation in rulemaking to mitigate process problems in CSCL. In B. K. Smith, M. Borge, E. Mercier, & K. Y. Lim (Eds.), Making a Difference: Prioritizing Equity and Access in CSCL. Proceedings of the 12th International Conference on Computer Supported Collaborative Learning (CSCL 2017) 18–22 June 2017, Philadelphia, PA, USA (Vol. 2, pp. 652–655). International Society of the Learning Sciences.
Marsh, A. A., Elfenbein, H. A., & Ambady, N. (2003). Nonverbal “accents”: Cultural differences in facial expressions of emotion. Psychological Science, 14(4), 373–376.
Martinez-Maldonado, R., Kay, J., Buckingham Shum, S., & Yacef, K. (2019). Collocated collaboration analytics: Principles and dilemmas for mining multimodal interaction data. Human–Computer Interaction, 34(1), 1–50.
Martinez-Maldonado, R., Power, T., Hayes, C., Abdiprano, A., Vo, T., Axisa, C., & Buckingham Shum, S. (2017). Analytics meet patient manikins: Challenges in an authentic small-group healthcare simulation classroom. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 90–94). New York: ACM. http://doi.acm.org/10.1145/3027385
Mayo, E. (1939). Routine interaction and the problem of collaboration. American Sociological Review, 4(3), 335–340. https://doi.org/10.2307/2084920
Ochoa, X., Chiluiza, K., Mendez, G., Luzardo, G., Guamán, B., & Castells, J. (2013). Expertise estimation based on simple multimodal features. Proceedings of the 15th ACM International Conference on Multimodal Interaction (ICMI ’13), 9–13 December 2013, Sydney, Australia (pp. 583–590). New York: ACM. https://doi.org/10.1145/2522848.2533789
Oviatt, S., & Cohen, A. (2014). Written activity, representations and fluency as predictors of domain expertise in mathematics. Proceedings of the 16th ACM International Conference on Multimodal Interaction (ICMI ’14), 12–16 November 2014, Istanbul, Turkey (pp. 10–17). New York: ACM. https://doi.org/10.1145/2663204.2663245
Oviatt, S., Cohen, A., & Weibel, N. (2013). Multimodal learning analytics: Description of math data corpus for ICMI grand challenge workshop. Proceedings of the 15th ACM International Conference on Multimodal Interaction (ICMI ’13), 9–13 December 2013, Sydney, Australia (pp. 563–568). New York: ACM. https://doi.org/10.1145/2522848.2533790
Praharaj, S., Scheffel, M., Drachsler, H., & Specht, M. (2018). Multimodal analytics for real-time feedback in co-located collaboration. Proceedings of the 13th European Conference on Technology Enhanced Learning (EC-TEL 2018), 3–5 September 2018, Leeds, UK (pp. 187–201). Lecture Notes in Computer Science, Springer.
Reimers, G., & Neovesky, A. (2015). Student focused dashboards. In S. Zvacek, M. T. Restivo, J. Uhomoibhi, M. Helfert (Eds.), Proceedings of the 7th International Conference on Computer Supported Education (CSEDU 2015), 23–25 May 2015, Lisbon, Portugal (Vol. 1, pp. 399–404). Springer.
Roschelle, J. (1999). Learning by collaborating: Convergent conceptual change. The Journal of the Learning Sciences, 2(3), 235–276.
Schneider, B., & Blikstein, P. (2015). Unraveling students’ interaction around a tangible interface using multimodal learning analytics. Journal of Educational Data Mining, 7(3), 89–116.
Schneider, B., & Pea, R. (2015). Does seeing one another’s gaze affect group dialogue? A computational approach. Journal of Learning Analytics, 2(2), 107–133. https://doi.org/10.18608/jla.2015.22.9
Schneider, B., Sharma, K., Cuendet, S., Zufferey, G., Dillenbourg, P., & Pea, R. (2018). Leveraging mobile eye-trackers to capture joint visual attention in co-located collaborative learning groups. International Journal of Computer-Supported Collaborative Learning, 13(3), 241–261.
Spikol, D., Cukurova, M., & Ruffaldi, E. (2017). Using multimodal learning analytics to identify aspects of collaboration in project-based learning. In B. K. Smith, M. Borge, E. Mercier, & K. Y. Lim (Eds.), Making a Difference: Prioritizing Equity and Access in CSCL. Proceedings of the 12th International Conference on Computer Supported Collaborative Learning (CSCL 2017) 18–22 June 2017, Philadelphia, PA, USA (Vol. 1, pp. 263–270). International Society of the Learning Sciences. https://doi.org/10.22318/cscl2017.37
Spikol, D., Avramides, K., & Cukurova, M. (2016). Exploring the interplay between human and machine annotated multimodal learning analytics in hands-on STEM activities. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 522–523). New York: ACM. https://doi.org/10.1145/2883851.2883920
Spikol, D., Superiore, S., Anna, S., & Landolfi, L. (2020). Modelling collaborative problem-solving competence with transparent learning analytics: Is video data enough? Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 270–275). New York: ACM. https://doi.org/10.1145/3375462.3375484
Stone, P. J., Bales, R. F., Namenwirth, J. Z., & Ogilvie, D. M. (1962). The general inquirer: A computer system for content analysis and retrieval based on the sentence as a unit of information. Systems Research and Behavioral Science, 7(4), 484–498.
Tausczik, Y. R., & Pennebaker, J. W. (2010). The psychological meaning of words: LIWC and computerized text analysis methods. Journal of Language and Social Psychology, 29(1), 24–54.
Teasley, S. D. (2017). Student facing dashboards: One size fits all? Technology, Knowledge and Learning, 22(3), 377–384.
Terken, J., & Sturm, J. (2010). Multimodal support for social dynamics in co-located meetings. Personal and Ubiquitous Computing, 14(8), 703–714. https://doi.org/10.1007/s00779-010-0284 -x
Tse, E., Greenberg, S., Shen, C., & Forlines, C. (2007). Multimodal multiplayer tabletop gaming. Computers in Entertainment, 5(2), 12. https://doi.org/10.1145/1279540.1279552
Van Leeuwen, A. V., Janssen, J., Erkens, G., & Brekelmans, M. (2015). Teacher regulation of cognitive activities during student collaboration: Effects of learning analytics. Computers & Education, 90, 80–94. http://doi.org/10.1016/j.compedu.2015.09.006
Vrzakova, H., Amon, M. J., Stewart, A., Duran, N. D., & Mello, S. K. D. (2020). Focused or stuck together: Multimodal patterns reveal triads’ performance in collaborative problem solving. Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 295–304). New York: ACM. https://doi.org/10.1145/3375462.3375467
Williams, R. (2015). The non-designer’s design book: Design and typographic principles for the visual novice. London: Pearson Education.
Wise, A. F., & Chiu, M. M. (2011). Analyzing temporal patterns of knowledge construction in a role-based online discussion. International Journal of Computer-Supported Collaborative Learning, 6(3), 445–470. https://doi.org/10.1007/s11412-011-9120-1
Worsley, M., & Blikstein, P. (2015). Leveraging multimodal learning analytics to differentiate student learning strategies. Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 360–367). New York: ACM. https://doi.org/10.1145/2723576.2723624
Worsley, M., & Blikstein, P. (2018). A multimodal analysis of making. International Journal of Artificial Intelligence in Education, 28(3), 385–419.
Worsley, M., & Blikstein, P. (2011). What’s an expert? Using learning analytics to identify emergent markers of expertise through automated speech, sentiment and sketch analysis. In M. Pechenizkiy et al. (Eds.), Proceedings of the 4th Annual Conference on Educational Data Mining (EDM2011), 6–8 July 2011, Eindhoven, Netherlands (pp. 235–240). International Educational Data Mining Society.
Worsley, M., & Ochoa, X. (2020). Towards collaboration literacy development through multimodal learning analytics. In M. Giannakos, D. Spikol, I. Molenaar, D. Di Mitri, K. Sharma, X. Ochoa, & R. Hammad (Eds.), Proceedings of CrossMMLA in Practice: Collecting, Annotating and Analyzing Multimodal Data Across Spaces. Co-located with the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), 24 March 2020, Cyberspace (Vol. 2610, pp. 53–63). New York: ACM. http://ceur-ws.org/Vol-2610/
Zhou, J., Hang, K., Oviatt, S., Yu, K., & Chen, F. (2014). Combining empirical and machine learning techniques to predict math expertise using pen signal features. Proceedings of the 2014 ACM Workshop on Multimodal Learning Analytics Workshop and Grand Challenge (MLA ’14), 12–16 November 2014, Istanbul, Turkey (pp. 29–36). New York: ACM. https://doi.org/10.1145/2666633.2666638
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) license that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
