Measuring Dynamical Interdependence in Small-Group Collaborations
DOI:
https://doi.org/10.18608/jla.2021.7241Keywords:
collaboration, interdependence, process data, temporal analytics, psychometricsAbstract
This paper addresses dynamical interdependence among the actions of group members. I assume that the actions of each member can be represented as nodes of a dynamical network and then collect the nodes into disjoint subsets (components) representing the individual group members. Interdependence among group members’ actions can then be defined with reference to a K-partite network, in which the partitions correspond to the group member components. Independence among group members’ actions can be defined with reference to a network in which the group member components are disconnected from one another. The degree to which the interactions of actual groups correspond to either of these theoretical network structures can be characterized using modified versions of existing network statistics. Taking this approach, I propose a number of network-based measures of dynamical interdependence, discuss the interpretation of the proposed measures, and consider how to assess their reliability and validity. These ideas are illustrated using an example in which dyads collaborated via online chat to complete a grade 12 level mathematics assessment.
References
Adams, R., Vista, A., Scoular, C., Awwal, N., Griffin, P., & Care, E. (2015). Automatic coding procedures for collaborative problem solving. In P. Griffin & E. Care (Eds.),Assessment and Teaching of 21st Century Skills(pp. 115–132). Dordrecht: Springer Netherlands. https://doi.org/10.1007/978-94-017-9395-76
Andrews-Todd, J., & Forsyth, C. M. (2020). Exploring social and cognitive dimensions of collaborative problem solving in anopen online simulation-based task. Computers in Human Behavior,104, 105759. https://doi.org/10.1016/j.chb.2018.10.025
Attali, Y., & Burstein, J. (2014). Automated essay scoring with e-rater®V.2.The Journal of Technology, Learning andAssessment,4(2), i–21. https://doi.org/10.1002/j.2333-8504.2004.tb01972.x
Barrat, A., Barthelemy, M., & Vespignani, A. (2011). Dynamical Processes on Complex Networks. Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/CBO9780511791383
Butts, C. T. (2008).A relational event framework for social action. Sociological Methodology,38(1), 155–200. https://doi.org/10.1111/j.1467-9531.2008.00203.x
Chen, B., Knight, S., & Wise, A. F. (2018). Critical issues in designing and implementing temporal analytics. Journal of Learning Analytics,5(1), 1–9. https://doi.org/10.18608/jla.2018.53.1
Chen, S., Lawrence, J. F., Zhou, J., Min, L., & Snow, C. E. (2018). The efficacy of a school-based book-reading intervention on vocabulary development of young Uyghur children: A randomized controlled trial. Early Childhood Research Quarterly,44, 206–219. https://doi.org/10.1016/j.ecresq.2017.12.008
Chi, M. T. H., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist,49(4), 219–243. https://doi.org/10.1080/00461520.2014.965823
Cohen, E. G. (1994). Restructuring the classroom: Conditions for productive small groups. Review of Educational Research,64(1), 1–35. https://doi.org/10.3102/00346543064001001
Daley, D. J., & Vere-Jones, D. (2003). An Introduction to the Theory of Point Processes(2nd ed.). New York: Springer. https://doi.org/10.1007/b97277
Davis, J. H. (1992). Some compelling intuitions about group consensus decisions, theoretical and empirical research, and interpersonal aggregation phenomena: Selected examples, 1950–1990.Organizational Behavior and Human DecisionProcesses,52(1), 3–38. https://doi.org/10.1016/0749-5978(92)90044-8
Dowell, N. M. M., Nixon, T. M., & Graesser, A. C. (2019). Group communication analysis: A computational linguistics approach for detecting sociocognitive roles in multiparty interactions. Behavior Research Methods,51(3), 1007–1041. https://doi.org/10.3758/s13428-018-1102-z
Durbin, J., & Koopman, S. J. (2012).Time Series Analysis by State Space Methods. Oxford, UK: Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199641178.001.0001
Griffin, P., & Care, E. (2015). Assessment and Teaching of 21st Century Skills: Methods and Approach. New York: Springer. https://doi.org/10.1007/978-94-007-9395-7
Halpin, P. F., & Bergner, Y. (2018). Psychometric models of small group collaborations. Psychometrika, 83, 941–962. https://doi.org/10.1007/s11336-018-9631-z
Halpin, P. F., von Davier, A. A., Hao, J., & Liu, L. (2017). Measuring student engagement during collaboration. Journal of Educational Measurement,54(1), 70–84. https://doi.org/10.1111/jedm.12133
Hanneman, R. A., & Riddle, M. (2014). Concepts and measures for basic network analysis. In The Sage Handbook of Social Network Analysis (pp. 340–369). London, UK: SAGE Publications Ltd. https://doi.org/10.4135/9781446294413.n24
Hao, J., Chen, L., Flor, M., Liu, L., & von Davier, A. A. (2017). CPS-rater: Automated sequential annotation for conversations in collaborative problem-solving activities.ETS Research Report Series,2017(1), 1–9. https://doi.org/10.1002/ets2.12184
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning(2nd ed.). New York: Springer. https://doi.org/10.1007/978-0-387-84858-7
Johnson, D. W., & Johnson, R. T. (2009). An educational psychology success story: Social interdependence theory and cooperative learning. Educational Researcher,38(5), 365–379. https://doi.org/10.3102/0013189X09339057
Joint Committee for Educational and Psychological Testing. (2014). Standards for Educational and Psychological Testing. Washington, DC: AERA APA, & NCME. Jost, J. (2007). Dynamical networks. In J. Feng, J. Jost, & M. Qian (Eds.), Networks: From Biology to Theory (pp. 35–62).London, UK: Springer. https://doi.org/10.1007/978-1-84628-780-03
Knight, S., Friend Wise, A., & Chen, B. (2017). Time for change: Why learning analytics needs temporal analysis. Journal of Learning Analytics,4(3), 7–17. https://doi.org/10.18608/jla.2017.43.2
Kozlowski, S. W. J., & Chao, G. T. (2018). Unpacking team process dynamics and emergent phenomena: Challenges, conceptual advances, and innovative methods. American Psychologist,73(4), 576–592. https://doi.org/10.1037/amp0000245
Laughlin, P. R. (2013). Group Problem Solving. Princeton, NJ, USA: Princeton University Press. Retrieved from https://www.jstor.org/stable/j.ctt7rmfz
Leenders, R. T. A. J., Contractor, N. S., & DeChurch, L. A. (2016). Once upon a time: Understanding team processes as relational event networks. Organizational Psychology Review, 6(1), 92–115. https://doi.org/10.1177/2041386615578312
Liu, L., Hao, J., von Davier, A. A., Kyllonen, P., & Zapata-Rivera, D. (2015). A tough nut to crack: Measuring collaborative problem solving. In Y. Rosen, S. Ferrara, & M. Mosharraf (Eds.), Handbook of Research on Computational Tools for Real-World Skill Development (pp. 344–359). Hershey, PA, USA: IGI-Global. https://doi.org/10.1136/bmj.330.7485.0-h
Lord, F. M. (1980). Applications of Item Response Theory to Practical Testing Problems. New York: Routledge.
Lorge, I., & Solomon, H. (1955). Two models of group behavior in the solution of eureka-type problems. Psychometrika, 20(2), 139–148. https://doi.org/10.1007/BF02288986
Lutkepohl, H. (2005). New Introduction to Multiple Time Series Analysis. Berlin, New York: Springer. https://doi.org/10.1007/978-3-540-27752-1
Madhyastha, T. M., Hamaker, E. L., & Gottman, J. M. (2011). Investigating spousal influence using moment-to-moment affect data from marital conflict. Journal of Family Psychology, 25(2), 292–300. https://doi.org/10.1037/a0023028
Mathieu, J. E., Tannenbaum, S. I., Donsbach, J. S., & Alliger, G. M. (2014). A review and integration of team composition models: Moving toward a dynamic and temporal framework. Journal of Management, 40(1), 130–160. https://doi.org/10.1177/0149206313503014
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019). A survey on bias and fairness in machine learning. (arXiv:1908.09635)
Mohan, K., Bergner, Y., & Halpin, P. (2020). Predicting group performance using process data in a collaborative assessment. Technology, Knowledge and Learning, 25, 367–388. https://doi.org/10.1007/s10758-020-09439-5
Molenaar, P. C. M. (1985). A dynamic factor model for the analysis of multivariate time series. Psychometrika, 50(2), 181–202. https://doi.org/10.1007/BF02294246
Oehlert, G. W. (1992). A note on the delta method. The American Statistician, 46(1), 27–29. https://doi.org/10.1080/00031305.1992.10475842
Oviatt, S., Schuller, B., Cohen, P., Sonntag, D., Potamianos, G., & Kr¨uger, A. (2018). The Handbook of Multimodal-Multisensor Interfaces: Signal Processing, Architectures, and Detection of Emotion and Cognition (Vol. 2). Morgan & Claypool. https://doi.org/10.1145/3107990
Quera, V. (2018). Analysis of interaction sequences. In E. Brauner, M. Boos, & M. Kolbe (Eds.), The Cambridge Handbook of Group Interaction Analysis (1st ed., pp. 295–322). Cambridge University Press. https://doi.org/10.1017/9781316286302.016
Rack, O., Zahn, C., & Mateescu, M. (2018). Coding and counting: Frequency analysis for group interaction research. In E. Brauner, M. Boos, & M. Kolbe (Eds.), The Cambridge Handbook of Group Interaction Analysis (1st ed., pp. 277–294). Cambridge University Press. https://doi.org/10.1017/9781316286302.015
Spedicato, G. A. (2017). Discrete time Markov chains with R. The R Journal, 9(2), 84. https://doi.org/10.32614/RJ-2017-036
Suthers, D. D., Dwyer, N., Medina, R., & Vatrapu, R. (2010). A framework for conceptualizing, representing, and analyzing distributed interaction. International Journal of Computer-Supported Collaborative Learning, 5(1), 5–42. https://doi.org/10.1007/s11412-009-9081-9
Swiecki, Z., Ruis, A. R., Farrell, C., & Shaffer, D. W. (2019). Assessing individual contributions to collaborative problem solving: A network analysis approach. Computers in Human Behavior, 104, 105876. https://doi.org/10.1016/j.chb.2019.01.009
Teasley, S. D. (1997). Talking about reasoning: How important is the peer in peer collaboration? In L. B. Resnick, R. Saljo, C. Pontecorvo, & B. Burge (Eds.), Discourse, Tools and Reasoning: Essays on Situated Cognition (pp. 361–384). Berlin, Heidelberg: Springer. https://doi.org/10.1007/978-3-662-03362-3 16
Thissen, D., & Wainer, H. (2001). Test Scoring. Mahwah, NJ, USA: Lawrence Earlbaum Associates.
Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N., & Malone, T. W. (2010). Evidence for a collective intelligence factor in the performance of human groups. Science, 330(6004), 686–688. https://doi.org/10.1126/science.1193147
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) license that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
