The Curious Case of Centrality Measures: A Large-Scale Empirical Investigation




learning analytics, meta-analysis, reproducibility, replicability, social network analysis, centrality measures, research paper


There has been extensive research using centrality measures in educational settings. One of the most common lines of such research has tested network centrality measures as indicators of success. The increasing interest in centrality measures has been kindled by the proliferation of learning analytics. Previous works have been dominated by single-course case studies that have yielded inconclusive results regarding the consistency and suitability of centrality measures as indicators of academic achievement. Therefore, large-scale studies are needed to overcome the multiple limitations of existing research (limited datasets, selective and reporting bias, as well as limited statistical power). This study aims to empirically test and verify the role of centrality measures as indicators of success in collaborative learning. For this purpose, we attempted to reproduce the most commonly used centrality measures in the literature in all the courses of an institution over five years of education. The study included a large dataset (n=3,277) consisting of 69 course offerings, with similar pedagogical underpinnings, using meta-analysis as a method to pool the results of different courses. Our results show that degree and eigenvector centrality measures can be a consistent indicator of performance in collaborative settings. Betweenness and closeness centralities yielded uncertain predictive intervals and were less likely to replicate. Our results have shown moderate levels of heterogeneity, indicating some diversity of the results comparable to single laboratory replication studies.


Aarts, A., Anderson, J., Anderson, C., Attridge, P., Attwood, A., Axt, J., Babel, M., Bahník, Š., Baranski, E., Barnett-Cowan, M., Bartmess, E., Beer, J., Bell, R., Bentley, H., Beyan, L., Binion, G., Borsboom, D., Bosch, A., Bosco, F., & Penuliar, M. (2015). Estimating the reproducibility of psychological science. Science, 349(6251).

Agudo-Peregrina, Á. F., Iglesias-Pradas, S., Conde-González, M. Á., & Hernández-García, Á. (2014). Can we predict success from log data in VLEs? Classification of interactions for learning analytics and their relation with performance in VLE-supported F2F and online learning. Computers in Human Behavior, 31(1), 542–550.

Alexander, J. L., Smith, K. A., Mataras, T., Shepley, S. B., & Ayres, K. M. (2015). A meta-analysis and systematic review of the literature to evaluate potential threats to internal validity in probe procedures for chained tasks. The Journal of Special Education, 49(3), 135–145.

Andres, J. M. L., Baker, R. S., Gašević, D., Siemens, G., Crossley, S. A., & Joksimović, S. (2018). Studying MOOC completion at scale using the MOOC replication framework. Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, NSW, Australia (pp. 71–78). ACM.

Andres, J. M. L., Baker, R. S., Siemens, G., Gašević, D., & Spann, C. A. (2015). Replicating 21 findings on student success in online learning. Technology, Instruction, Cognition, and Learning, 10, 313–333.

Baker, M., & Penny, D. (2016). Is there a reproducibility crisis? Nature, 533(7604), 452–454.

Baldwin, T. T., Bedell, M. D., & Johnson, J. L. (1997). The social fabric of a team-based M.B.A. program: Network effects on student satisfaction and performance. Academy of Management Journal, 40(6), 1369–1397.

Bernard, R. M., Abrami, P. C., Borokhovski, E., Wade, C. A., Tamim, R. M., Surkes, M. A., & Bethel, E. C. (2009). A meta-analysis of three types of interaction treatments in distance education. Review of Educational Research, 79(3), 1243–1289.

Borgatti, S. P. (2005). Centrality and network flow. Social Networks, 27(1), 55–71.

Borgatti, S. P., & Brass, D. J. (2019). Centrality: Concepts and measures. In D. J. Brass & S. P. Borgatti (Eds.), Social Networks at Work (pp. 9–22). Routledge.

Borgatti, S. P., & Everett, M. G. (2006). A graph-theoretic perspective on centrality. Social Networks, 28(4), 466–484.

Borgatti, S. P., Mehra, A., Brass, D. J., & Labianca, G. (2009). Network analysis in the social sciences. Science, 323(5916), 892–895.

Borokhovski, E., Bernard, R. M., Tamim, R. M., Schmid, R. F., & Sokolovskaya, A. (2016). Technology-supported student interaction in post-secondary education: A meta-analysis of designed versus contextual treatments. Computers & Education, 96, 15–28.

Boroujeni, M. S., Hoppe, H. U., Hecking, T., & Dillenbourg, P. (2017). Dynamics of MOOC discussion forums. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 128–137). ACM.

Cadima, R., Ojeda, J., & Monguet, J. M. (2012). Social networks and performance in distributed learning communities. Educational Technology & Society, 15(4), 296–304.

Cela, K. L., Sicilia, M. Á., & Sánchez, S. (2015). Social network analysis in e-learning environments: A preliminary systematic review. Educational Psychology Review, 27(1), 219–246.

Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004). Empirical evidence for selective reporting of outcomes in randomized trials. JAMA, 291(20), 2457.

Chen, J., Wang, M., Kirschner, P. A., & Tsai, C.-C. (2018). The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis. Review of Educational Research, 88(6), 799–843.

Cho, H., Gay, G., Davidson, B., & Ingraffea, A. (2007). Social networks, communication styles, and learning performance in a CSCL community. Computers & Education, 49(2), 309–329.

Conijn, R., Snijders, C., Kleingeld, A., & Matzat, U. (2017). Predicting student performance from LMS data: A comparison of 17 blended courses using Moodle LMS. IEEE Transactions on Learning Technologies, 10(1), 17–29.

Csardi, G., & Nepusz, T. (2006). The Igraph software package for complex network research. InterJournal, Complex Sy, 1695.

Dado, M., & Bodemer, D. (2017). A review of methodological applications of social network analysis in computer-supported collaborative learning. Educational Research Review, 22, 159–180.

Davidson, N., & Major, C. H. (2014). Boundary crossings: Cooperative learning, collaborative learning, and problem-based learning. Journal on Excellence in College Teaching, 25(3&4), 7–55.

Dawson, S., Joksimovic, S., Poquet, O., & Siemens, G. (2019). Increasing the impact of learning analytics. Proceedings of the 9th International Conference on Learning Analytics and Knowledge (LAK ’19), 4–8 March 2019, Tempe, AZ, USA (pp. 446–455). ACM.

de-Marcos, L., García-López, E., García-Cabot, A., Medina-Merodio, J.-A., Domínguez, A., Martínez-Herráiz, J.-J., & Diez-Folledo, T. (2016). Social network analysis of a gamified e-learning course: Small-world phenomenon and network metrics as predictors of academic performance. Computers in Human Behavior, 60, 312–321.

Dwan, K., Gamble, C., Williamson, P. R., & Kirkham, J. J. (2013). Systematic review of the empirical evidence of study publication bias and outcome reporting bias: An updated review. PLOS ONE, 8(7).

Enyedy, N., & Stevens, R. (2016). Analyzing collaboration. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 191–212). Cambridge University Press.

Erkens, M., Bodemer, D., & Hoppe, H. U. (2016). Improving collaborative learning in the classroom: Text mining based grouping and representing. International Journal of Computer-Supported Collaborative Learning, 11(4), 387–415.

Fincham, E., Gašević, D., & Pardo, A. (2018). From social ties to network processes: Do tie definitions matter? Journal of Learning Analytics, 5(2), 9–28.

Freeman, L. C. (1978). Centrality in social networks conceptual clarification. Social Networks, 1(3), 215–239.

Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68–84.

Gašević, D., Joksimović, S., Eagan, B. R., & Shaffer, D. W. (2019). SENS: Network analytics to combine social and cognitive perspectives of collaborative learning. Computers in Human Behavior, 92, 562–577.

Gašević, D., Zouaq, A., & Janzen, R. (2013). “Choose your classmates, your GPA is at stake!” American Behavioral Scientist, 57(10), 1460–1479.

Hagger, M. S., Chatzisarantis, N. L. D., Alberts, H., Anggono, C. O., Batailler, C., Birt, A. R., Brand, R., Brandt, M. J., Brewer, G., Bruyneel, S., Calvillo, D. P., Campbell, W. K., Cannon, P. R., Carlucci, M., Carruth, N. P., Cheung, T., Crowell, A., De Ridder, D. T. D., Dewitte, S., . . . Zwienenberg, M. (2016). A multilab preregistered replication of the ego-depletion effect. Perspectives on Psychological Science, 11(4), 546–573.

Hardy, R. J., & Thompson, S. G. (1998). Detecting and describing heterogeneity in meta-analysis. Statistics in Medicine, 17(8), 841–856.<841::AID-SIM781>3.0.CO;2-D

Hernández-García, Á., González-González, I., Jiménez-Zarco, A. I., & Chaparro-Peláez, J. (2015). Applying social learning analytics to message boards in online distance learning: A case study. Computers in Human Behavior, 47, 68–80.

Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558.

Hoppe, H. U. (2017). Computational methods for the analysis of learning and knowledge building communities. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), Handbook of learning analytics (pp. 23–33). Society for Learning Analytics Research (SoLAR).

Ifenthaler, D., & Yau, J. Y.-K. (2020). Utilising learning analytics to support study success in higher education: A systematic review. Educational Technology Research and Development, 68(4), 1961–1990.

IntHout, J., Ioannidis, J. P. A., Rovers, M. M., & Goeman, J. J. (2016). Plea for routinely presenting prediction intervals in meta-analysis. BMJ Open, 6(7).

Jeong, H., & Hmelo-Silver, C. E. (2016). Seven affordances of computer-supported collaborative learning: How to support collaborative learning? How can technologies help? Educational Psychologist, 51(2), 247–265.

Jiang, S., Fitzhugh, S. M., & Warschauer, M. (2014). Social positioning and performance in MOOCs. CEUR Workshop Proceedings, 1183, 55–58.

Jo, I., Park, Y., & Lee, H. (2017). Three interaction patterns on asynchronous online discussion behaviours: A methodological comparison. Journal of Computer Assisted Learning, 33(2), 106–122.

Joksimović, S., Manataki, A., Gašević, D., Dawson, S., Kovanović, V., & de Kereki, I. F. (2016). Translating network position into performance. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 314–323). ACM.

Joober, R., Schmitz, N., Annable, L., & Boksa, P. (2012). Publication bias: What are the challenges and can they be overcome? Journal of Psychiatry & Neuroscience, 37(3), 149–152.

Klein, R. A. (2019). Correction to Klein et al. (2014). Social Psychology, 50(3), 211–213.

Kreijns, K., Kirschner, P. A., & Vermeulen, M. (2013). Social aspects of CSCL environments: A research framework. Educational Psychologist, 48(4), 229–242.

Li, N., Marsh, V., Rienties, B., & Whitelock, D. (2017). Online learning experiences of new versus continuing learners: A large-scale replication study. Assessment & Evaluation in Higher Education, 42(4), 657–672.

Liu, C. H., & Matthews, R. (2005). Vygotsky’s philosophy: Constructivism and its criticisms examined. International Education Journal, 6(3), 386–399.

Liu, S., Chai, H., Liu, Z., Pinkwart, N., Han, X., & Hu, T. (2019). Effects of proactive personality and social centrality on learning performance in SPOCs. Proceedings of the 11th International Conference on Computer Supported Education (CSEDU 2019) 2–4 May 2019, Heraklion, Crete, Greece (Vol. 2, pp. 481–487). ScitePress.

Liu, Z., Kang, L., Domanska, M., Liu, S., Sun, J., & Fang, C. (2018). Social network characteristics of learners in a course forum and their relationship to learning outcomes. Proceedings of the 10th International Conference on Computer Supported Education (CSEDU 2018) 15–17 March 2018, Funchal, Madeira, Portugal (Vol. 1, pp. 15–21). ScitePress.

Liu, Z., Kang, L., Su, Z., Liu, S., & Sun, J. (2018). Investigate the relationship between learners’ social characteristics and academic achievements. Journal of Physics: Conference Series, 1113(1), 012021.

Lü, L., Chen, D., Ren, X.-L., Zhang, Q.-M., Zhang, Y.-C., & Zhou, T. (2016). Vital nodes identification in complex networks. Physics Reports, 650, 1–63.

Makel, M. C., & Plucker, J. A. (2014). Facts are more important than novelty. Educational Researcher, 43(6), 304–316.

Marín-Martínez, F., & Sánchez-Meca, J. (2010). Weighting by inverse variance or by sample size in random-effects meta-analysis. Educational and Psychological Measurement, 70(1), 56–73.

Martin, A. J., & Dowson, M. (2009). Interpersonal relationships, motivation, engagement, and achievement: Yields for theory, current issues, and educational practice. Review of Educational Research, 79(1), 327–365.

Osatuyi, B., & Passerini, K. (2016). Twittermania: Understanding how social media technologies impact engagement and academic performance of a new generation of learners. Communications of the Association for Information Systems, 39(1), 509–528.

Plesser, H. E. (2018). Reproducibility vs. replicability: A brief history of a confused terminology. Frontiers in Neuroinformatics, 11(January), 1–4.

Poquet, O., & Jovanovic, J. (2020). Intergroup and interpersonal forum positioning in shared-thread and post-reply networks. Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 187–196). ACM.

Poquet, O., Saqr, M., & Chen, B. (2021). Recommendations for network research in learning analytics: To open a conversation. In O. Poquet, B. Chen, M. Saqr, & T. Hecking (Eds.), Proceedings of the NetSciLA2021 Workshop “Using Network Science in Learning Analytics: Building Bridges towards a Common Agenda” (NetSciLA2021) (Issue 2868, pp. 34–41).

Poquet, O., Tupikina, L., & Santolini, M. (2020). Are forum networks social networks? Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 366–375). ACM.

Putnik, G., Costa, E., Alves, C., Castro, H., Varela, L., & Shah, V. (2016). Analysing the correlation between social network analysis measures and performance of students in social network-based engineering education. International Journal of Technology and Design Education, 26(3), 413–437.

R Core Team. (2018). R: A language and environment for statistical computing.

Reychav, I., Raban, D. R., & McHaney, R. (2018). Centrality measures and academic achievement in computerized classroom social networks. Journal of Educational Computing Research, 56(4), 589–618.

Rienties, B., Cross, S., & Zdrahal, Z. (2017, February). Implementing a learning analytics intervention and evaluation framework: What works? In B. Kei Daniel (Ed.), Big data and learning analytics in higher education (pp. 147–166). Springer International Publishing.

Romero, C., López, M.-I., Luna, J.-M., & Ventura, S. (2013). Predicting students’ final performance from participation in on-line discussion forums. Computers & Education, 68, 458–472.

Saqr, M., & Alamro, A. (2019). The role of social network analysis as a learning analytics tool in online problem based learning. BMC Medical Education, 19(1), 160.

Saqr, M., Fors, U., & Nouri, J. (2018). Using social network analysis to understand online problem-based learning and predict performance. PLOS ONE, 13(9).

Saqr, M., Fors, U., & Tedre, M. (2018). How the study of online collaborative learning can guide teachers and predict students’ performance in a medical course. BMC Medical Education, 18(1), 24.

Saqr, M., Fors, U., Tedre, M., & Nouri, J. (2018). How social network analysis can be used to monitor online collaborative learning and guide an informed intervention. PLOS ONE, 13(3).

Saqr, M., & López-Pernas, S. (2021a). Modelling diffusion in computer-supported collaborative learning: A large-scale learning analytics study. International Journal of Computer-Supported Collaborative Learning, in-press.

Saqr, M., & López-Pernas, S. (2021b). Idiographic learning analytics: A definition and a case study. Proceedings of the 21st IEEE International Conference on Advanced Learning Technologies (ICALT 2021), 12–15 July 2021, online (pp. 163–165). IEEE Computer Society.

Saqr, M., Nouri, J., & Jormanainen, I. (2019). A learning analytics study of the effect of group size on social dynamics and performance in online collaborative learning. In M. Scheffel, J. Broisin, V. Pammer-Schindler, A. Ioannou, & J. Schneider (Eds.), Lecture Notes in Computer Science (Vol. 11722, pp. 466–479). Springer.

Saqr, M., Nouri, J., Vartiainen, H., & Malmberg, J. (2020). What makes an online problem-based group successful? A learning analytics study using social network analysis. BMC Medical Education, 20(1), 80.

Saqr, M., & Viberg, O. (2020). Using diffusion network analytics to examine and support knowledge construction in CSCL settings. In C. Alario-Hoyos, M. J. Rodríguez-Triana, M. Scheffel, I. Arnedillo-Sánchez, & S. M. Dennerlein (Eds.), Proceedings of EC-TEL 2020: Addressing Global Challenges and Quality Education (Vol. 12315, pp. 158–172). Springer International Publishing.

Saqr, M., Viberg, O., & Vartiainen, H. (2020). Capturing the participation and social dimensions of computer-supported collaborative learning through social network analysis: Which method and measures matter? International Journal of Computer-Supported Collaborative Learning, 15(2), 227–248.

Schriger, D. L., Altman, D. G., Vetter, J. A., Heafner, T., & Moher, D. (2010). Forest plots in reports of systematic reviews: A cross-sectional study reviewing current practice. International Journal of Epidemiology, 39(2), 421–429.

Schwarzer, G., Carpenter, J. R., & Rücker, G. (2015). Meta-analysis with R. Springer International Publishing.

Shafie, T. (2019). A multigraph approach to social network analysis. Journal of Social Structure, 16(1), 0–21.

Sie, R. L. L., Ullmann, T. D., Rajagopal, K., Cela, K., Rijpkema, M. B., & Sloep, P. B. (2012). Social network analysis for technology-enhanced learning: Review and future directions. International Journal of Technology Enhanced Learning, 4(3/4), 172.

Skrypnyk, O., Joksimović, S., Kovanović, V., Gasšević, D., & Dawson, S. (2015). Roles of course facilitators, learners, and technology in the flow of information of a CMOOC. International Review of Research in Open and Distance Learning, 16(3), 188–217.

Stephenson, K., & Zelen, M. (1989). Rethinking centrality: Methods and examples. Social Networks, 11(1), 1–37.

Suthers, D. D. (2015). From contingencies to network-level phenomena. Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 368–377). ACM.

Suthers, D. D., & Desiato, C. (2012). Exposing chat features through analysis of uptake between contributions. Proceedings of the 45th Hawaii International Conference on System Sciences (HICSS-45), 4–7 January 2012, Maui, HI, USA (pp. 3368–3377). IEEE Computer Society.

Traxler, A. L., Gavrin, A., & Lindell, R. S. (2016). CourseNetworking and community: Linking online discussion networks and course success. Proceedings of the 2016 Physics Education Research Conference (PERC 2016), 20–21 July 2016, Sacramento, CA, USA (pp. 352–355).

Wecker, C., & Fischer, F. (2014). Where is the evidence? A meta-analysis on the role of argumentation for the acquisition of domain-specific knowledge in computer-supported collaborative learning. Computers & Education, 75, 218–228.

Weinberger, A., & Fischer, F. (2006). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers & Education, 46(1), 71–95.

Wise, A. F., & Cui, Y. (2018). Unpacking the relationship between discussion forum participation and learning in MOOCs. Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, NSW, Australia (pp. 330–339). ACM.

Wise, A. F., Cui, Y., & Jin, W. Q. (2017). Honing in on social learning networks in MOOC forums. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 383–392). ACM.

Wise, A. F., & Schwarz, B. B. (2017). Visions of CSCL: Eight provocations for the future of the field. International Journal of Computer-Supported Collaborative Learning, 12(4), 423–467.




How to Cite

Saqr, M., & López-Pernas, S. (2022). The Curious Case of Centrality Measures: A Large-Scale Empirical Investigation. Journal of Learning Analytics, 9(1), 13-31.



Special Section: Networks in Learning Analytics