Real-Time Prediction of Students’ Activity Progress and Completion Rates




instructional timing, progress predictions, classroom orchestration


In classrooms, some transitions between activities impose (quasi-)synchronicity, meaning there is a need for learners to move between activities at the same time. To make real-time decisions about when to move to the next activity, teachers need to be able to balance the progress of their students as they work at different paces. In this paper, we present a set of estimators that can be used in real time to predict the progress and completion rates of students working on computer-supported activities that can be divided into sequential subtasks. With our estimators, we investigate what effect the average progress rate of the class, a given number of previous steps, or weighting the proportion of progress assigned to each subtask has on predictions of students’ progress. We find that accounting for the average class progress rate near the beginning of the activity can improve predictions over baseline. Additionally, weighted subtasks decrease prediction accuracy for activities where the behaviour of faster students diverges from the average behaviour of the class. This paper contributes to our ability to provide accurate student progress predictions and to understand the behaviour of students as they progress through the activity. These real-time predictions can enable teachers to optimize learning time in their classrooms.


Alavi, H. S., & Dillenbourg, P. (2012). An ambient awareness tool for supporting supervised collaborative problem solving.

IEEE Transactions on Learning Technologies, 5(3), 264–274.

Baker, R., Corbett, A. T., & Koedinger, K. R. (2004). Detecting student misuse of intelligent tutoring systems. In 7th International Conference on Intelligent Tutoring Systems (ITS2004), 30 August–3 September 2004, Macei´o, Alagoas, Brazil (pp. 531–540). Springer.

Baker, R., & Siemens, G. (2014). Educational data mining and learning analytics. In R. K. Sawyer (Ed.), Cambridge Handbook of the Learning Sciences. Cambridge, UK: Cambridge University Press.

Beck, J. E. (2005). Engagement tracing: Using response times to model student disengagement. In C. Looi, G. I. McCalla, B. Bredeweg, & J. Breuker (Eds.), Artificial Intelligence in Education—Supporting Learning through Intelligent and Socially Informed Technology, Proceedings of the 12th International Conference on Artificial Intelligence in Education, AIED 2005, 18–22 July 2005, Amsterdam, Netherlands (Vol. 125, pp. 88–95). IOS Press. Retrieved from

Boroujeni, M. S., Sharma, K., Kidzi´nski, Ł., Lucignano, L., & Dillenbourg, P. (2016). How to quantify student’s regularity? In Proceedings of the 11th European Conference on Technology Enhanced Learning (EC-TEL 2016), 13–16 September 2016, Lyon, France (pp. 277–291). Springer.

Boyer, S., & Veeramachaneni, K. (2015). Transfer learning for predictive models in massive open online courses. In Proceedings of the 17th International Conference on Artificial Intelligence in Education (AIED 2015), 22–26 June 2015, Madrid, Spain (pp. 54–63). Springer.

Campbell, S., & Skinner, C. H. (2004). Combining explicit timing with an interdependent group contingency program to decrease transition times: An investigation of the timely transitions game. Journal of Applied School Psychology, 20(2), 11–27.

Charleer, S., Santos, J. L., Klerkx, J., & Duval, E. (2014). Improving teacher awareness through activity, badge and content visualizations. In Proceedings of the 13th International Conference on Web-Based Learning (ICWL 2014), 14–17 August 2014, Tallinn, Estonia (pp. 143–152). Springer.

Clark, D., & Linn, M. C. (2003). Designing for knowledge integration: The impact of instructional time. The Journal of the Learning Sciences, 12(4), 451–493.

Corbett, A. T., & Anderson, J. R. (1994). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4), 253–278.

Crossley, S., Paquette, L., Dascalu, M., McNamara, D. S., & Baker, R. S. (2016). Combining click-stream data with NLP tools to better understand MOOC completion. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK 2016), 25–29 April 2015, Edinburgh, United Kingdom (pp. 6–14). New York, USA: ACM.

Dekker, G. W., Pechenizkiy, M., & Vleeshouwers, J. M. (2009). Predicting students drop out: A case study. In T. Barnes, M. Desmarais, C. Romero, & S. Ventura (Eds.), Proceedings of the Second International Conference on Educational Data Mining (EDM ’09), 1–3 July 2009, Cordoba, Spain. Retrieved from

Dillenbourg, P. (2013). Design for classroom orchestration. Computers & Education, 69, 485–492.

Dillenbourg, P. (2015). Orchestration Graphs. Lausanne, Switzerland: EPFL Press. Retrieved from

Dillenbourg, P., Li, N., & Kidzinski, L. (2016). The Complications of the Orchestration Clock (Tech. Rep.). Portland Press. Retrieved from

Dillenbourg, P., Prieto, L. P., & Olsen, J. K. (2018). Classroom orchestration. In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, & P. Reimann (Eds.), International Handbook of the Learning Sciences (pp. 180–190). Routledge.

Dillenbourg, P., Zufferey, G., Alavi, H. S., Jermann, P., Do, L. H. S., Bonnard, Q., . . . Kaplan, F. (2011). Classroom orchestration: The third circle of usability. In Proceedings of the Ninth Annual Conference on Computer-Supported Collaborative Learning (CSCL2011), 4–8 July 2011, Hong Kong, China (Vol. 1, pp. 510–517). International Society of the Learning Sciences. Retrieved from

D’mello, S. K., Craig, S. D., Witherspoon, A., Mcdaniel, B., & Graesser, A. (2008). Automatic detection of learner’s affect from conversational cues. User Modeling and User-Adapted Interaction, 18(1-2), 45–80.

Feng, M., Heffernan, N., & Koedinger, K. (2009). Addressing the assessment challenge with an online system that tutors as it assesses. User Modeling and User-Adapted Interaction, 19(3), 243–266.

Fiel, J., Lawless, K. A., & Brown, S. W. (2018). Timing matters: Approaches for measuring and visualizing behaviours of timing and spacing of work in self-paced online teacher professional development courses. Journal of Learning Analytics, 5(1), 25–40.

Fraser, B. J.,Walberg, H. J.,Welch, W.W., & Hattie, J. A. (1987). Syntheses of educational productivity research. International Journal of Educational Research, 11(2), 147–252.

Gettinger, M., & Seibert, J. K. (2002). Best practices in increasing academic learning time. Best Practices in School Psychology, IV, 773–787. Retrieved from

Haklev, S., Faucon, L., Hadzilacos, T., & Dillenbourg, P. (2017). Orchestration graphs: Enabling rich social pedagogical scenarios in MOOCs. In Proceedings of the Fourth ACM Conference on Learning@Scale (L@S 2017), 20–21 April 2017, Cambridge, Massachusetts, USA (pp. 261–264). New York, USA: ACM.

Hine, J. F., Ardoin, S. P., & Foster, T. E. (2015). Decreasing transition times in elementary school classrooms: Using computer-assisted instruction to automate intervention components. Journal of Applied Behavior Analysis, 48(3), 495–510.

Holstein, K., Hong, G., Tegene, M., McLaren, B. M., & Aleven, V. (2018). The classroom as a dashboard: Co-designing wearable cognitive augmentation for K–12 teachers. In Proceedings of the Eighth International Conference on Learning Analytics and Knowledge (LAK 2018), 5–9 March 2018, Sydney, Australia (pp. 79–88). New York, USA: ACM.

Hutt, S., Mills, C., Bosch, N., Krasich, K., Brockmole, J., & D’Mello, S. (2017). Out of the fr-eye-ing pan: Towards gaze-based models of attention during learning with technology in the classroom. In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization (UMAP 2017), 9–12 July 2017, Bratislava, Slovakia (pp. 94–103). New York, USA: ACM.

Jovanovi´c, J., Gaˇsevi´c, D., Dawson, S., Pardo, A., & Mirriahi, N. (2017). Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education, 33(4), 74–85. Retrieved from

Karweit, N., & Slavin, R. E. (1981). Measurement and modeling choices in studies of time and learning. American Educational Research Journal, 18(2), 157–171.

Klerkx, J., Verbert, K., & Duval, E. (2017). Learning analytics dashboards. Handbook of Learning Analytics, 1, 143–150.

Kollar, I., & Fischer, F. (2013). Orchestration is nothing without conducting—But arranging ties the two together!: A response to Dillenbourg (2011). Computers & Education, 69, 507–509.

Loboda, T. D., Guerra, J., Hosseini, R., & Brusilovsky, P. (2014). Mastery grids: An open source social educational progress visualization. In Proceedings of the Ninth European Conference on Technology Enhanced Learning (EC-TEL 2014), 16–19 September 2014, Graz, Austria (pp. 235–248). Springer.

Malone, T. W. (1981). Toward a theory of intrinsically motivating instruction. Cognitive Science, 5(4), 333–369.

Martinez-Maldonado, R., Clayphan, A., & Kay, J. (2015). Deploying and visualising teacher’s scripts of small group activities in a multi-surface classroom ecology: A study in-the-wild. Computer Supported Cooperative Work (CSCW), 24(2-3), 177–221.

Ming, N., & Ming, V. (2012). Predicting student outcomes from unstructured data. In Workshop and Poster Proceedings of the 20th Conference on User Modeling, Adaptation, and Personalization (UMAP 2012), 16–20 July 2012, Montreal, Canada. Retrieved from

Molenaar, I., & Knoop-van Campen, C. (2017). Teacher dashboards in practice: Usage and impact. In Proceedings of the 12th

European Conference on Technology Enhanced Learning (EC-TEL 2017), 12–15 September 2017, Tallinn, Estonia (pp. 125–138). Springer.

Pavlik Jr., P. I., Cen, H., & Koedinger, K. R. (2009). Performance factors analysis—A new alternative to knowledge tracing. (online submission) Retrieved from

Pel´anek, R. (2015). Metrics for evaluation of student models. Journal of Educational Data Mining, 7(2), 1–19. Retrieved


Phiri, L., Meinel, C., & Suleman, H. (2016). Streamlined orchestration: An orchestration workbench framework for effective teaching. Computers & Education, 95, 231–238.

Prieto, L. P., Dlab, M. H., Guti´errez, I., Abdulwahed, M., & Balid, W. (2011). Orchestrating technology enhanced learning: A literature review and a conceptual framework. International Journal of Technology Enhanced Learning, 3(6), 583.

Rhymer, K. N., Skinner, C. H., Jackson, S., McNeill, S., Smith, T., & Jackson, B. (2002). The 1-minute explicit timing intervention: The influence of mathematics problem difficulty. Journal of Instructional Psychology, 29(4), 305–311. Retrieved from

Schwendimann, B. A., Rodriguez-Triana, M. J., Vozniuk, A., Prieto, L. P., Boroujeni, M. S., Holzer, A., . . . Dillenbourg, P. (2016). Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies, 10(1), 30–41.

Slotta, J. D., Tissenbaum, M., & Lui, M. (2013). Orchestrating of complex inquiry: Three roles for learning analytics in a smart classroom infrastructure. In Proceedings of the Third International Conference on Learning Analytics and Knowledge (LAK 2013), 8–12 April 2013, Leuven, Belgium (pp. 270–274). New York, USA: ACM.

Song, Y., Wong, L.-H., & Looi, C.-K. (2012). Fostering personalized learning in science inquiry supported by mobile technologies. Educational Technology Research and Development, 60(4), 679–701.

Taylor, C., Veeramachaneni, K., & O’Reilly, U.-M. (2014). Likely to stop? Predicting stopout in massive open online courses. arXiv preprint arXiv:1408.3382.

Van Houten, R., Hill, S., & Parsons, M. (1975). An analysis of a performance feedback system: The effects of timing and feedback, public posting, and praise upon academic performance and peer interaction 1. Journal of Applied Behavior Analysis, 8(4), 449–457.

Van Houten, R., & Thompson, C. (1976). The effects of explicit timing on math performance. Journal of Applied Behavior Analysis, 9(2), 227.

van Leeuwen, A., Rummel, N., & van Gog, T. (2019). What information should CSCL teacher dashboards provide to help teachers interpret CSCL situations? International Journal of Computer-Supported Collaborative Learning, 14, 261–289.

van Leeuwen, A., van Wermeskerken, M., Erkens, G., & Rummel, N. (2017). Measuring teacher sense making strategies of learning analytics: A case study. Learning: Research and Practice, 3(1), 42–58.

Vanlehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227–265.

Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Assche, F., Parra, G., & Klerkx, J. (2014). Learning dashboards: An overview and future research opportunities. Personal and Ubiquitous Computing, 18(6), 1499–1514.

Wachtler, J., Khalil, M., Taraghi, B., & Ebner, M. (2016). On using learning analytics to track the activity of interactive MOOC videos. In Proceedings of the LAK 2016 Workshop on Smart Environments and Analytics in Video-Based Learning, 26 April 2016, Edinburgh, United Kingdom (pp. 8–17). Retrieved from

Wang, P., Tchounikine, P., & Quignard, M. (2017). Chao: A framework for the development of orchestration technologies for technology-enhanced learning activities using tablets in classrooms. International Journal of Technology Enhanced Learning, 10(1/2), 1–21.

Whitehill, J., Mohan, K., Seaton, D., Rosen, Y., & Tingley, D. (2017). Delving deeper into MOOC student dropout prediction. arXiv preprint arXiv:1702.06404.

Xu, D., Wang, H., & Wang, M. (2005). A conceptual model of personalized virtual learning environments. Expert Systems with Applications, 29(3), 525–534.

Yarbrough, J. L., Skinner, C. H., Lee, Y. J., & Lemmons, C. (2004). Decreasing transition times in a second grade classroom: Scientific support for the timely transitions game. Journal of Applied School Psychology, 20(2), 85–107.




How to Cite

Faucon, L., Olsen, J. K., Haklev, S., & Dillenbourg, P. (2020). Real-Time Prediction of Students’ Activity Progress and Completion Rates. Journal of Learning Analytics, 7(2), 18-44.