Exploratory versus Explanatory Visual Learning Analytics: Driving Teachers’ Attention through Educational Data Storytelling

Roberto Martinez-Maldonado
Simon Buckingham Shum
Katherine Chiluiza
Roger Granda
Cristina Conati

Abstract


From a human-centred computing perspective, supporting the interpretation of educational dashboards and visualizations by the people intended to use them exposes critical design challenges that may often be trivialized. Empirical evidence already shows that “usable” visualizations are not necessarily effective from an educational perspective. Since an educator’s interpretation of visualized data is essentially the construction of a narrative about student progress, in this paper, we propose the concept of “Educational Data Storytelling” as an approach for explaining student data by aligning educational visualizations with the intended learning design. We present a pilot study that explores the effectiveness of these data storytelling elements based on educator responses to prototypes by analyzing the kinds of stories they articulate, their eye-tracking behaviour, and their preferences after inspecting a series of student data visualizations. The dual purpose is to understand the contribution of each visual element for data storytelling, as well as the effectiveness of the enhancements when combined.


Full Text:

PDF

References

Bach, B., Kerracher, N., Hall, K. W., Carpendale, S., Kennedy, J., & Henry Riche, N. (2016). Telling Stories About Dynamic Networks with Graph Comics. Paper presented at the Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, (pp. 3670-3682). ACM.

Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gašević, D., Mulder, R., Williams, D., Dawson, S., & Lockyer, L. (2016). A conceptual framework linking learning design with learning analytics. Paper presented at the Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, (pp. 329-338). ACM.

Bodily, R., Kay, J., Aleven, V., Jivet, I., Davis, D., Xhakaj, F., & Verbert, K. (2018). Open learner models and learning analytics dashboards: a systematic review. In Proceedings of the Proceedings of the 8th International Conference on Learning Analytics and Knowledge, (pp. 41-50). Sydney, New South Wales, Australia. 3170409: ACM.

Bodily, R., & Verbert, K. (2017). Trends and Issues in Student-facing Learning Analytics Reporting Systems Research. Paper presented at the Proceedings of the Seventh International Learning Analytics & Knowledge Conference, (pp. 309-318). ACM.

Bryan, C., Ma, K.-L., & Woodring, J. (2017). Temporal summary images: An approach to narrative visualization via interactive annotation generation and placement. IEEE transactions on visualization and computer graphics, 23(1), 511-520.

Corrin, L., & de Barba, P. (2015). How Do Students Interpret Feedback Delivered via Dashboards? Paper presented at the Proceedings of the Fifth International Conference on Learning Analytics And Knowledge, (pp. 430-431). ACM.

Duval, E. (2011). Attention please!: learning analytics for visualization and recommendation. In Proceedings of the Proceedings of the 1st International Conference on Learning Analytics and Knowledge, (pp. 9-17). Banff, Alberta, Canada. New York: ACM.

Dykes, B. (2015). Data storytelling: What it is and how it can be used to effectively communicate analysis results. Applied Marketing Analytics, 1(4), 299-313.

Echeverria, V., Martinez-Maldonado, R., Chiluiza, K., & Buckingham Shum, S. (2017). DBCollab: Automated feedback for face-to-Face Group Database Design. Paper presented at the International Conference on Computers in Education, ICCE 2017, Christchurch, New Zealand.

Echeverria, V., Martinez-Maldonado, R., Granda, R., Chiluiza, K., Conati, C., & Buckingham Shum, S. (2018). Driving Data Storytelling from Learning Design. In Proceedings of the Eighth International Learning Analytics & Knowledge Conference, (pp. 131-140). ACM.

Feng, M., Krumm, A. E., Bowers, A. J., & Podkul, T. (2016). Elaborating Data Intensive Research Methods Through Researcher-practitioner Partnerships. Paper presented at the Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, (pp. 540-541). ACM.

Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics. Educational Technology & Society, 15(3), 42–57.

Holstein, K., McLaren, B. M., & Aleven, V. (2017). Intelligent tutors as teachers' aides: exploring teacher needs for real-time analytics in blended classrooms. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference, (pp. 257-266). ACM.

Iliinsky, N., & Steele, J. (2011). Designing data visualizations: representing informational relationships: O'Reilly Media, Inc.

Jivet, I., Scheffel, M., Drachsler, H., & Specht, M. (2017). Awareness is not enough: pitfalls of learning analytics dashboards in the educational practice. Paper presented at the European Conference on Technology Enhanced Learning, (pp. 82-96). Springer.

Jivet, I., Scheffel, M., Specht, M., & Drachsler, H. (2018, March 7-9, 2018). License to Evaluate: Preparing Learning Analytics Dashboards for Educational Practice. Paper presented at the International Conference on Learning Analytics and Knowledge, (pp. Sydney, NSW, Australia, March 7-9, 2018.

Kim, S., Dey, A. K., Lee, J., & Forlizzi, J. (2011). Usability of car dashboard displays for elder drivers. Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, (pp. 493-502). ACM.

Klerkx, J., Verbert, K., & Duval, E. (2017). Learning analytics dashboards.

Knaflic, C. N. (2015). Storytelling with data: A data visualization guide for business professionals: John Wiley & Sons.

Kodinariya, T. M., & Makwana, P. R. (2013). Review on determining number of Cluster in K-Means Clustering. International Journal, 1(6), 90-95.

Krumm, A., Means, B., & Bienkowski, M. (2018). Learning Analytics Goes to School: A Collaborative Approach to Improving Education: Routledge.

Lallé, S., Conati, C., & Carenini, G. (2016). Prediction of individual learning curves across information visualizations. User Modeling and User-Adapted Interaction, 26(4), 307-345.

Lee, B., Riche, N. H., Isenberg, P., & Carpendale, S. (2015). More Than Telling a Story: Transforming Data into Visually Shared Stories. IEEE Computer Graphics and Applications, 35(5), 84-90.

Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439-1459.

Ma, K. L., Liao, I., Frazier, J., Hauser, H., & Kostis, H. N. (2012). Scientific Storytelling Using Visualization. IEEE Computer Graphics and Applications, 32(1), 12-19.

Marzouk, Z., Rakovic, M., Liaqat, A., Vytasek, J., Samadi, D., Stewart-Alonso, J., Ram, I., Woloshen, S., Winne, P. H., & Nesbit, J. C. (2016). What if learning analytics were based on learning science? Australasian Journal of Educational Technology, 32(6).

Metoyer, R., Zhi, Q., Janczuk, B., & Scheirer, W. (2018). Coupling Story to Visualization: Using Textual Analysis as a Bridge Between Data and Interpretation. Paper presented at the 23rd International Conference on Intelligent User Interfaces, (pp. 503-507). ACM.

Molenaar, I., & Knoop-van Campen, C. (2017). Teacher dashboards in practice: Usage and impact. Paper presented at the European Conference on Technology Enhanced Learning, (pp. 125-138). Springer.

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., & Dubourg, V. (2011). Scikit-learn: Machine learning in Python. Journal of machine learning research, 12(Oct), 2825-2830.

Reimers, G., & Neovesky, A. (2015). Student focused dashboards. In Proceedings of the 7th International Conference on Computer Supported Education (Lisbon), Lisbon.

Roberts, L. D., Howell, J. A., & Seaman, K. (2017). Give Me a Customizable Dashboard: Personalized Learning Analytics Dashboards in Higher Education. Technology, Knowledge and Learning, 22(3), 317-333.

Rodríguez-Triana, M. J., Martínez-Monés, A., Asensio-Pérez, J. I., & Dimitriadis, Y. (2015). Scripting and monitoring meet each other: Aligning learning analytics and learning design to support teachers in orchestrating CSCL situations. British Journal of Educational Technology, 46(2), 330-343.

Ryan, L. (2016). The Visual Imperative : Creating a Visual Culture of Data Discovery: Elsevier Science.

Schwendimann, B. A., Rodríguez-Triana, M. J., Vozniuk, A., Prieto, L. P., Boroujeni, M. S., Holzer, A., Gillet, D., & Dillenbourg, P. (2017). Perceiving Learning at a Glance: A Systematic Literature Review of Learning Dashboard Research. IEEE Transactions on Learning Technologies, 10(1), 30-41.

Segel, E., & Heer, J. (2010). Narrative Visualization: Telling Stories with Data. IEEE Transactions on Visualization and Computer Graphics, 16(6), 1139-1148.

Stevens, A., Quimby, A., Board, A., Kersloot, T., & Burns, P. (2002). Design guidelines for safety of in-vehicle information systems. TRL Limited.

Teasley, S. D. (2017). Student Facing Dashboards: One Size Fits All? Technology, Knowledge and Learning, 22(3), 377-384.

Toker, D., Conati, C., Steichen, B., & Carenini, G. (2013). Individual user characteristics and information visualization: connecting the dots through eye tracking. Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, (pp. 295-304). New York, NY, USA: ACM.

Tufte, E. R., & Schmieg, G. M. (1985). The visual display of quantitative information (Vol. 53): American Association of Physics Teachers.

Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning Analytics Dashboard Applications. American Behavioral Scientist, 57(10), 1500-1509.



PDF


DOI: https://doi.org/10.18608/jla.2018.53.6

Share this article: