Towards data-informed teaching practice:

A model for integrating analytics with teacher inquiry

Authors

DOI:

https://doi.org/10.18608/jla.2022.7505

Keywords:

data-informed teacher inquiry, teaching analytics, learning analytics, teacher inquiry model, thematic analysis

Abstract

Data-informed decision making in teachers’ practice, now recommended by different teacher inquiry models and policy documents, implies deep practice change for many teachers. However, not much is known how teachers perceive the different steps that analytics-informed teacher inquiry into their own practice entails. This paper presents the results of a study into developing an Analytics Model for Teacher Inquiry (AMTI), which was then used to understand how teachers (N=10) construe the steps in the model and to explore the possible constraints as well as incentives for Teaching and Learning Analytics (TLA)-informed teacher practices. In the final iteration experts (N=7) and teacher-researchers (N=2) tested and evaluated the developed model. Their feedback was used to improve the model and provide example cases with insights into possible scenarios for TLA-informed analyses of teaching.

References

ALLEA, A. (2017). The European Code of Conduct for Research Integrity, revised edition. Berlin2017. Retrieved from: http://www. allea. org. https://allea.org/code-of-conduct/

Alumäe, T., & Tilk, O. (2019). Advanced Rich Transcription System for Estonian Speech. arXiv preprint arXiv:1901.03601. https://doi.org/10.3233/978-1-61499-912-6-1

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa

Brown, C., Schildkamp, K., & Hubers, M. D. (2017). Combining the best of two worlds: A conceptual proposal for evidence-informed school improvement. Educational research, 59(2), 154-172. https://doi.org/10.1080/00131881.2017.1304327

Burks, B. A., Beziat, T. L., Danley, S., Davis, K., Lowery, H., & Lucas, J. (2015). Adapting to change: Teacher perceptions of implementing the Common Core State Standards. Education, 136(2), 253-258. Retrieved from: https://www.ingentaconnect.com/content/prin/ed/2015/00000136/00000002/art00013

Clow, D. (2012, April). The learning analytics cycle: closing the loop effectively. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 134-138). https://doi.org/10.1145/2330601.2330636

Corrin, L., De Barba, P. G., Lockyear, L., Gašević, D., Williams, D., Dawson, S., Mulder, R., Copeland, S., & Bakharia, A. (2016). Completing the Loop: Returning Meaningful Learning Analytic Data to Teachers. Australian Government Office for Learning and Teaching. Retrieved from: https://ltr.edu.au/resources/ID13_3068_Kennedy_Report_2016.pdf

Creswell , J.W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Boston: Pearson Education.

Dawson, S., Joksimovic, S., Poquet, O., & Siemens, G. (2019, March). Increasing the impact of learning analytics. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge (pp. 446-455). https://doi.org/10.1145/3303772.3303784

Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program planning: A guide to nominal group and Delphi processes: Scott. Foresman Glenview, IL.

Ebbeler, J., Poortman, C. L., Schildkamp, K., & Pieters, J. M. (2016). Effects of a data use intervention on educators’ use of knowledge and skills. Studies in educational evaluation, 48, 19-31. https://doi.org/10.1016/j.stueduc.2015.11.002

Edström, K. (2008). Doing course evaluation as if learning matters most. Higher education research & development, 27(2), 95-106. https://doi.org/10.1080/07294360701805234

Ferguson, R., Macfadyen, L., Clow, D., Tynan, B., Alexander, S. and Dawson, S. (2014). Setting Learning Analytics in Context: Overcoming the Barriers to Large‐Scale Adoption. Journal of Learning Analytics, 1, 3, 120-144. https://doi.org/10.18608/jla.2014.13.7

Fullan, M. (Ed.). (2014). Teacher development and educational change. Routledge. https://doi.org/10.4324/9781315870700

Fuller, K. A., Karunaratne, N. S., Naidu, S., Exintaris, B., Short, J. L., Wolcott, M. D., ... & White, P. J. (2018). Development of a self-report instrument for measuring in-class student engagement reveals that pretending to engage is a significant unrecognized problem. PloS one, 13(10), e0205828. https://doi.org/10.1371/journal.pone.0205828

Gaertner, H. (2014). Effects of student feedback as a method of self-evaluating the quality of teaching. Studies in Educational Evaluation, 42, 91-99. https://doi.org/10.1016/j.stueduc.2014.04.003

Gleason, P., Crissey, S., Chojnacki, G., Zukiewicz, M., Silva, T., Costelloe, S., & O'Reilly, F. (2019). Evaluation of Support for Using Student Data to Inform Teachers' Instruction. NCEE 2019-4008. National Center for Education Evaluation and Regional Assistance. Retrieved from: https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=NCEE20194008

Goertz, M.E., Nabors Oláh, L. & Riggan, M. (2009). From testing to teaching: The use of interim assessments in classroom instruction. CPRE Research Report #RR-65. Philadelphia, PA: Consortium for Policy Research in Education. https://doi.org/10.12698/CPRE.2009.RR65

Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics. Journal of Educational Technology & Society, 15(3), 42-57. Retrieved from: http://www.jstor.org/stable/jeductechsoci.15.3.42

Hansen, C. J., & Wasson, B. (2016). Teacher Inquiry into Student Learning:-The TISL Heart Model and Method for use in Teachers’ Professional Development. Nordic Journal of Digital Literacy, 11(01), 24-49. https://doi.org/10.18261/issn.1891-943x-2016-01-02

Hea Teadustava. (2017). Estonian Code of Conduct for Research Integrity. Tartu. Retrieved from: https://www.eetika.ee/sites/default/files/www_ut/hea_teadustava_eng_trukis.pdf

Ingram, D., Seashore Louis, K., & Schroeder, R. (2004). Accountability policies and teacher decision making: Barriers to the use of data to improve practice. Teachers college record, 106(6), 1258-1287. https://doi.org/10.1111/j.1467-9620.2004.00379.x

Joksimović, S., Kovanović, V., & Dawson, S. (2019). The journey of learning analytics. HERDSA Review of Higher Education, 6, 27-63. Retrieved from: https://www.herdsa.org.au/herdsa-review-higher-education-vol-6/37-63

Lai, M. K., & Schildkamp, K. (2013). Data-based decision making: An overview. In Data-based decision making in education (pp. 9-21). Springer, Dordrecht. https://doi.org/10.1007/978-94-007-4816-3

Mandinach, E. B. (2012). A perfect time for data use: Using data-driven decision making to inform practice. Educational Psychologist, 47(2), 71-85. https://doi.org/10.1080/00461520.2012.667064

Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making in education: Evidence from recent RAND research. Retrieved from: https://www.rand.org/pubs/occasional_papers/OP170.html

Maulana, R., Helms-Lorenz, M., & Van de Grift, W. (2017). Validating a model of effective teaching behaviour of pre-service teachers. Teachers and Teaching, 23(4), 471-493. Retrieved from: https://psycnet.apa.org/record/2017-09817-007

McKenney, S., & Reeves, T. C. (2012). Conducting educational design research. London: Routledge.

Meyers, G. L., Jacobsen, M., & Henderson, E. (2018). Design-Based Research: Introducing an innovative research methodology to infection prevention and control. Canadian Journal of Infection Control, 33(3). Retrieved from: https://ipac-canada.org/photos/custom/CJIC/CJIC_Fall2018_Meyers.pdf

Molenaar, I., & Knoop-van Campen, C. A. (2018). How teachers make dashboard information actionable. IEEE Transactions on Learning Technologies, 12(3), 347-355. https://doi.org/10.1109/TLT.2018.2851585

Mullen, R., Kydd, A., Fleming, A., & McMillan, L. (2021). A practical guide to the systematic application of nominal group technique. Nurse researcher, 29(1). https://doi.org/10.7748/nr.2021.e1777

Ndukwe, I. G., & Daniel, B. K. (2020). Teaching analytics, value and tools for teacher data literacy: A systematic and tripartite approach. International Journal of Educational Technology in Higher Education, 17(1), 1-31. https://doi.org/10.1186/s41239-020-00201-6

Poortman, C. L., & Schildkamp, K. (2016). Solving student achievement problems with a data use intervention for teachers. Teaching and teacher education, 60, 425-433. https://doi.org/10.1016/j.tate.2016.06.010

Priestley, M., Edwards, R., Priestley, A., & Miller, K. (2012). Teacher agency in curriculum making: Agents of change and spaces for manoeuvre. Curriculum inquiry, 42(2), 191-214. https://doi.org/10.1111/j.1467-873X.2012.00588.x

Rolfe, G., Freshwater, D., & Jasper, M. (2001). Critical Reflection for Nursing and the Helping Professions a User's Guide. Basingstoke, Palgrave Macmillan

Schildkamp, K., & Datnow, A. (2020). When Data Teams Struggle: Learning from Less Successful Data Use Efforts. Leadership and Policy in Schools, 1-20. https://doi.org/10.1080/15700763.2020.1734630

Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors. Teaching and teacher education, 26(3), 482-496. doi.org/ https://doi.org/10.1016/j.tate.2009.06.007

Schildkamp, K., & Poortman, C. (2015). Factors influencing the functioning of data teams. Teachers college record, 117(4). Retrieved from: https://research.utwente.nl/en/publications/factors-influencing-the-functioning-of-data-teams

Schildkamp, K., Poortman, C. L., & Handelzalts, A. (2016). Data teams for school improvement. School effectiveness and school improvement, 27(2), 228-254. https://doi.org/10.1080/09243453.2015.1056192

Schildkamp, K., Poortman, C. L., Ebbeler, J., & Pieters, J. M. (2019). How school leaders can build effective data teams: Five building blocks for a new wave of data-informed decision making. Journal of educational change, 20(3), 283-325. https://doi.org/10.1007/s10833-019-09345-3

Seidel, T., Stürmer, K., Blomberg, G., Kobarg, M., & Schwindt, K. (2011). Teacher learning from analysis of videotaped classroom situations: Does it make a difference whether teachers observe their own teaching or that of others?. Teaching and teacher education, 27(2), 259-267. https://doi.org/10.1016/j.tate.2010.08.009

Sergis, S., & Sampson, D. G. (2017). Teaching and learning analytics to support teacher inquiry: A systematic literature review. Learning analytics: Fundaments, applications, and trends, 25-63. https://doi.org/10.1007/978-3-319-52977-6_2

Timperley, H. (2010, February). Using evidence in the classroom for professional learning. In Étude présentée lors du Colloque ontarien sur la recherche en éducation. Retrieved from: https://cdn.auckland.ac.nz/assets/education/about/schools/tchldv/docs/Using%20Evidence%20in%20the%20Classroom%20for%20Professional%20Learning.pdf

Varga-Atkins, T., McIsaac, J., & Willis, I. (2017). Focus Group meets Nominal Group Technique: an effective combination for student evaluation?. Innovations in Education and Teaching International, 54(4), 289-300. https://doi.org/10.1080/14703297.2015.1058721

Viberg, O., & Gronlund, A. (2021). Desperately seeking the impact of learning analytics in education at scale: Marrying data analysis with teaching and learning. Retrieved from: arXiv:2105.06680v1

Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational technology research and development, 53(4), 5-23. Retrieved from: https://www.jstor.org/stable/30221206

Wayman, J. C., & Jimerson, J. B. (2014). Teacher needs for data-related professional learning. Studies in Educational Evaluation, 42, 25-34. https://doi.org/10.1016/j.stueduc.2013.11.001

Wise, A. F., & Jung, Y. (2019). Teaching with analytics: Towards a situated model of instructional decision-making. Journal of Learning Analytics, 6(2), 53-69. https://doi.org/10.18608/jla.2019.62.4

Downloads

Published

2022-10-23

How to Cite

Saar, M., Rodríguez-Triana, M. J. ., & Prieto Santos, L. P. (2022). Towards data-informed teaching practice: : A model for integrating analytics with teacher inquiry. Journal of Learning Analytics, 1-16. https://doi.org/10.18608/jla.2022.7505

Issue

Section

Research Papers

Most read articles by the same author(s)