How do Students Learn to See Concepts in Visualizations? Social Learning Mechanisms with Physical and Virtual Representations


  • Martina A. Rau University of Wisconsin - Madison



Visual representations, physical and virtual manipulatives, frequent pattern mining, grounded theory, social mediaton


STEM instruction often uses visual representations. To benefit from these, students need to understand how representations show domain-relevant concepts. Yet, this is difficult for students. Prior research shows that physical representations (objects that students manipulate by hand) and virtual representations (objects on a computer screen that students manipulate via mouse and keyboard) have complementary advantages for conceptual learning. However, physical and virtual representations are often embedded into different social classroom contexts, which may affect social mechanisms through which students construct connections between concepts and representations. Therefore, this paper focuses on the social events that precede concept-representation connections. Twelve high-school students worked collaboratively with physical and virtual representations of chemistry. Frequent pattern mining of discourse data was used to identify social events that often preceded concept-representation connections. Qualitative analysis investigated social mechanisms through which these events may enhance concept-representation connections. Results show that students construct concept-representation connections incrementally. Further, meta-cognitive strategies and instructor prompts often preceded concept-representation connections. Finally, differences between physical and virtual representations were mainly due to different social supports being available in the context in which the representations were embedded. I discuss contributions to literature on learning with multiple representations and to interventions that blend representation modes.


Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16(3), 183-198. doi: 10.1016/j.learninstruc.2006.03.001

Ainsworth, S. (2008). The educational value of multiple-representations when learning complex scientific concepts. In J. K. Gilbert, M. Reiner & A. Nakama (Eds.), Visualization: Theory and practice in science education (pp. 191-208). Netherlands: Springer.

Ainsworth, S., Bibby, P., & Wood, D. (2002). Examining the effects of different multiple representational systems in learning primary mathematics. Journal of the Learning Sciences, 11(1), 25-61. doi: 10.1207/S15327809JLS1101_2

Airey, J., & Linder, C. (2009). A disciplinary discourse perspective on university science learning: Achieving fluency in a critical constellation of modes. Journal of Research in Science Teaching, 46(1), 27-49. doi: 10.1002/tea.20265

Bakker, S., Antle, A. N., & Van Den Hoven, E. (2012). Embodied metaphors in tangible interaction design. Personal and Ubiquitous Computing, 16(4), 433-449. doi: 10.1007/s00779-011-0410-4

Bodemer, D., & Faust, U. (2006). External and mental referencing of multiple representations. Computers in Human Behavior, 22(1), 27-42. doi: 10.1016/j.chb.2005.01.005

Bodner, G. M., & Domin, D. S. (2000). Mental models: The role of representations in problem solving in chemistry. University Chemistry Education, 4(1), 24-30.

Boulter, C. J., & Gilbert, J. K. (2000). Challenges and opportunities of developing models in science education. In J. K. Gilbert & C. J. Boulter (Eds.), Developing Models in Science Education (pp. 343-362). Netherlands: Kluwer Adademic Publishers.

Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13(2), 145-182. doi: 10.1016/0364-0213(89)90002-5

Chini, J. J., Madsen, A., Gire, E., Rebello, N. S., & Puntambekar, S. (2012). Exploration of factors that affect the comparative effectiveness of physical and virtual manipulatives in an undergraduate laboratory. Physical Review Special Topics-Physics Education Research, 8(1), 010113. doi: DOI:

Clements, D. H. (1999). 'Concrete'Manipulatives, Concrete Ideas. Contemporary Issues in Early Childhood, 1(1), 45-60. doi:

de Jong, T., Linn, M. C., & Zacharia, Z. C. (2013). Physical and virtual laboratories in science and engineering education. Science, 340(6130), 305-308. doi: 10.1126/science.1230579

Dori, Y. J., & Barak, M. (2001). Virtual and physical molecular modeling: Fostering model perception and spatial understanding. Educational Technology & Society, 4(1), 61-74.

Dreher, A., & Kuntze, S. (2014). Teachers Facing the Dilemma of Multiple Representations Being Aid and Obstacle for Learning: Evaluations of Tasks and Theme-Specific Noticing. Journal für Mathematik-Didaktik, 1-22. doi: 10.1007/s13138-014-0068-3

Finkelstein, N. D., Adams, W. K., Keller, C. J., Kohl, P. B., Perkins, K. K., Podolefsky, N. S., . . . LeMaster, R. (2005). When learning about the real world is better done virtually: A study of substituting computer simulations for laboratory equipment. Physical Review Special Topics—Physics Education Research, 1, 1-8.

Gentner, D. (1983). Structure-Mapping: A Theoretical Framework for Analogy. Cognitive Science, 7(2), 155-170. doi: 10.1207/s15516709cog0702_3

Gilbert, J. K. (2005). Visualization: A metacognitive skill in science and science education. In J. K. Gilbert (Ed.), Visualization: Theory and practice in science education (pp. 9-27). Dordrecht, Netherlands: Springer.

Jaakkola, T., Nurmi, S., & Veermans, K. (2011). A comparison of students' conceptual understanding of electric circuits in simulation only and simulation-laboratory contexts. Journal of research in science teaching, 48(1), 71-93. doi: 10.1002/tea.20386

Justi, R., & Gilbert, J. K. (2002). Models and modelling in chemical education. In O. de Jong, R. Justi, D. F. Treagust & J. H. van Driel (Eds.), Chemical education: Towards research-based practice (pp. 47-68). Dordrecht, The Netherlands: Kluwer Academic Publishers.

Klahr, D., Triona, L. M., & Williams, C. (2007). Hands on what? The relative effectiveness of physical versus virtual materials in an engineering design project by middle school children. Journal of Research in Science Teaching, 44(1), 183-203. doi: 10.1002/tea.20152

Kozma, R., & Russell, J. (2005a). Multimedia learning of chemistry. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 409-428). New York, NY: Cambridge University Press.

Kozma, R., & Russell, J. (2005b). Students becoming chemists: Developing representational competence. In J. Gilbert (Ed.), Visualization in science education (pp. 121-145). Dordrecht, Netherlands: Springer.

Luna, J. M. (2016). Pattern mining: Current status and emerging topics. Progress in Artificial Intelligence, 1-6. doi: 10.1007/s13748-016-0090-4

Manches, A., O’Malley, C., & Benford, S. (2010). The role of physical representations in solving number problems: A comparison of young children’s use of physical and virtual materials. Computers & Education, 54(3), 622-640.

McElhaney, K. W., Chang, H. Y., Chiu, J. L., & Linn, M. C. (2015). Evidence for effective uses of dynamic visualisations in science curriculum materials. Studies in Science Education, 51(1), 49-85. doi: 10.1080/03057267.2014.984506

Michalchik, V., Rosenquist, A., Kozma, R., Kreikemeier, P., & Schank, P. (2008). Representational resources for constructing shared understandings in the high school chemistry classroom. In J. K. Gilbert, M. Reiner & M. B. Nakhleh (Eds.), Visualization: Theory and practice in science education (pp. 233-282). Dordrecht, Netherlands: Springer.

Muller, M. (2014). Curiosity, creativity, and surprise as analytic tools: Grounded theory method. In J. S. Olson & W. A. Kellogg (Eds.), Ways of Knowing in HCI (pp. 25-48). New York, NY: Springer.

NRC. (2006). Learning to Think Spatially. Washington, D.C.: National Academies Press.

Olympiou, G., & Zacharia, Z. C. (2012). Blending physical and virtual manipulatives: An effort to improve students' conceptual understanding through science laboratory experimentation. Science Education, 96(1), 21-47. doi: 10.1002/sce.20463

Rau, M. A. (2015). Enhancing undergraduate chemistry learning by helping students make connections among multiple graphical representations. Chemistry Education Research and Practice, 16, 654-669. doi: 10.1039/C5RP00065C

Rau, M. A. (2016). Conditions for the effectiveness of multiple visual representations in enhancing STEM learning. Educational Psychology Review, 1-45. doi: 10.1007/s10648-016-9365-3

Rau, M. A., Aleven, V., & Rummel, N. (2015). Successful learning with multiple graphical representations and self-explanation prompts. Journal of Educational Psychology, 107(1), 30-46. doi: 10.1037/a0037211

Rau, M. A., Aleven, V., Rummel, N., & Pardos, Z. (2014). How should Intelligent Tutoring Systems sequence multiple graphical representations of fractions? A multi-methods study. International Journal of Artificial Intelligence in Education, 24(2), 125-161. doi: 10.1007/s40593-013-0011-7

Romero, C., J.M. Luna, J. M., J.R. Romero, J. R., & S. Ventura, S. (2011). RM-Tool: A framework for discovering and evaluating association rules. Advances in Engineering Software, 42(8), 566-576.

Roschelle, J. (1992). Learning by Collaborating: Convergent Conceptual Change. Journal of the Learning Sciences, 2(3), 235-276. doi: 10.1207/s15327809jls0203_1

Saldana, J. (2016). An introduction to codes and coding. In J. Seaman (Ed.), The Coding Manual for Qualitative Researchers (pp. 1-31). London, UK: SAGE Publications Ltd.

Schnotz, W. (2005). An integrated model of text and picture comprehension. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 49-69). New York, NY: Cambridge University Press.

Schnotz, W. (2014). An integrated model of text and picture comprehension. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2 ed., pp. 72-103). New York, NY: Cambridge University Press.

Seufert, T., & Brünken, R. (2006). Cognitive load and the format of instructional aids for coherence formation. Applied Cognitive Psychology, 20, 321-331. doi: 10.1002/acp.1248

Strickland, A. M., Kraft, A., & Bhattacharyya, G. (2010). What happens when representations fail to represent? Graduate students’ mental models of organic chemistry diagrams. Chemistry Education Research and Practice, 11(4), 293-301. doi: 10.1039/C0RP90009E

Tortosa, M. (2012). The use of microcomputer based laboratories in chemistry secondary education: Present state of the art and ideas for research-based practice. Chemistry Education Research and Practice, 13(3), 161-171.

Uttal, D. H., & O’Doherty, K. (2008). Comprehending and learning from ‘visualizations’: A developmental perspective. In J. Gilbert (Ed.), Visualization: Theory and practice in science education (pp. 53-72). Netherlands: Springer.

Vygotsky, L. S. (1978). Internalization of Higher Psychological Functions. In M. W. Cole, V. John-Steiner, S. Scribner & E. Souberman (Eds.), Mind in society (pp. 52-57). Cambridge, MA: Harvard University Press.

Wertsch, J. V. (1997). Properties of Mediated Action. In J. V. Wertsch (Ed.), Mind as Action (pp. 23-72). New York: Oxford University Press.

Wertsch, J. V., & Kazak, S. (2011). Saying more than you know in instructional settings. In T. Koschmann (Ed.), Theories of learning and studies of instructional practice (pp. 153-166). New York: Springer.

Winn, W., Stahr, F., Sarason, C., Fruland, R., Oppenheimer, P., & Lee, Y. L. (2006). Learning oceanography from a computer simulation compared with direct experience at sea. Journal of Research in Science Teaching, 43(1), 25-42.

Wu, H. K., Krajcik, J. S., & Soloway, E. (2001). Promoting understanding of chemical representations: Students' use of a visualization tool in the classroom. Journal of research in science teaching, 38(7), 821-842. doi: 10.1002/tea.1033

Zacharia, Z. C., Loizou, E., & Papaevripidou, M. (2012). Is physicality an important aspect of learning through science experimentation among kindergarten students? Early Childhood Research Quarterly, 27(3), 447-457.

Zhang, Z. H., & Linn, M. C. (2011). Can generating representations enhance learning with dynamic visualizations? Journal of Research in Science Teaching, 48(10), 1177-1198.




How to Cite

Rau, M. A. (2017). How do Students Learn to See Concepts in Visualizations? Social Learning Mechanisms with Physical and Virtual Representations. Journal of Learning Analytics, 4(2), 240–263.