Visualizing Data to Support Judgement, Inference, and Decision Making in Learning Analytics: Insights from Cognitive Psychology and Visualization Science

Sakinah S.J. Alhadad


Understanding human judgement and decision making during visual inspection of data is of both practical and theoretical interest. While visualizing data is a commonly employed mechanism to support complex cognitive processes such as inference, judgement, and decision making, the process of supporting and scaffolding cognition through effective design is less well understood. Applying insights from cognitive psychology and visualization science, this paper critically discusses the role of human factors — visual attention, perception, judgement, and decision making — toward informing methodological choices when visualizing data. The value of visualizing data is discussed in two key domains: 1) visualizing data as a means of communication; and 2) visualizing data as research methodology. The first applies cognitive science principles and research evidence to inform data visualization design for communication. The second applies data- and cognitive-science to deepen our understanding of data, of its uncertainty, and of analysis when making inferences. The evidence for human capacity limitations — attention and cognition — are discussed in the context of data visualizations to support inference-making in both domains, and are followed by recommendations. Finally, how learning analytics can further research on understanding the role data visualizations can play in supporting complex cognition is proposed.

Full Text:



Aguilar, S.J. (2017). Learning analytics: at the nexus of big data, digital innovation, and social justice in education. TechTrends.

Alhadad, S. S. J. (2016). Attentional and cognitive processing of analytics visualisations: Can design features affect interpretations and decisions about learning and teaching? In S. Barker, S. Dawson, A. Pardo, & C. Colvin (Eds.), Show Me The Learning. Proceedings ASCILITE (pp. 20-32). Adelaide: Australia.

Ali, N., & Peebles, D. (2013). The effect of Gestalt laws of perceptual organization on the comprehension of three-variable bar and line graphs. Human Factors, 55(1), 10.1177/0018720812452592

Anderson, B.A. & Yantis. S. (2013). Persistence of value-driven attentional capture. Journal of Experimental Psychology: Human Perception & Performance, 39, 6-9.

Anderson, B.A. (2016). The attention habit: How reward learning shapes attentional selection. Annals of the New York Academy of Sciences, 1369, 24-39.

Anscombe, F.J. (1973). Graphs in statistical analysis. The American Statistician, 27(1), 17-21.

Ansorge, U., Horstmann, G., & Scharlau, I. (2010). Top–down contingent attentional capture during feed-forward visual processing. Acta Psychologica, 135, 123–126.

Awh, E., Belopolsky, A.V., Theeuwes, J. (2012). Top-down versus bottom-up attentional control: A failed theoretical dichotomy. Trends in Cognitive Science, 16(8), 437-443,

Ayres, P., & Sweller, J. (2014). The split-attention principle in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 206–226). New York: Cambridge University Press.

Baldassi, S., Megna, N., & Burr, D.C. (2006). Visual clutter causes high-magnitude errors. PLOS: Biology, 4(3), 387-394.

Beck, D. M., & Kastner, S. (2008). Top-down and bottom-up mechanisms in biasing competition in the human brain. Vision Research, 49, 1154–1165.

Becker, S.I., Folk, C.L., Remington, R.W. (2010). The role of relational information in contingent capture. Journal of Experimental Psychology: Human Perception and Performance, 36(6), 1460-1476,

Becker, S.I., Lewis, A.J., & Axtens, J.E. (2017). Top-down knowledge modulates onset capture in a feedforward manner. Psychological Bulletin & Review, 24, 436-446.

Belia, S, Fidler, F, Williams, J, & Cumming, G (2005). Researchers misunderstand confidence intervals and standard error bars. Psychological Methods, 10(4), 389-396.

Bergner, Y. (2017). Measurement and its Uses in Learning Analytics. In C. Lang, G. Siemens, A. F. Wise, & Gašević, D. (Eds.) The Handbook of Learning Analytics (pp. 34–48). Society for Learning Analytics Research (SoLAR), Alberta, Canada.

Bewick, V., Cheek, L., & Ball, J. (2003). Statistics review 7: Correlation and regression. Critical Care, 7(6), 451-459.

Bishara, A.J. & Hittner, J.B. (2017). Confidence intervals for correlations when data are not normal. Behavior Research Methods, 49(1), 294-309.

Bonneau, G.P., Hege, H., Johnson, C.R., Oliveira, M.M., Potter, K., Rhenigans, P., & Schultz, T. (2014). Overview and State-of-the-Art of Uncertainty Visualization. In: C. Hansen, M. Chen, C. Johnson, A. Kaufman, & H. Hagen (Eds.) Scientific Visualization. Mathematics and Visualization (pp. 3-27). Springer, London.

Braithwaite, D. W., & Goldstone, R. L. (2013). Flexibility in data interpretation: effects of representational format. Frontiers in Psychology, 4, 980.

Burton, O., Pomeroy, D., Radenovic, V., & McCarley, J.S. (2017). Visualization of uncertainty aids spatial judgements but fails to improve metacognitive efficiency. Proceedings of the Human Factors & Ergonomics Society Annual Meeting, 61, 1390-1393.

Carpenter, P. A., & Shah, P. (1998). A model of the perceptual and conceptual processes in graph comprehension. Journal of Experimental Psychology: Applied, 4(2), 75-100.

Cierniak, G., Sceiter, K., & Gerjets, P. (2009). Explaining the split-attention effect: Is the reduction of extraneous cognitive load accompanied by an increase in germane cognitive load? Computers in Human Behavior, 225(2), 315-324.

Cleveland, W.S., Diaconis, P., & McGill, R. (1982). Variables on scatterplots look more highly correlated when the scales are increased. Science, 216(4550), 1138-1141.

Connor, C.E., Egeth H.E., & Yantis S. (2004). Visual attention: bottom-up versus top-down. Current Biology, 14(19), R850-R852.

Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), 7-29.

Cumming, G., Fidler, F., & Vaux, D. L. (2007). Error bars in experimental biology. The Journal of Cell Biology, 177(1), 7–11.

Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18, 193–222.

Fisher, D., Popov, I., Drucker, S.M., & Scraefel, M. (2012). Trust me, I’m partially right: Incremental visualization lets analysts explore large datasets faster. In Proceedings of the 2012 CHI Conference on Human Factors in Computing Systems. Austin, Texas: USA.

Gelman, A., & Hennig, C. (2017). Beyond subjective and objective in statistics. Journal of the Royal Statistical Society: Statistics in Society Series A, 180, 1-31.

Gelman, A., Pasarica, C., & Dodhia, R. (2002). Let’s practice what we preach: Turning tables into graphs. The American Statistician, 56(2), 121-130.

Gigerenzer, G., & Edwards, A. (2003). Simple tools for understanding risks: from innumeracy to insight. BMJ : British Medical Journal, 327(7417), 741–744.

Gobet, F. (2005). Chunking models of expertise: Implications for education. Applied Cognitive Psychology. 19, 183-204.

Gobet, F., Lane, P.C.R., Croker, S., Cheng, P.C.H., Jones, G., Oliver, I., & Pine, J.M. (2001). Chunking mechanisms in human learning. Trends in Cognitive Sciences, 5(6, 236–243.

Goldstein, D. G., & Gigerenzer, G. (2002). Models of ecological rationality: The recognition heuristic. Psychological Review, 109(1), 75-90.

Gottlieb, D.A., Weiss, T., Chapman, G.B. (2007). The format in which uncertainty information is presented affects decision biases. Psychological Science, 18(3), 240-246.

Hair, J.F., Black, W.C., Babin, B.J., Anderson, R.E. (2010). Multivariate data analysis. (5th ed.). USA: Pearson Prentice Hall.

Hawkins, D.L. (1989). Using U statistics to derive the asymptotic distribution of Fisher’s Z statistic. American Statistician, 43, 235–237.

Hegarty, M. (2011). The Cognitive Science of Visual-Spatial Displays: Implications for Design. Topics in Cognitive Science, 3, 446–474.

Hegarty, M., Smallman, H. S., Stull, A. T., & Canham, M. (2009). Naïve Cartography: How intuitions about display configuration can hurt performance. Cartographica, 44, 171–186.

Hoekstra, R., Morey, R.D., Rouder, J.N., & Wagenmakers, E. (2014). Robust misinterpretation of confidence intervals. Psychonomic Bulletin, 21(5), 1157-1164.

Jarmasz, J. & Hollands, J.G. (2009). Confidence intervals in repeated-measures designs: The number of observations principle. Canadian Journal of Experimental Psychology, 63(2), 124-138.

Kadel, H., Feldmann-Wüstefeld, T., & Schubö, A. (2017). Selection history alters attentional filter settings persistently beyond top-down control. Psychophysiology, 54(5), 736-754.

Kahneman, D. (2003). Maps of bounded rationality: Psychology for behavioral economics. The American Economic Review, 93(5), 1449-1475.

Kastellec, J.P., & Leoni, E.L. (2007). Using graphs instead of tables in political science. Perspectives on Politics, 5(4), 755-771.

Kirschner, P.A., Ayres, P., & Chandler, P. (2011). Contemporary cognitive load theory research: The good, the bad, and the ugly. Computers in Human Behavior, 27(1), 99-105.

Kosslyn, S. M. (2006). Graph design for the eye and mind. New York: Oxford University Press.

Kramer, R.S.S., Telfer, C.G.R., & Towler, A. (2017). Visual comparison of two data sets: Do people use the means and the variability? Journal of Numerical Cognition, 3(1), 97-111.

Krämer, W., & Gigerenzer, G. (2005). How to confuse with statistics or: The use and misuse of conditional probabilities. Statistical Science, 20(3), 223-230.

Krzywinski, M., & Altman, N. (2014). Points of significance: Visualizing samples with box plots. Nature Methods, 11, 1038/nmeth.2813

Kubina Jr., R.M., Kostewicz, D.E., Brennan, K.M., & King, S.A. (2017). A critical review of line graphs in behavior analytics journals. Educational Psychology Review, 29, 583-598.

Le Pelley, M.E., Pearson, M., Griffiths, O., & Beesley, T. (2015). When goals conflict with values: counterproductive attentional and oculomotor capture by reward-related stimuli. Journal of Experimental Psychology: General, 144(1), 158–171.

Lem, S., Onghena, P., Verschaffel, L., & Van Dooren, W. (2013). On the misinterpretation of histograms and box plots. Educational Psychology, 33(2), 155-174,

Lem, S., Onghena, P., Verschaffel, L., & Van Dooren, W. (2014). European Journal of Psychology of Education, 29(4), 557-575.

Lodge, J.M., Alhadad, S.S.J., Lewis, M.J., & Gašević, D. (2017). Inferring Learning from Big Data: The Importance of a Transdisciplinary and Multidimensional Approach. Technology, Knowledge & Learning, 22(3), 385-400.

Louis, T.A., & Zeger, S.L. (2009). Effective communication of standard errors and confidence intervals. Biostatistics, 10(1), 1-2.

Loy, A., Follett, L., & Hofmann, H. (2016). Variations of Q-Q plots: The power of our eyes! Statistical Computing & Graphics, 70(2), 202-214.

Matejka, J., & Fitzmaurice, G. (2017a). Same stats, different graphs: generating datasets with varied appearance and identical statistics through simulated annealing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 1290–1294). Denver, CO, USA.

Matejka, J., & Fitzmaurice, G. (2017b). The datasaurus dozen - same stats, different graphs: Generating datasets with varied appearance and identical statistics through simulated annealing. Retrieved from

Matejka, J., Glueck, M., Grossman, T., & Fitzmaurice, G. (2016). The effect of visual appearance on the performance of continuous sliders and visual analogue scales. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 5421–5432). San Jose, CA, USA.

Matzen, L.E., Haass, M.J., Divis, K.M., & Stites, M.C. (2017). Patterns of attention: How data visualizations are read. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 10284, 176-191.

Mautone, P.D., & Mayer, R.E. (2007). Cognitive aids for guiding graph comprehension. Journal of Educational Psychology, 99(3), 640-652.

Mayer, R.E., & Fiorella, L. (2014). Principles for reducing extraneous processing in multimedia learning: Coherence, signaling, redundancy, spatial contiguity, and temporal contiguity principles. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 279-315). New York: Cambridge University Press.

Meyer, B. J. F. (1975). The Organization of Prose and Its Effects on Memory. Elsevier, New York.

Meyer, J., Shamo, M. K., & Gopher, D. (1999). Information structure and the relative efficacy of tables and graphs. Human Factors: The Journal of the Human Factors and Ergonomics Society, 41, 570-587.

Moore, P., & Fitz, C. (1993). Gestalt theory and instructional design. Journal of Technical Writing & Communication, 23(2), 137-157.

Morey, R.D., Hoekstra, R., Rouder, J.N., Lee, M.D., & Wagenmakers, E. (2016). The fallacy of placing confidence in confidence intervals. Psychological Bulletin Review, 23(1), 103-123.

Newman, G.E. & Scholl, B.J. (2012). Bar graphs depicting averages are perceptually misinterpreted: The within-the-bar bias. Psychonomic Bulletin & Review, 19(4), 601-607.

Nicholls, A. (2016). Confidence limits, error bars and method comparison in molecular modeling. Part 2: Comparing methods. Journal of Computer-Aided Molecular Design, 30 (2), 103-126.

O’Brien, F. & Cousineau, D. (2014). Representing error bars in within-subjects designs in typical software packages. The Quantitative Methods for Psychology, 10(1), 56-67.

Okan Y., Galesic M., Garcia-Retamero R. (2016). How people with low and high graph literacy process health graphs: Evidence from eye-tracking. Journal of Behavioral Decision Making, 29, 271–294.

Olston, C. and Mackinlay, J.D. (2002). Visualizing data with bounded uncertainty. Proceedings of the IEEE Symposium on Information Visualization (pp. 37–40). Los Alamitos, CA: IEEE Computer Society.

Paas, F., & Sweller, J. (2014). Implications of cognitive load theory for multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 27-42). New York: Cambridge University Press.

Pastore, M., Lionetti, F., & Altoe, G. (2017). When one shape does not fit all: A commentary essay on the use of graphs in psychological research. Frontiers in Psychology, 8(1666).

Peebles, D., & Ali, N. (2015). Expert interpretation of bar and line graphs: the role of graphicacy in reducing the effect of graph format. Frontiers in Psychology, 6, 1673.

Pentoney, C.S., & Berger, D.E. (2016). Confidence intervals and within-the-bar bias. The American Statistician, 70(2), 215-220.

Pinker, S. (1990). A theory of graph comprehension. I.L. Freedle, (Ed.), Artificial Intelligence and the Future of Testing, (pp. 73-126). Hiltdale, NI: Lawrence Erlbaum Associates

Pinto, Y., van der Leij, A.R., Sligte, I.G., Lamme, V.A., & Scholte, S. (2013). Bottom-up and top-down attention are independent. Journal of Vision, 13(3), 1-14,

Porat, T., Oron-Gilad, T., & Meyer, J. (2009). Task-dependent processing of tables and graphs. Behaviour & Information Technology, 28(3), 293-307.

Ramamurthy, M., & Blaser, E. (2017). New rules for visual selection: Isolating procedural attention. Journal of Vision, 17(2):18, 1-16.

Rosenholtz R., Li Y., & Nakano L. (2007). Measuring visual clutter. Journal of Vision 7(2), 17–22.

Scaife, M., & Rogers, Y. (1996). External cognition: How do graphical representations work? International Journal of Human-Computer Studies, 45, 115–143.

Scown, H., Bartlett, M., & McCarley, J.S. (2014). Statistically lay decision makers ignore error bars in two-point comparisons. Proceedings of the Human Factors & Ergonomics Society Annual Meeting, 58, 1746-1750.

Shah, P. (1997). A model of the cognitive and perceptual processes in graphical display comprehension. In M. Anderson (Ed.), Reasoning with diagrammatic representations (pp. 94–101). Menlo Park, CA: AAAI Press.

Shah, P., & Freedman, E.G. (2009). Bar and line graph comprehension: An interaction of top-down and bottom-up processes. Topics in Cognitive Science, 3(3), 1-19.

Shah, P., & Hoeffner, J. (2002). Review of graph comprehension research: Implications for instruction. Educational Psychology Review, 14(1). 47-69.

Shah, P., Mayer, R.E., & Hegarty, M. (1999). Graphs as Aids to Knowledge Construction: Signaling Techniques for Guiding the Process of Graph Comprehension. Journal of Educational Psychology, 91(4), 690-702.

Sher, V., Bemis, K. G., Liccardi, I. & Chen, M. (2017). An empirical study on the reliability of perceiving correlation indices using scatterplots. Computer Graphics Forum, 36, 61–72.

Skeels, M., Lee, B., Smith, G., & Robertson, G.G. (2010). Revealing uncertainty for information visualization. Information Visualization, 9, 70-81.

Smallman, H. S., & St. John, M. (2005). Naïve realism: Misplaced faith in realistic displays. Ergonomics in Design, 13, 14–19.

Stofer, K. & Che, X. (2014). Comparing experts and novices on scaffolded data visualizations using eye-tracking. Journal of Eye Movement Research, 7(5), 1-15.

Strahan, R. F., & Hansen, C. J. (1978). Underestimating correlation from scatterplots. Applied Psychological Measurement, 2, 543-550.

Sui, J. & Humphreys, G. (2016). Introduction to special issue: Social attention in mind and brain. Cognitive Neuroscience, 7(1-4), 1-4.

Sweller, J., van Merriënboer, J.J.G., & Paas, F.G.W.C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251-296.

Tak, S., Toet, A., & van Erp, J. (2014). The perception of visual uncertainty representation by non-experts. IEEE Transactions on Visualization & Computer Graphics, 20, 10.1109/TVCG.2013.247

Theeuwes, J. (2010). Top-down and bottom-up control of visual selection. Acta Psychologica, 135(2), 77-99.

Thudt, A., Walny, J., Perin, C., Rajabiyazdi, F., MacDonald, L., Vardeleon, R., … Carpendale, S. (2016). Assessing the Readability of Stacked Graphs. In Proceedings of the 2016 Graphics Interface Conference.(pp. 167-174). Victoria, BC, Canada.

Trafton, J.G., Kirschenbaum, S.S., Tsui, T.L., Miyamoto, R.T., Ballas, J.A., & Raymond, P.D. (2000). Turning pictures into numbers: Extracting and generating information from complex visualizations. International Journal of Human-Computer Studies, 53(5), 827-850.

Trafton, J.G., Trickett, S.B., & Mintz, F.E. (2005). Connecting internal and external representations: Spatial transformations of scientific visualizations. Foundations of Science, 10, 89-106.

van Zoest, W., Donk, M., & Theeuwes, J. (2004). The role of stimulus-driven and goal-driven control in saccadic visual selection. Journal of Experimental Psychology: Human Perception and Performance, 30, 746–759.

van Zoest, W., Van der Stigchel, S., & Donk. M. (2017). Conditional control in visual selection. Attention, Perception, & Psychophysics, 79, 1555-1572.

Vessey, I., & Galletta, D. (1991). Cognitive fit: An empirical study of information acquisition. Information Systems Research, 2(12), 63-84.

Ward, M.O., Grinstein, G., & Keim, D. (2015). Interactive Data Visualization: Foundations, Techniques, and Applications, Second Edition, CRC Press. ProQuest Ebook Central.

Wilkinson, L., & Task Force on Statistical Inference (1994). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54 (8), 594-604.

Wolfe, J.M. (1994). Guided Search 2.0: A revised model of visual search. Psychonomic Bulletin & Review, 1(2), 202–238.

Woodward, J. F. (2011). Data and phenomena: A restatement and defense. Synthese, 182(1), 165–179.

Yantis, S. & Johnston, J.C. (1990). On the locus of visual selection: Evidence from focused attention tasks. Journal of Experimental Psychology: Human, Perception, & Performance, 16(1), 135-149.

Zacks, J. & Tversky, B. (1999). Bars and lines: A study of graphic communication. Memory & Cognition, 27(6), 1073-1079.

Zacks, J., Levy,E., Schiano, D.J., & Tversky, B. (1998). Reading bar graphs: Effects of extraneous depth cues and graphical context. Journal of Experimental Psychology: Applied, 4(2), 119-138.

Zouaq, A., Jovanović,J., Joksimović, S., & Gašević, D. (2017). In C. Lang, G. Siemens, A. F. Wise, & Gašević, D. (Eds.) The Handbook of Learning Analytics (pp. 347–355). Society for Learning Analytics Research (SoLAR), Alberta, Canada.

Zuk T., & Carpendale S. (2007). Visualization of Uncertainty and Reasoning. In: Butz A., Fisher B., Krüger A., Olivier P., Owada S. (eds) Smart Graphics. SG 2007. Lecture Notes in Computer Science, 4569 (pp. 164-177). Springer, Berlin, Heidelberg.



Share this article: