What Constitutes an ‘Actionable Insight’ in Learning Analytics?

Rasmus Leth Jørnø

Abstract


The possibilities of Learning Analytics as a tool for empowering teachers and educators have created a steep interest in how to provide so-called actionable insights. However, the literature offers little in the way of defining or discussing what the term “actionable insight” means. This selective literature review provides a look into the use of the term in current literature. The review points to a dominant perspective in the literature that assumes the perspective of a rational actor, where actionable insights are treated as insights mined from data and subsequently acted upon. It also finds evidence of other perspectives and discusses the need for clarification of the term in order to establish a more precise and fruitful use of the term.


Full Text:

PDF

References

Arnold, K. E., & Pistilli, M. D. (2012). Course Signals at Purdue: Using Learning Analytics to Increase Student Success. In S. Buckingham Shum, D. Gašević, and R. Ferguson (Eds.). International Conference on Learning Analytics and Knowledge. ACM Press, pp. 267–270.

Atif, A., Richards. D., Bilgin, A. & Marrone, M. (2013). A Panorama of Learning Analytics Featuring the Technologies for the Learning and Teaching Domain. In H. Carter, M. Gosper and J. Hedberg (Eds.). Electric Dreams. Proceedings ascilite 2013 Sydney, pp.68-72.

Atkinson, S.P. (2015). Adaptive Learning and Learning Analytics: a new learning design paradigm. Retrieved from: http://i.unisa.edu.au/siteassets/staff/tiu/documents/adaptive-learning-and-learning-analytics—a-new-learning-design-paradigm.pdf

Avella, J.T., Kebritchi, M., Nunn, S.G. & Kanai, T. (2016). Learning analytics methods, benefits, and challenges in higher education: A systematic literature review. Online Learning Consortium: Special Issue on Learning Analytics, 20(2), pp. 13-29.

Baker, R. S., & Siemens, G. (2014). Educational data mining and learning analytics. In R. Sawyer (Ed). The Cambridge handbook of the learning sciences. Cambridge University Press, pp. 253–272.

Baker, R.S.J.D. & Yacef, K. (2009). The state of educational data mining in 2009: a review and future visions. Journal of Educational Data Mining, 1(1), pp. 3-16.

Baker, R.S., Corbett, A.T., Koedinger, K.R. (2004). Detecting Student Misuse of Intelligent

Tutoring Systems. Proceedings of the 7th International Conference on Intelligent Tutoring

Systems, pp. 531-540.

Baker, R.S.J.d., Corbett, A.T., Gowda, S.M., Wagner, A.Z., MacLaren, B.M., Kauffman, L.R.,

Mitchell, A.P., Giguere, S. (2010). Contextual Slip and Prediction of Student Performance After Use of an Intelligent Tutor. Proceedings of the 18th Annual Conference on User Modeling, Adaptation, and Personalization, pp. 52-63.

Beck, J.E., Chang, K.-m., Mostow, J., Corbett, A.T. (2008) Does help help? Introducing the

Bayesian evaluation and assessment methodology. Proceedings of Intelligent Tutoring

Systems, ITS 2008, pp. 383–394.

Bichsel, J. (2012). Analytics in Higher Education: Benefits, Barriers, Progress, and Recommendations. Louisville, CO: EDUCAUSE Center for Applied Research. Available from http://www.educause.edu/ecar.

Blikstein, P. (2013). Multimodal learning analytics. Proceedings of the Third Conference on Learning Analytics and Knowledge (LAK’13), pp. 102–106. http://dx.doi.org/10.1145/2460296.2460316

Bowers, A.J. (2010). Analyzing the Longitudinal K-12 Grading Histories of Entire Cohorts of

Students: Grades, Data Driven Decision Making, Dropping Out and Hierarchical Cluster

Analysis. Practical Assessment, Research & Evaluation (PARE), 15(7), pp. 1-18.

Brown, M. (2011). Learning analytics: The coming third wave. Educause Learning Initiative Brief. Retrieved on 22/6-2017 from https://net.educause.edu/ir/library/pdf/ELIB1101.pdf

Brown, M. (2012). Learning Analytics: Moving from Concept to Practice. EDUCAUSE Learning Initiative. http://net.educause.edu/ir/library/pdf/ELIB1203.pdf

Brown, M., Dehoney, J. & Millichap, N. (2015). The next generation digital learning environment - a report on research. Educause Learning Initiative.

Buerck, J.P. & Mudigonda, S.P. (2014). A resource-constrained approach to implementing analytics in an institution of higher education: an experience report. Journal of Learning Analytics, 1(1), pp. 129-139.

Campbell, J.P. & Oblinger, D.G. (2007). Academic Analytics. EDUCAUSE Quarterly. October.

Campbell, J.P., DeBlois, P. & Oblinger, D.G. (2007). Academic Analytics: A New Tool

for a New Era. EDUCAUSE Review, 42(4), pp. 40–57.

Chatti, M. A., Dyckhoff, A. L., Schroeder, U., & Thüs, H. (2012). A reference model for learning analytics. International Journal of Technology Enhanced Learning, 4(5), pp. 318-331.

Claxton, G. (2006). Expanding the capacity to learn: A new end for education? British Educational Research Association Annual Conference, 6 september 2006, Warwick University.

Clow, D. (2012). The learning analytics cycle: closing the loop effectively. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge - LAK ’12, pp. 134-137.

Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education, 18(6), pp. 683-695.

Clow, D. (2014). Data wranglers: Human interpreters to help close the feedback loop. Proceedings of the Fourth International Conference on Learning Analytics And Knowledge LAK ’14, ACM Press, pp. 49–53.

Colvin, C., Rogers, T., Wade, A., Dawson, S., Gasevic, D., Buckingham Shum, S., Nelson, K., Alexander, S., Lockyer, L., Kennedy, G., Corrin, L., & Fisher, J. (2016). Student retention and learning analytics: a snapshot of Australian practices and a framework for advancement. Canberra, ACT: Australian Government Office for Learning and Teaching. Website accessed 15 July 2016: http://he-analytics.com

Cooper, A. (2012a). A Brief History of Analytics. JISC CETIS Analytics Series, 1(9). University of Bolton.

Cooper, A. (2012b). What is analytics? Definition and essential characteristics. JISC CETIS Analytics Series, 1(5). University of Bolton.

Cooper, A. (2012c). A framework of characteristics for analytics. JISC CETIS Analytics Series, 1(7). University of Bolton.

Davenport, T.H. (2006). Competing on analytics. Harward Business Review, 84(1), pp. 98-107.

Davenport, T.H., Harris, J.G. & Morrison, R. (2010). Analytics at work: smarter decisions, better results. Boston, MA: Harvard Business School Publishing Corporation.

Dawson, S.P., Macfadyen, L. & Lockyer, L. (2009). Learning or performance: predicting driver of student motivation. In R. Atkinson & C. McBeath (Eds.). Same places, different spaces. Auckland, NZ: Ascilite, pp. 184-193.

Deakin Crick, R. (2007). Learning how to learn: the dynamic assessment of learning power. The Curriculum Journal, 18 (2), pp. 135-153.

Deakin-Crick, R., Huang, S., Shafi, A. & Goldspink, C. (2015). Developing Resilient Agency in Learning: The Internal Structure of Learning Power. British Journal of Educational Studies, 63 (2). pp. 121-160.

Dede, C. Ho, A. & Mitros, P. (2016). Big data analysis in higher education: promises and pitfalls. Educause Review, sept./oct. 2016, pp. 22-34.

De Laat, M., & Schreurs, B. (2013). Visualizing informal professional development networks building a case for learning analytics in the workplace. American Behavioral Scientist, 57(10), pp. 1421–1438.

Dweck, C. S. (2006). Mindset: The new psychology of success. New York: Random House.

Dyckhoff, A. L., Zielke, D., Bültmann, M., Chatti, M. A., & Schroeder, U. (2012). Design and Implementation of a Learning Analytics Toolkit for Teachers. Educational Technology & Society, 15 (3), pp. 58–76.

Elias, T. (2011). Learning analytics: definitions, processes and potential. retrieved from http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf

Essa, A. & Ayad, H. (2012). Student Success System: Risk Analytics and Data Visualization using Ensembles of Predictive Models. The 1st Learning Analytics and Knowledge Conference, Vancouver, British Columbia.

Ferguson, R. (2012a). Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6) pp. 304–317.

Ferguson, R. (2012b). The State Of Learning Analytics in 2012: A Review and Future Challenges. Technical Report KMI-12-01, Knowledge Media Institute, The Open University, UK.

Ferguson, R. (2016). Student retention and learning analytics: a snapshot of Australian practices and a framework for advancement (LAEP inventory). A Cloudworks synopsis. Accessed online on http://cloudworks.ac.uk/cloud/view/9673

Ferguson, R. and Buckingham Shum, S. (2012). Social Learning Analytics: Five Approaches. Proceedings of the 2nd International Conference on Learning Analytics & Knowledge. ACM Press: New York, pp. 23-33.

Ferguson, R., Brasher, A., Clow, D., Cooper, A., Hillaire, G., Mittelmeier, J., Rienties, B., Ullmann, T., Vuokari, R. (2016). Research evidence on the use of learning analytics - implications for education policy. correR. Vuokari, J. Castaño Muñoz, J. (eds.) Joint Research Center Science for Policy Report, EUR 28294.

Fernandez-Gallego, B., Lama, M., Vidal, J.C. & Mucientes, M. (2013). Learning analytics Framework for educational virtual worlds. Procedia Computer Science 25, 2013 International Conference on Virtual and Augmented Reality in Education, pp. 443-447.

Fiaidhi, J. (2014). The next step for learning analytics. IT Pro, IEEE Computer Society, sept./oct. 2014, pp. 4-8.

Gašević, D., Dawson, S., Pardo, A. (2016). How do we start? State and Directions of Learning Analytics Adoption. Oslo, Norway: International Council for Open and Distance Education.

Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2014). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting learning success. Internet and Higher Education, 28, pp. 68-84.

Gašević, D., Dawson, S. & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), pp. 64-71.

Gibbs, G. (2010). Dimensions of quality. York, UK: The Higher Education Academy. Retrieved from http://www.heacademy.ac.uk/resources/detail/evidence_informed_practice/Dimensions_of_Quality

Goldstein, P.J., & Katz, R.N. (2005). Academic analytics: The uses of management information and technology in higher education. Boulder, CO: EDUCAUSE Center for Applied Research.

Greller, W., & Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning Analytics. Educational Technology & Society, 15 (3), pp. 42–57.

Gunn, C., McDonald, J., Donald, C., Milne, J., Nichols, M., & Heinrich, E. (2015). A practitioner’s guide to learning analytics. In T. Reiners, B.R. von Konsky, D. Gibson, V. Chang, L. Irving, & K. Clarke (Eds.). Globally connected, digitally enabled. Proceedings ascilite 2015 in Perth, pp. 672-675.

Hall Jr., O.P. (2014). Teaching and using analytics in management education. In J. Wang (ed.) Encyclopedia of Business Analytics and Optimization. PA, USA: IGI Publishing Hershey, pp. 2479-2491.

Haythornthwaite, C., de Laat, M., & Dawson, S. (2013). Introduction to the Special Issue on Learning Analytics. American Behavioral Scientist, 57(10), 1371-1379.

Hershkovitz, A., Knight, S., Dawson, S., Jovanović, J. & Gašević, D. (2016). About “learning” and “analytics”. Journal of Learning Analytics, 3(2), pp. 1-5. http://dx.doi.org/10.18608/jla.2016.32.1

HM Government (2013). Seizing the data opportunity. Policy Paper, UK Department for Business, Innovation & Skills. Retrieved from https://www.gov.uk/government/publications/uk-data-capability-strategy

Ice, P., Stephens-Helm, J. & Powell, K. (2011). Introduction to Analytics for E-Learning. In C. Ho & M. Lin (Eds.). Proceedings of E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2011, Association for the Advancement of Computing in Education (AACE), pp. 684-686.

Jacobson, M. (2016). The third wave of edtech: bringing the future of learning to today’s classes. Blogpost accessed 22/6-2017 at

http://staging.openforum.com.au/the-third-wave-of-edtech-bringing-the-future-of-learning-to-todays-classes/

Jayaprakash, S.M., Moody, E.W., Lauria, E., Regan, J.R. & Baron, J.D. (2014). Early alert of academically at-risk students: An open source analytics initiative. The Journal of Learning Analytics, 1(1), pp. 6-47.

Jones, D., Beer, C. & Clark, D. (2013). The IRAC framework: locating the performance zone for learning analytics. Proceedings of the 30th ascilite conference, pp. 446-450.

Khalil, M., & Ebner, M. (2015). Learning Analytics: Principles and Constraints. Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications, pp. 1326-1336.

Kitchenham, B. & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. Keele University and Durham University.

Klug, J., Ogrin, S. & Keller, S. (2011). A plea for self-regulated learning as a process: Modelling, measuring and intervening. Psychological Test and Assessment Modeling, 53(1), pp. 51–72.

Kumar, V.S., Kinshuk, Clemens, C. & Harris, S. (2015). Causal models and big data learning analytics. In Kinshuk & R. Huang (eds.) Ubiquitous learning environments and technologies. New York: Springer. pp. 31-54.

LAK (2011). Definition of learning analytics from the 1st international conference on learning analytics and knowledge. Available at https://tekri.athabascau.ca/analytics/call-papers

Liu, D.Y., Bartimote-Aufflick, K., Pardo, A. & Bridgeman, A.J. (2017). Data-driven personalization of student learning support in higher education. In A. Peña-Ayala, A. (ed.). Learning analytics: fundaments, applications, and trends - a view of the current state of the art to enhance e-learning. Cham, Switzerland: Springer, pp. 143-170.

Lockyer, L., Heathcote, E. & Dawson, S. (2013). Informing pedagogical action: Aligning Learning Analytics with Learning Design. American Behavioral Scientist, 57(19), pp. 1439-1459.

Long. P.D. & Siemens, G. (2011). Penetrating the fog: analytics in learning and education. EDUCAUSE Review, sept. 2011.

Lonn, S., Krumm, A.E., Waddington, R.J. & Teasley, S.D. (2012). Bridging the gap from knowledge to action: putting analytics in the hands of academic advisors. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, New York, NY: ACM, pp. 184-187.

Macfadyen, L. P., & Dawson, S. (2012). Numbers are not enough: Why e–learning analytics failed to inform an institutional strategic plan. Educational Technology & Society, 15(3), pp. 149–163.

Macfadyen, L.P., Dawson, S., Pardo, A. & Gašević, D. (2014). Embracing big data in complex educational systems: The learning analytics imperative and the policy challenge. Research & Practice in Assessment, 9, pp. 17-28.

Marzouk, Z., Rakovic, M., Liaqat, A., Vytasek, J., Samadi, D., Stewart-Alonso, J., Ram, I., Woloshen, S., Winne, P. & Nesbit, J. (2016). What if learning analytics were based on learning science? Australasian Journal of Educational Technology, 32(6), pp. 1-18.

Miliron, M.D., Malcom, L. & Kil, D. (2014). Insight and action analytics: Three case studies to consider. Research and Practice in Assessment, 9, pp. 70-89.

Moissa, B. Gasparini, I. & Kemczinski, A. (2015). A systematic mapping on the learning analytics field and its analysis in the massive open online courses context. International Journal of Distance Education Technologies, 13(3), pp. 1-24.

Nguyen, Q., Rienties, B. & Toetenel, L. (2017). Unravelling the dynamics of instructional practice: A longitudinal study on learning design and vle activities. ACM International Conference Proceeding Series. New York, NY: ACM, pp. 168-177.

Nguyen, Q., Tempelaar, D., Rienties, B. & Giesbers, B. (2016). What learning analytics based prediction models tell us about feedback preferences of students. The Quarterly Review of Distance Education, 17(3) pp. 13–33.

Norris, D.M. & Baer, L.L. (2013). Building organizational capacity for analytics. EDUCAUSE

Ocumpaugh, J., Baker, R., San Pedro, M., Hawn, M., Heffernan, C., Heffernan, N. & Slater, S. (2017). Guidance counselor reports of the ASSISTments college prediction model (ACPM). Proceedings of the Seventh International Learning Analytics & Knowledge Conference. New York, NY: ACM, pp. 479-488.

Pardo, A., Dawson, S. Gašević, D. & Steigler-Peters, S. (2016). The role of Learning Analytics in future education models. Technical report, Telstra. Available at: https://www.telstra.com.au/content/dam/tcom/business-enterprise/industries/pdf/tele0126_whitepaper_5_spreads_lr_notrims.pdf

Pardo, P., Mirriahi, N., Martinez-Maldonado, R., Jovanović, J., Dawson, S., Gašević (2016). Generating Actionable Predictive Models of Academic Performance. Proceedings of the 6th International Conference on Learning Analytics & Knowledge (LAK 2016). Edinburgh, Scotland, UK, 2016, pp. 474-478.

Picciano, A.G. (2012). The evolution of big data and learning analytics in american higher education. Journal of asynchronous learning networks, 16(3), pp. 9-20.

Piety, P.J., Hickey, D.T. & Bishop, M.J. (2014). Educational data sciences - framing emergent practices for analytics of learning, organizations, and systems. Proceedings of the 4th International Conference on Learning Analytics and Knowledge, LAK14, pp. 193-202.

Polikoff, M. S. (2010). Instructional sensitivity as a psychometric property of assessments. Educational Measurement: Issues and Practice, 29, pp. 3–14.

Prieto, L.P., Sharma, K., Dillenbourg, P. & Jesús, M. (2016). Teaching analytics: towards automatic extraction of orchestration graphs using wearable sensors. Proceedings of the Sixth International Conference on Learning Analytics & Knowledge. New York, NY: ACM, pp. 148-157.

Prinsloo, P. & Slade, S. (2014). Educational triage in open distance learning - walking a moral tightrope. The International Review of Research in Open and Distance Learning, 15(4), pp. 306-331.

Sabourin, J., Rowe, J., Mott, B., Lester, J. (2011). When Off-Task in On-Task: The Affective

Role of Off-Task Behavior in Narrative-Centered Learning Environments. Proceedings of the

th International Conference on Artificial Intelligence in Education, pp. 534-536.

Saqr, M., Fors, U. & Tedre, M. (2017). How learning analytics can early predict under-achieving students in a blended medical education course. Medical Teacher, 18. Elsevier, pp. 1-11.

Schneider, D., Class, B., Benetos, K. & Lange, M. (2012). Requirements for learning scenario and learning process analytics. In T. Amiel & B. Wilson (Eds.). Proceedings of EdMedia: World Conference on Educational Media and Technology 2012, Association for the Advancement of Computing in Education (AACE), (pp. 1632-1641).

Schön, D. (1983). The reflective practitioner: how professionals think in action. New York: Basic Books.

Sclater, N. (2014). Code of practice for learning analytics - a literature review of the ethical and legal issues. JISC, Bristol. Retrieved from http://repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf

Sclater, N., Peasgood, A. & Mullan, J. (2016). Learning analytics in higher education - a review of UK and international practice. JISC, Bristol. Retrieved from https://www.jisc.ac.uk/sites/default/files/learning-analytics-in-he-v2_0.pdf

Sclater, N., Webb, M. & Danson, M. (2016). The future of data-driven decision-making. JISC report. Available at https://www.jisc.ac.uk/reports/the-future-of-data-driven-decision-making

Sergis, S. & Sampson, D.G. (2017). Teaching and learning analytics to support teacher inquiry: A systematic literature review. In A. Peña-Ayala, A. (ed.) Learning analytics: fundaments, applications, and trends - a view of the current state of the art to enhance e-learning. Cham, Switzerland: Springer, pp. 25-63.

Shacklock, X. (2016). From bricks to clicks - the potential of data and analytics in higher education. Higher Education Commission report. Retrieved from http://www.policyconnect.org.uk/hec/sites/site_hec/files/he_commission_press_release_-_from_bricks_to_clicks.pdf

Siemens, G. & Long, P. (2011). Penetrating the fog - analytics in learning and education. Educause review, september/october, 46(5), pp. 30-40.

Strader, R., & Thille, C. (2012). The open learning initiative: Enacting instruction online. In D.G. Oblinger (Ed.). Game Changers: Education and Information Technologies. Educause, pp. 201–213.

Sundorph, E. & Mosseri-Marlio, W. (2016). Smart campuses: how big data will transform higher education. Reform report, London:UK. Retrieved at http://www.reform.uk/publication/smart-campuses/

Swan K. (2012). A special issue on learning analytics. Journal of Asynchronous Learning Networks, 16(3).

Thaler, R.H. & Sunstein, C.R. (2008). Nudge: Improving decision about health, wealth, and happiness. New Haven & London: Yale University Press.

Toffler, A. (1980). The third wave. New York, NY: Bantam Books.

van Barnefeld, A., Arnold, K.E. & Campbell, J.P. (2012). Analytic in higher education: etablishing a common language. Educause Learning Intiative. Available at: http://net.educause.edu/ir/library/pdf/ELI3026.pdf

van Harmelen, M. & Workman, D. (2012). Analytics for learning and teaching. JISC Cetis Analytics Series, 1(3).

van Leeuwen, A. (2015). Learning analytics to support teachers during synchronous CSCL: balancing between overview and overload. Journal of Learning Analytics, 2(2), pp. 138-162.

Waddington, R. & Nam, S. (2014). Practice exams make perfect: incorporating course resource use into an early warning system. Proceedings of the Fourth International Conference on Learning Analytics and Knowledge. New York, NY: ACM, pp. 188-192.

Wagner, E. & Ice, P. (2012). Data changes everything - delivering on the promise of learning analytics in higher education. Educause Review, July/August 2012, pp. 32-42.

Weiner, N. (1951). Cybernetics and society. New York, NY: Executive Techniques.

*Wise, A. (2014). Designing pedagogical interventions to support student use of learning analytics. Proceedings of the Fourth International Conference on Learning Analytics And Knowledge, LAK ’14, New York, NY: ACM Press, pp. 203-211

Wise, A., Vytasek, J., Hausknecht, S. & Zhao, Y. (2016). Developing Learning analytics design knowledge in the ‘middle space’: The student tuning model and align design framework for learning analytics use. Online Learning Consortium, 20(2), pp. 155-182.

Wong, W.Y. & Lavrencic, M. (2016). Using a risk management approach in analytics for curriculum and program quality improvement. 1st Learning Analytics for Curriculum and Program Quality Improvements Workshop, PCLA, Edinburgh, UK, Vol. 1590, pp. 10-14.

Bartle, E. (2015). Personalised Learning: an overview. A discussion paper prepared by the institute for teaching and learning innovation. Queensland University. Available at: https://itali.uq.edu.au/filething/get/1865/Personalised_learning_overview_Final_16_Mar_15.pdf



PDF


DOI: https://doi.org/10.18608/jla.2018.53.13

Share this article: