The SHEILA Framework: Informing Institutional Strategies and Policy Processes of Learning Analytics

Pedro Manuel Moreno-Marcos
Ioana Jivet
Maren Scheffel
Kairit Tammets
Kaire Kollom
Dragan Gašević

Abstract


This paper introduces a learning analytics policy and strategy framework developed by a cross-European research project team — SHEILA (Supporting Higher Education to Integrate Learning Analytics), based on interviews with 78 senior managers from 51 European higher education institutions across 16 countries. The framework was developed adapting the RAPID Outcome Mapping Approach (ROMA), which is designed to develop effective strategies and evidence-based policy in complex environments. This paper presents four case studies to illustrate the development process of the SHEILA framework and how it can be used iteratively to inform strategic planning and policy processes in real world environments, particularly for large-scale implementation in higher education contexts. To this end, the selected cases were analyzed at two stages, each a year apart, to investigate the progression of adoption approaches that were followed to solve existing challenges, and identify new challenges that could be addressed by following the SHEILA framework.


Full Text:

PDF

References

Ali, L., Asadi, M., Gašević, D., Jovanović, J., & Hatala, M. (2013). Factors influencing beliefs for adoption of a learning analytics tool: An empirical study. Computers & Education, 62, 130–148. https://doi.org/10.1016/j.compedu.2012.10.023

Arnold, K. E., Lynch, G., Huston, D., Wong, L., Jorn, L., & Olsen, C. W. (2014). Building Institutional Capacities and Competencies for Systemic Learning Analytics Initiatives. In Proceedings of the Fourth International Conference on Learning Analytics And Knowledge (pp. 257–260). New York, NY, USA: ACM. https://doi.org/10.1145/2567574.2567593

Arroway, P., Morgan, G., O’Keefe, M., & Yanosky, R. (2016). Learning Analytics in Higher Education (pp. 1–44). ECAR. Retrieved from https://library.educause.edu/resources/2016/2/learning-analytics-in-higher-education

Bichsel, J. (2012). Analytics in Higher Education: Benefits, Barriers, Progress, and Recommendations (p. 31). ECAR. Retrieved from http://net.educause.edu/ir/library/pdf/ers1207/ers1207.pdf

Code of practice for learning analytics. (2015). Retrieved June 7, 2016, from https://www.jisc.ac.uk/guides/code-of-practice-for-learning-analytics

Colvin, C., Dawson, S., Wade, A., & Gašević, D. (2017). Addressing the Challenges of Institutional Adoption. In C. Lang, G. Siemens, A. Wise, & D. Gaśević (Eds.), Handbook of Learning Analytics (First, pp. 281–289). Society for Learning Analytics Research.

Colvin, C., Rogers, T., Wade, A., Dawson, S., Gasevic, D., Shum, S. B., … Fisher, J. (2015). Student Retention and Learning Analytics: A Snapshot of Australian Practices and a Framework for Advancement (pp. 1–91). The Australian Government Office for Learning and Teaching. Retrieved from http://he-analytics.com/

Drachsler, H., & Greller, W. (2016). Privacy and Analytics: It’s a DELICATE Issue a Checklist for Trusted Learning Analytics. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 89–98). New York, NY, USA: ACM. https://doi.org/10.1145/2883851.2883893

Ferguson, R., Brasher, A., Clow, D., Cooper, A., Hillaire, G., Mittelmeier, J., … Vuorikari, R. (2016). Research Evidence on the Use of Learning Analytics: Implications for Education Policy (JRC Science for Policy Report) (pp. 1–148).

Ferguson, R., Macfadyen, L., Clow, D., Tynan, B., Alexander, S., & Dawson, S. (2014). Setting Learning Analytics in Context: Overcoming the Barriers to Large-Scale Adoption. Journal of Learning Analytics, 1(3), 120–144.

Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64–71. https://doi.org/10.1007/s11528-014-0822-x

Greller, W., & Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning Analytics. Educational Technology & Society, 15(3), 42–57.

Higher Education Commission. (2016). From Bricks to Clicks - The Potential of Data and Analytics in Higher Education | Higher Education Commission (Policy Connect). Retrieved from http://www.policyconnect.org.uk/hec/research/report-bricks-clicks-potential-data-and-analytics-higher-education

Jeremic, Z., Kumar, V., & Graf, S. (2017). MORPH: Supporting the integration of learning analytics at institutional level. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 596–597). New York, NY, USA: ACM. https://doi.org/10.1145/3027385.3029478

Jivet, I., Scheffel, M., Specht, M., & Drachsler, H. (2018). License to Evaluate: Preparing Learning Analytics Dashboards for Educational Practice. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (pp. 31–40). New York, NY, USA: ACM. https://doi.org/10.1145/3170358.3170421

Knight, D. B., Brozina, C., Stauffer, E. M., Frisina, C., & Abel, T. D. (2015). Developing a Learning Analytics Dashboard for Undergraduate Engineering Using Participatory Design (p. 26.485.1-26.485.11). Presented at the 2015 ASEE Annual Conference & Exposition. Retrieved from https://peer.asee.org/developing-a-learning-analytics-dashboard-for-undergraduate-engineering-using-participatory-design

Macfadyen, L., & Dawson, S. (2012). Numbers are not enough. Why e-learning analytics failed to inform an institutional strategic plan. Faculty of Education - Papers (Archive), 149–163.

Macfadyen, L., Dawson, S., Pardo, A., & Gaševic, D. (2014). Embracing Big Data in Complex Educational Systems: The Learning Analytics Imperative and the Policy Challenge. Research & Practice in Assessment, 9, 17–28.

Norris, D. M., & Baer, L. L. (2013). Building Organizational Capacity for Analytics (p. 58). Retrieved from https://library.educause.edu/resources/2013/2/building-organizational-capacity-for-analytics

Pardo, A., & Siemens, G. (2014). Ethical and privacy principles for learning analytics. British Journal of Educational Technology, 45(3), 438–450. https://doi.org/10.1111/bjet.12152

Park, Y., & Jo, I.-H. (2015). Development of the learning analytics dashboard to support students’ learning performance. Journal of Universal Computer Science, 21(1), 110–133.

Prinsloo, P., & Slade, S. (2015). Student Privacy Self-management: Implications for Learning Analytics. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (pp. 83–92). New York, NY, USA: ACM. https://doi.org/10.1145/2723576.2723585

Prinsloo, P., & Slade, S. (2017). Ethics and Learning Analytics: Charting the (Un)Charted. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), Handbook of Learning Analytics (First, pp. 49–57). Society for Learning Analytics Research.

Roberts, L. D., Howell, J. A., Seaman, K., & Gibson, D. C. (2016). Student attitudes toward learning analytics in higher education: “The fitbit version of the learning world.” Frontiers in Psychology, 7, 1–11. https://doi.org/10.3389/fpsyg.2016.01959

Rubel, A., & Jones, K. M. L. (2016). Student privacy in learning analytics: An information ethics perspective. The Information Society, 32(2), 143–159. https://doi.org/10.1080/01972243.2016.1130502

Siemens, G., Dawson, S., & Lynch, G. (2013). Improving the Quality and Productivity of the Higher Education Sector: Policy and Strategy for Systems-level Deployment of Learning Analytics. Sidney, Australia: Society for Learning Analytics Research for the Australian Office for Learning and Teaching. Retrieved from http://www.voced.edu.au/content/ngv%3A64739

Slade, S., & Prinsloo, P. (2013). Learning Analytics: Ethical Issues and Dilemmas. American Behavioral Scientist, 57(10), 1510–1529. https://doi.org/10.1177/0002764213479366

Slade, S., & Prinsloo, P. (2014). Student perspectives on the use of their data: Between intrusion, surveillance and care. In Challenges for Research into Open & Distance Learning: Doing Things Better – Doing Better Things (pp. 291–300). Oxford, UK. Retrieved from http://oro.open.ac.uk/41229/

The Open University. (2014, September). Policy on Ethical use of Student Data for Learning Analytics. The Open University. Retrieved from http://www.open.ac.uk/students/charter/sites/www.open.ac.uk.students.charter/files/files/ecms/web-content/ethical-use-of-student-data-policy.pdf

Tsai, Y.-S., & Gašević, D. (2017a). Learning analytics in higher education — challenges and policies: A review of eight learning analytics policies. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 233–242). New York, NY, USA: ACM. https://doi.org/10.1145/3027385.3027400

Tsai, Y.-S., & Gašević, D. (2017b). The State of Learning Analytics in Europe – Executive Summary – SHEILA (Executive summary). Retrieved from http://sheilaproject.eu/2017/04/18/the-state-of-learning-analytics-in-europe-executive-summary/

Wolff, A., Moore, J., Zdrahal, Z., Hlosta, M., & Kuzilek, J. (2016). Data Literacy for Learning Analytics. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 500–501). New York, NY, USA: ACM. https://doi.org/10.1145/2883851.2883864

Yanosky, R., & Arroway, P. (2015). The Analytics Landscape in Higher Education (p. 34). ECAR. Retrieved from https://library.educause.edu/~/media/files/library/2015/5/ers1504cl.pdf

Young, J., & Mendizabal, E. (2009). Helping Researchers Become Policy Entrepreneurs-How to Develop Engagement Strategies for Evidence-based Policy-making (ODI briefing Papers) (pp. 1–4). London: Overseas Development Institute. Retrieved from http://www.alnap.org/resource/8431



PDF


DOI: https://doi.org/10.18608/jla.2018.53.2

Share this article: