An Infrastructure for Workplace Learning Analytics: Tracing Knowledge Creation with the Social Semantic Server

Adolfo Ruiz Calleja
Sebastian Dennerlein
Dominik Kowald
Dieter Theiler
Elisabeth Lex
Tobias Ley

Abstract


In this paper, we propose the Social Semantic Server (SSS) as a service-based infrastructure for workplace and professional learning analytics (LA). The design and development of the SSS have evolved over eight years, starting with an analysis of workplace learning inspired by knowledge creation theories and their application in different contexts. The SSS collects data from workplace learning tools, integrates it into a common data model based on a semantically enriched artifact-actor network, and offers it back for LA applications to exploit the data. Further, the SSS design’s flexibility enables it to be adapted to different workplace learning situations. This paper contributes by systematically deriving requirements for the SSS according to knowledge creation theories, and by offering support across a number of different learning tools and LA applications integrated into the SSS. We also show evidence for the usefulness of the SSS extracted from 4 authentic workplace learning situations involving 57 participants. The evaluation results indicate that the SSS satisfactorily supports decision making in diverse workplace learning situations and allow us to reflect on the importance of knowledge creation theories for this analysis.


Full Text:

PDF

References

Alario-Hoyos, C., Bote-Lorenzo, M., G ́omez-S ́anchez, E., Asensio-P ́erez, J., Vega-Gorgojo, G., & Ruiz-Calleja, A. (2013).GLUE! An Architecture for the Integration of External Tools in Virtual Learning Environments.Computers & Education,60(1), 122–137.

Bachl, M., Zaki, D., Schmidt, A. P., & Kunzmann, C. (2014). Living documents as a collaboration and knowledge maturingplatform. InProceedings of the 14th International Conference on Knowledge Technologies and Data-driven Business(i-KNOW 14)(pp. 30:1–30:4). Graz, Austria: Springer.

Berendt, B., Vuorikari, R., Littlejohn, A., & Margaryan, A. (2014). Learning analytics and their application in technology-enhanced professional learning. In A. Littlejohn & A. Margaryan (Eds.),Technology-enhanced professional learning:Processes, practices and tools(pp. 144–157). Routledge.

Buckingham-Shum, S., & Ferguson, R. (2012). Social Learning Analytics.Journal of Educational Technology & Society,15(3), 3–26.Cardinali, F. (2015).Towards Learning Analytics Interoperability at the Workplace (LAW Profile). Learning Analytics Review no.5(Tech. Rep. No. ISSN: 2057-7494). (Available at:http://www.laceproject.eu/learning-analytics-review/law-interoperability/, last accessed June 2018.)

de Laat, M., & Schreurs, B. (2013). Visualizing informal professional development networks. Building a case for LearningAnalytics in the workplace.American Behavioral Scientist,57(10), 1421–1438.

Dennerlein, S., Kowald, D., Lex, E., Theiler, D., Lacic, E., Ley, T., . . . Ruiz-Calleja, A. (2015). The Social Semantic Server: AFlexible Framework to Support Informal Learning at the Workplace. InProceedings of the 15th International Conferenceon Knowledge Technologies and Data-driven Business (I-KNOW 2015)(pp. 26:1–26:8). Graz, Austria: ACM.

Dennerlein, S., Rella, M., Tomberg, V., Theiler, D., Treasure-Jones, T., Kerr, M., . . . Trattner, C. (2014). Making sense of bitsand pieces: A sensemaking tool for informal workplace learning. InProceedings of the 9th European Conference onTechnology Enhanced Learning (EC-TEL 2014)(pp. 391–397). Graz, Austria: Springer.

Dennerlein, S., Seitlinger, P., Lex, E., & Ley, T. (2016). Take up My Tags: Exploring Benefits of Meaning Making in aCollaborative Learning Task at the Workplace. InProceedings of the European Conference on Technology EnhancedLearning (EC-TEL)(pp. 377–383). Lyon, France: Springer.

Dennerlein, S., Theiler, D., Marton, P., Santos Rodriguez, P., Cook, J., Lindstaedt, S., & Lex, E. (2015). KnowBrain: AnOnline Social Knowledge Repository for Informal Workplace Learning. InProceedings of the European Conference onTechnology Enhanced Learning (ECTEL 2015)(p. 509-512). Toledo, Spain: Springer.

Derntl, M., G ̈unnemann, N., & Klamma, R. (2013). A Dynamic Topic Model of Learning Analytics Research. InProceedingsof the LAK Data Challenge, held at the Third Conference on Learning Analytics and Knowledge (LAK 2013)(pp. 1 – 5).Leuven, Belgium: CEUR.

Dewan, P. (2001). An integrated approach to designing and evaluating collaborative applications and infrastructures.ComputerSupported Cooperative Work,1(10), 75–111.

Duval, E. (2011). Attention please!: Learning Analytics for visualization and recommendation. InProceedings of the 1stInternational Conference on Learning Analytics and Knowledge (LAK 2011)(pp. 9–17). Banff, Canada.

Earl, T. (2005).Service-Oriented Architecture: Concepts, Technology and Design. Upper Saddle River, USA: Prentice HallPTR.

Eraut, M. (2004). Informal learning in the workplace.Studies in continuing education,26(2), 247–273.

Fidalgo-Blanco, A., Sein-Echaluce, M. L., Garcia-Penalvo, F. J., & Conde, M. A. (2015). Using Learning Analytics to improveteamwork assessment.Computers in Human Behavior,47(1), 149-156.

Gasevic, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning.TechTrends,59(1),64–71.

Kaschig, A., Maier, R., Sandow, A., Brown, A., Ley, T., Magenheim, J., . . . Seitlinger, P. (2012). Technological andorganizational arrangements sparking effects on individual, community and organizational learning. InProceedings ofthe 7th European Conference on Technology-Enhanced Learning (EC-TEL 2012)(p. 180-193). Lyon, France: Springer.

Klamma, R. (2013). Community Learning Analytics–Challenges and opportunities. InProceedings of the 12th InternationalConference on Web-based Learning (ICWL 2013)(p. 284 - 293).

Kenting, Taiwan: Springer.Kooken, J., Ley, T., & De Hoog, R. (2007). How do people learn at the workplace? Investigating four workplace learningassumptions. InProceedings of the 2nd European Conference on Technology Enhanced Learning (EC-TEL 2007)(pp.158–171). Heidelberg, Germany: Springer.

Kowald, D., Kopeinik, S., Seitlinger, P., Ley, T., Albert, D., & Trattner, C. (2015). Refining frequency-based tag reusepredictions by means of time and semantic context. InMining, Modeling, and Recommending’Things’ in Social Media(pp. 55–74). Springer.

Kowald, D., Lacic, E., & Trattner, C. (2014). TagRec: Towards a standardized tag recommender benchmarking framework. InProceedings of the 25th ACM conference on Hypertext and Social Media(pp. 305–307). Santiago, Chile.

Kowald, D., Seitlinger, P., Trattner, C., & Ley, T. (2014a). Long time no see: the probability of reusing tags as a functionof frequency and recency. InProceedings of the 23rd International Conference on World Wide Web (WWW’14)(pp.463–468). Seoul, Korea: ACM.

Kowald, D., Seitlinger, P., Trattner, C., & Ley, T. (2014b). Long Time no See: The Probability of Reusing Tags as a Functionof Frequency and Recency. InProceedings of the 23rd International Conference on World Wide Web (WWWC 2014)(pp.463–468). Seoul, Korea: ACM.

Kravcik, M., & Klamma, R. (2012). Supporting Self-Regulation by Personal Learning Environments. InProceedings of the12th International Conference on Advanced Learning Technologies (ICALT 2012)(p. 710-711). Rome, Italy: IEEE.

Krull, E., & Leijen, A. (2015). Perspectives for defining student teacher perfomance-based teaching skills indicators to provideformative feedback through learning analytics.Creative Education,6(10), 914-926.

Ley, T., Cook, J., Dennerlein, S., Kravcik, M., Kunzmann, C., Pata, K., . . . Trattner, C. (2014). Scaling informal learning at theworkplace: A model and four designs from a large-scale design-based research effort.British Journal of EducationalTechnology,45(6), 1036-1048.

Ley, T., & Kump, B. (2013). Which user interactions predict levels of expertise in work-integrated learning? InProceedings ofthe 8th European Conference on Technology-enhanced Learning (EC-TEL 2013)(pp. 178–190). Paphos, Cyprus.

Ley, T., & Seitlinger, P. (2015). Dynamics of Human Categorization in a Collaborative Tagging System: How Social Processesof Semantic Stabilization Shape Individual Sensemaking.Computers in Human Behavior,51, 140-151.

Manouselis, N., Drachsler, H., Verbert, K., & Duval, E. (2012).Recommender systems for learning. Springer.

Mayring, P.(2014).Qualitative content analysis: theoretical foundation, basic procedures and software solution(Tech. Rep.). Klagenfurt, Austria: SSOAR.(Available at:http://www.psychopen.eu/fileadmin/userupload/books/mayring/ssoar-2014-mayring-Qualitativecontentanalysistheoreticalfoundation.pdf, last accessed June 2017.)

Newman, S. (2005).Building Microservices. Designing Fine-Grained Systems. O’Reilly Media.Niemann, K., & Wolpers, M. (2014). Usage-Based Clustering of Learning Resources to Improve Recommendations. InProceedings of the 9th European Conference on Technology-enhanced Learning (EC-TEL 2014)(pp. 317–330). Graz,Austria: Springer.

Nonaka, I. (1994). A dynamic theory of organizational knowledge creation.Organization science,5(1), 14–37.Nussbaumer, A., Berthold, M., Dahrendorf, D., Schmitz, H., Kravcik, M., & Albert, D. (2012). A mashup recommender forcreating personal learning environments. InProceedings of the 11th International Conference on Web-based Learning(ICWL 2012)(pp. 79–88). Sinaia, Romania: Springer.

Paavola, S., & Hakkarainen, K. (2005). The knowledge creation metaphor–An emergent epistemological approach to learning.Science & Education,14(6), 535–557.

Peschl, M. F., & Fundneider, T. (2014). Designing and enabling spaces for collaborative knowledge creation and innovation:From managing to enabling innovation as socio-epistemological technology.Computers in Human Behavior,37,346–359.

Power, D. (2011). Evaluation: from Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation.Journal of Machine Learning Technologies,2(1), 37-63.

Rajagopal, K., van Bruggen, J. M., & Sloep, P. B. (2016). Recommending peers for learning: Matching on dissimilarityin interpretations to provoke breakdown.British Journal of Educational Technology. Retrieved fromhttp://dx.doi.org/10.1111/bjet.12366(In press.) doi: 10.1111/bjet.12366

Ravenscroft, A., Schmidt, A., Cook, J., & Bradley, C. (2012). Designing social media for informal learning and knowledgematuring in the digital workplace.Journal of Computer Assisted Learning,28(3), 235-349.

Renzel, D., & Klamma, R. (2013). From micro to macro: Analyzing activity in the role sandbox. InProceedings of the 3rdInternational Conference on Learning Analytics and Knowledge (LAK 2013)(pp. 250–254). Leuven, Belgium.

Ruiz-Calleja, A., Dennerlein, S., Ley, T., & Lex, E. (2016). Visualizing Workplace Learning Data with the SSS Dashboard. InCrossLAK 2016: Learning Analytics Across Physical and Digital Spaces(p. 79-86). Edinburgh, UK.

Ruiz-Calleja, A., Dennerlein, S., Tomberg, V., Ley, T., Theiler, D., & Lex, E. (2015). Integrating data across workplace learningapplications with a social semantic infrastructure. InProceedings of the 14th International Conference on Web-basedLearning (ICWL 2015)(p. 208-217). Guangzhou, China: Springer.

Ruiz-Calleja, A., Dennerlein, S., Tomberg, V., Pata, K., Ley, T., Theiler, D., & Lex, E. (2015). Supporting Learning Analyticsfor Informal Workplace Learning with a Social Semantic Server. InProceedings of the 10th European Conference onTecnology Enhanced Learning (EC-TEL 2015)(p. 634-637). Toledo, Spain: Springer.

Ruiz-Calleja, A., Prieto, L., Ley, T., Rodr ́ıguez-Triana, M., & Dennerlein, S. (2017). Learning Analytics for Professionaland Workplace Learning: A Literature Review. InProceedings of the European Conference on Technology EnhancedLearning (ECTEL 2017)(p. 164-178). Tallinn, Estonia: Springer.

Santos, P., Dennerlein, S., Theiler, D., Cook, J., Treasure-Jones, T., Holley, D., . . . Lex, E. (2016). Going beyond yourpersonal learning network, using recommendations and trust through a multimedia question-answering service fordecision-support: A case study in the healthcare.Journal of Universal Computer Science,22(3), 340-359.

Schafer, J. B., Frankowski, D., Herlocker, J., & Sen, S. (2007). The Adaptive Web: Methods and Strategies of WebPersonalization. In (pp. 291–324). Springer Berlin Heidelberg. Retrieved fromhttp://dx.doi.org/10.1007/978-3-540-72079-99doi: 10.1007/978-3-540-72079-99

Schmidt, A., Hinkelmann, K., Ley, T., Lindstaedt, S., Maier, R., & Riss, U. (2009). Conceptual foundations for a service-oriented knowledge and learning architecture: Supporting content, process and ontology maturing. In S. Schaffert &T. Tochtermann K. Pellegrini (Eds.),Networked Knowledge - Networked Media: Integrating Knowledge Management,New Media Technologies and Semantic Systems(pp. 79–94). Springer.

Schmitz, H. C., Wolpers, M., Kirschenmann, U., & Niemann, K. (2011). Contextualized Attention Metadata. In C. Roda (Ed.),Human attention in digital environments(p. 186-209). New York, USA: ACM.

Schoefegger, K., Seitlinger, P., & Ley, T. (2010). Towards a user model for personalized recommendations in work-integratedlearning: A report on an experimental study with a collaborative tagging system. InProceedings of the 1st Workshop onRecommender Systems for Technology Enhanced Learning (RecSysTEL 2010)(Vol. 1, pp. 2829–2838). Amsterdam,Holland: Elsevier.

Seitlinger, P., Kowald, D., Trattner, C., & Ley, T. (2013). Recommending tags with a model of human categorization. InProceedings of the 22nd ACM International Conference on Information & Knowledge Management (CIKM 2013) (p. 2381-2386). San Francisco, USA.

Seitlinger, P., Ley, T., Kowald, D., Theiler, D., Hasani-Mavriqi, I., Dennerlein, S., . . . Albert, D. (2017). Balancing theFluency-Consistency Trade-off in Collaborative Information Search with a Recommender Approach.InternationalJournal of Human Computer Interaction,34(6), 557–575.

Siadaty, M., Gasevíc, D., & Hatala, M. (2016a). Associations between technological scaffolding and micro-level processes ofself-regulated learning: A workplace study.Computers in Human Behavior,55(1), 1007-1019.

Siadaty, M., Gasevic, D., & Hatala, M. (2016b). Measuring the impact of technological scaffolding interventions on micro-levelprocesses of self-regulated workplace learning.Computers in Human Behavior,59(1), 469-482.

Siadaty, M., Gasevic, D., Jovanovic, J., Milikic, N., Jeremic, Z., Ali, L., . . . Hatala, M. (2012). Learn-B: A social analytics-enabled tool for self-regulated workplace learning. InProceedings of the 2nd International Conference on LearningAnalytics and Knowledge (LAK 2012)(pp. 115–119). Vancouver, Canada.

Siemens, G., Gasevic, D., Haythornthwaite, C., Dawson, S., Buckingham Shum, S., Ferguson, R., . . . Baker, R. (2011).Open Learning Analytics: An intergrated & modularized platform(Tech. Rep.). Society for Learning AnalyticsResearch (SOLAR). (Available at:https://solaresearch.org/core/open-learning-analytics-an-integrated-modularized-platform/, last accessed June 2017)

Southavilay, V., Yacef, K., Reimann, P., & Calvo, R. A. (2013). Analysis of collaborative writing processes using revisionmaps and probabilistic topic models. InProceedings of the Third International Conference on Learning Analytics andKnowledge (LAK 13)(pp. 38–47). Leuven, Belgium.

Suthers, D., & Dwyer, N. (2014). Multilevel Analysis of Uptake, Sessions, and Key Actors in a Socio-Technical Network. InProceedings of the Computational Approaches to Connecting Levels of Analysis in Networked Learning CommunitiesWorkshop at Learning Analytics and Knowledge(pp. 1–6). Indianapolis, USA.

Thus, H., Chatti, M. A., Brandt, R., & Schroeder, U. (2015). Evolution of interests in the learning context data model.InProceedings of the 10th European Conference on Technology Enchanced Learning (EC-TEL 2015)(pp. 479–484).Toledo, Spain: Springer.



PDF


DOI: https://doi.org/10.18608/jla.2019.62.9

Share this article: