STELA Live - Learning Analytics for Student Success

Project overview

'STELA Live: Learning Analytics for Student Success' is a project funded under the National Forum for the Enhancement of Teaching and Learning SATLE 2020 call led by Dr Angelica Rísquez (Educational Technologies and Learning Analytics Lead) in collaboration with Sarah Gibbons/Claire Halpin (Student Experience Lead) and Dr Mohd Fazil (STELA Live Lead Researcher) in CTL; Dominic Burns (ITD Business Intelligence); and first year module leaders: Michael P O’Brien (KBS), Donal Palcic (KBS), Teresa Curtin (S&E), Chris McInerney (AHSS). The project also worked with the support from Academic Registry, Data Protection Officer and Registrar.

The project, which has run between May 2021 and April 2023, builds on the ongoing sectoral work on learning analytics and the Online Resource for Learning Analytics (ORLA) in order to take an informed approach to Learning Analytics (Using Learning Analytics to support the enhancement of teaching and learning in Higher Education) and the previous work in the former STELA (STudent Evaluation and Learning Analytics), led by the Quality Office in UL (2019-2021). In STELA Live, we worked to lay the foundations to implement the Policy on the Use of Data to Enhance Teaching, Learning and Assessment (Learning Analytics) and contribute to building institutional data and insight on the possible application of learning analytics interventions in UL at this time, while building a community of practice around their application. In doing so, we worked to progress Aim 3 under Transforming Teaching in the UL Learning, Teaching and Assessment Strategy 2022-2027 to establish new learner analytics capabilities and supports to inform and enhance teaching, learning and assessment.

Aim 3 Transforming Teaching - Wisdom in Action

In summary, the project has conducted a baseline analysis based on machine learning models with 8000 students over four academic years, and a pilot implementation of an intervention for student success with four large first year cohorts. A pilot intervention was designed and evaluated, where students were notified mid-semester of their likelihood to succeed in the module, and referred to appropriate supports. Learnings and insights are being currently shared based on the evaluation of the impact of the initiative. In doing so, we aim to contribute to building institutional data and insights on the possible application of learning analytics interventions at UL. Also, we set the ground to explore sustainable and contextually relevant solutions to the applications of learning analytics through Brightspace, the new VLE at UL.

The project aimed to achieve three main outputs divided into three distinct phases:

A more detailed account of each of these phases is provided below:

Phase I

Scoping and harvesting teaching, learning and assessment data

Scoping of extraction of student success data to identify the variables that best predict performance and progression, accessing and combining existing data including information in SI, engagement analytics from the VLE (Sulis) and other supported platforms, and other variables of engagement; in liaison with UL relevant data owners and custodians (Academic Registry, Data Protection Officer, Business Intelligence Unit, Quality Support and Registrar).
Drafting proposed dataset based on the national ORLA Data Conceptual model, which includes both demographic and learning analytics data.
Collaborative drafting of the Data Management Plan with ITD Business Intelligence and SITS Data Governance:
Compilation of SITS Data Sharing Templates, following approval from SITS Data Governance for the data to be harvested for this project (approved by Associate Vice President Academic Services & Deputy Registrar, September 2021)

Phase II Baseline analysis

Recruitment of a dedicated position for the project to work in collaboration with the BI unit in ITD as a researcher to extract, combine, anonymize, and analyse data.
Production and approval of a Data Protection Impact Assessment in collaboration with the Data Protection Officer
Selection of four voluntary large first year undergraduate modules, in collaboration with module leaders as data owners: MI4007 – Michael P O’Brien (KBS); EC4101 – Darragh Flannery and Donal Palcic (KBS); CH4701 – Teresa Curtin (S&E); and HP4001 – Chris McInerney (AHSS).
Approval for data access from Office of the Provost
Ethics research approval from AHSS Ethics Research Committee for baseline analysis

Phase III Design, intervention and evaluation

Creation of a dataset with common data definitions to facilitate appropriate access by staff to support student success in collaboration with the Business Intelligence Unit in ITD (Dominic Burns) in collaboration with module leaders as data owners.
Design of a protocol for intervention to improve the academic experience of students through the delivery of timely, personalised, and actionable feedback under the umbrella of the Policy on the Use of Data to Enhance Teaching, Learning and Assessment (Learning Analytics). This intervention will be based on the insights from the preceding baseline analysis and current data of engagement and informed from the perspective of teachers, students, and relevant support services at UL.
Implementation of intervention in SEM1 2022/23 with pilot modules following national best practice published in ORLA.
Evaluation of impact based on voluntary student participation, dissemination of results and reporting.

Presentation for National Forum Student Success Webinar

Presentation for UL Teaching, Assessment and Learning Committee