ltr.edu.au€¦ · web viewisbn 978-1-76028-464-0 docx. table of contents appendix a - dvc ......
Post on 06-Aug-2021
1 Views
Preview:
TRANSCRIPT
Learning Analytics: Assisting Universities with Student Retention
2015 Final Report - Part 2 Appendices
Charles Darwin UniversityBatchelor Institute of Indigenous Tertiary Education
Griffith UniversityMurdoch University
The University of Newcastle
Deborah West
Charles Darwin University
http://www.letstalklearninganalytics.edu.au/
Support for the production of this report has been provided by the Australian Government Office for Learning and Teaching. The views expressed in this report do not necessarily reflect the views of the Australian Government Office for Learning and Teaching.
With the exception of the Commonwealth Coat of Arms, and where otherwise noted, all material presented in this document is provided under Creative Commons Attribution-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-sa/4.0/.
The details of the relevant licence conditions are available on the Creative Commons website (accessible using the links provided) as is the full legal code for the Creative Commons Attribution-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-sa/4.0/legalcode.
Requests and inquiries concerning these rights should be addressed to:Office for Learning and TeachingDepartment of Education and Training
GPO Box 9880, Location code N255EL10Sydney NSW 2001
<learningandteaching@education.gov.au>
2015
ISBN 978-1-76028-462-6 PRINT
ISBN 978-1-76028-463-3 PDF
ISBN 978-1-76028-464-0 DOCX
Table of contents
Appendix A - DVC Certification 4
Appendix B – References 5
Appendix C – Demographic Frequency Data of Academic Level Survey Participants 12
Appendix D - Relevant OLT Projects 13
Appendix E – Impact Statement 15
Appendix F – Completed Dissemination Activities 18
Appendix G – Scheduled Dissemination Activities 19
Appendix H – High Level Framework Summary 20
Appendix I – Discussion Questions 22
Appendix J – Evaluation Report 26
Learning Analytics: Assisting Universities with Student Retention 3
Appendix A - DVC Certification
Learning Analytics: Assisting Universities with Student Retention 4
Appendix B – References
ACODE (2014). Benchmarks for Technology Enhanced Learning. Retrieved 12 May, 2015 from http://www.acode.edu.au/pluginfile.php/579/mod_resource/content/3/TEL_Benchmarks.pdf
Australian Government Department of Education and Training (2014) 2013 Student Data: Appendix 4 - Attrition, Success and Retention. Retrieved 26 May, 2015 from https://www.education.gov.au/selected-higher-education-statistics-2013-student-data
Arnold, K., Lonn, S., & Pistilli, M. (2014). An exercise in institutional reflection: The Learning Analytics Readiness Instrument (LARI). Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ‘14), Indianapolis, IN, USA. doi:10.1145/2567574.2567621
Arnold, K. & Pistilli, M. (2012). Course signals at Purdue: Using learning analytics to increase student success, Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ’12), Vancouver, British Columbia, Canada. doi:10.1145/2330601.2330666
Baer, L., Norris, D., Duin, A., & Brodnick, R. (2013). Crafting transformative strategies for personalized learning/analytics. Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), Leuven, Belgium. doi:10.1145/2460296.2460354
Baker, R., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1 (1), 3-17.
Balacheff, N., & Lund, K. (2013). Multidisciplinarity vs. multivocality: The case of “Learning Analytics”. Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), Leuven, Belgium. doi:10.1145/2460296.2460299
Beer, C., Jones, D. & Clark, D. (2012). Analytics and complexity: Learning and leading for thefuture. Proceedings Future Challenges: Sustainable Futures, ascilite, Wellington, NZ. Retrieved April 13, 2015 from http://eprints.usq.edu.au/23092/2/Beer_Jones_Clark_ascilite_2012_PV.pdf
Behrendt, L., Larkin, S., Griew, R., & Kelly, P. (2012). Review of Higher Education Access and Outcomes for Aboriginal and Torres Strait Islander People: Final Report. Canberra: Australian Government. Retrieved May, 12, 2012 from http://docs.education.gov.au/documents/review-higher-education-access-and-outcomes-aboriginal-and-torres-strait-islander-people-0
Bradley, D., Noonan, P., Nugent, H., & Scales, B. (2008). Review of Australian Higher Education: Final Report. Canberra: Department of Education, Employment and Workplace Relations. Retrieved May, 12, 2015 from www.mq.edu.au/pubstatic/public/download.jsp?id=111997
Learning Analytics: Assisting Universities with Student Retention 5
Buckingham Shum, S., & Ferguson, R. (2012). Social learning analytics. Educational Technology & Society, 15(3), 3-26.
Camilleri, V., de Freitas, S., Montebello, M., & McDonagh Smith, P. (2013). A case study ‐inside virtual worlds: Use of analytics for immersive spaces. Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), Leuven, Belgium. doi:10.1145/2460296.2460341
Campbell, J., DeBlois, P., & Oblinger, D. (2007). Academic analytics: A new tool for a new era, Educause Review, July/August 2007. Retrieved May 12, 2015 from https://net.educause.edu/ir/library/pdf/ELIWEB0710.pdf
Campbell, J., and Oblinger, D. (2007) Academic analytics: White paper. Educause. Retrieved 12 May, 2015 from http://net.educause.edu/ir/library/pdf/pub6101.pdf
Chaloux, B., & Miller, G. (2013). E-learning and the transformation of higher education. Chapter 1 (pp. 3-23) in G. Miller, M. Benke, B. Chaloux, L. Ragan, R. Schroeder, W. Smutz, Wayne, K. Swan, (Eds.) Leading the e-Learning transformation in higher education: meeting the challenges of technology and distance education. Stirling, VA: Stylus Publishing.
Chatti, M., Dyckhoff, A., Schroeder, U., & Thüs, H. (2012). A reference model for learning analytics. International Journal of Technology Enhanced Learning, 4(5/6), 318–331.
Chowdry, H., Crawford, C., Dearden, L., Goodman, A., & Vignoles, A. (2013), Widening participation in higher education: analysis using linked administrative data. Journal of the Royal Statistical Society: Series A (Statistics in Society), 176: 431–457. doi: 10.1111/j.1467-985X.2012.01043.x
Clarke, J., Nelson, K., & Stoodley, I. (2013). The place of higher education institutions in assessing student engagement, success and retention: Proceedings Higher Education Research and Development Society of Australasia, Auckland, New Zealand. Retrieved May 12, 2015 from http://eprints.qut.edu.au/60024/2/60024.pdf
Clow, D. (2013). MOOCs and the funnel of participation. Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), Leuven, Belgium. doi:10.1145/2460296.2460332
Dawson, S., Gašević, D., Siemens, G., & Joksimovic, S. (2014). Current state and future trends: A citation network analysis of the learning analytics field. Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ‘14), Indianapolis, IN, USA. doi:10.1145/2567574.2567585
Dawson, S., Tan, J., & McWilliam, E. (2011). Measuring creative potential: Using social network analysis to monitor a learners’ creative capacity. Australian Journal of Educational Technology, 27(6), 924-942.
Learning Analytics: Assisting Universities with Student Retention 6
Dekker, G., Pechenizkiy, M. and Vleeshouwers, J. (2009) Predicting students’ drop out: a case study. Proceedings of the 2nd International Conference on Educational Data Mining (EDM '09), Cordoba, Spain. http://www.educationaldatamining.org/EDM2009/uploads/proceedings/dekker.pdf
Diaz, D. (2000). Comparison of student characteristics, and evaluation of student success, in an online health education course. Unpublished doctoral dissertation, Nova Southeastern University. Retrieved 12 May 2015 from http://home.earthlink.net/~davidpdiaz/LTS/pdf_docs/dissertn.pdf
Department of Industry, Innovation, Climate Change, Science, Research and Tertiary Education (DIICCSRTE) (2012a). 2011 full year higher education student statistics Appendix 2 – Equity Groups. Canberra: Australian Government. Retrieved 18 June, 2013 fromhttp://www.innovation.gov.au/HigherEducation/HigherEducationStatistics/StatisticsPublications/Pages/2011StudentFullYear.aspx
Department of Industry, Innovation, Climate Change, Science, Research and Tertiary Education (DIICCSRTE) (2012b). 2011 full year higher education student statistics Appendix 4 - attrition, progress and retention. Canberra: Australian Government. Retrieved 14 June 2013 fromhttp://www.innovation.gov.au/HigherEducation/HigherEducationStatistics/StatisticsPublications/Pages/2011StudentFullYear.aspx
Department of Industry, Innovation, Climate Change, Science, Research and Tertiary Education (DIICCSRTE). (2013a). Students: selected higher education statistics 2012 (Appendix 2: equity data). Canberra: DIICCSRTE.
Department of Industry, Innovation, Climate Change, Science, Research and Tertiary Education (DIICCSRTE). (2013b). Moving to an enhanced indicator of higher education students’ socio-economic status. Canberra: DIICCSRTE.
EDUCAUSE (2010). 7 Things you should know about analytics. Retrieved May 12, 2015 fromhttp://www.educause.edu/ir/library/pdf/ELI7059.pdf
EduTech Wiki (2013). Learning analytics. Retrieved 14 June, 2013 from http://edutechwiki.unige.ch/en/Learning_analytics#Introduction
Ferguson, R., (2012). Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6), 304–317.
Fiaidhi, J. (2014). The next step for learning analytics. IT PRO/Institute of Electrical and Electronics Engineers, September/October 2014. Retrieved May 12, 2015 from https://www.computer.org/csdl/mags/it/2014/05/mit2014050004.pdf
Frankola, K. (2001). Why online learners drop out. Retrieved May 12, 2015 from http://www.workforce.com/articles/why-online-learners-drop-out
Learning Analytics: Assisting Universities with Student Retention 7
Gallagher, M. (2014). Micro-economic reform of the Australian higher education industry: Implications of the Abbott Government’s Budget of 13 May 2014. Address to the EduTECH Higher Education Leaders Congress, Brisbane, Australia. Retrieved May 10, 2015 from https://go8.edu.au/sites/default/files/docs/article/edutech_presentation_-_4_june_2014-pdf_version.pdf
Gašević, D., Mirriahi, N., Long, P., & Dawson, S. (2014). Editorial – inaugural issue of the Journal of Learning Analytics. Journal of Learning Analytics, 1(1), 1–2.
Goldstein, P., & Katz, R. (2005). Academic analytics: The uses of management information and technology in higher education. ECAR Research Study, Vol. 8. Retrieved May 10, 2015 fromhttp://www.educause.edu/library/resources/academic-analytics-uses-management-information-and-technology-higher-education
Hanna, D. (2000). Higher education in an era of digital competition: choices and challenges. Madison, WI: Atwood Publishing.
Heathcote, E., & Dawson, S. (2005) Data mining for evaluation, benchmarking and reflective practice in a LMS. Proceedings e-Learn 2005: World conference on e-Learning in corporate, government, healthcare & higher education, Vancouver, Canada. Retrieved April 13, 2015 from http://eprints.qut.edu.au/2368/1/2368.pdf
Holman, C., Aguilar, S., & Fishman, B. (2013). GradeCraft: what can we learn from a game-inspired learning management system. Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), Leuven, Belgium. doi:10.1145/2460296.2460350
James, R., Krause, K-L., & Jennings, C. (2010). The first year experience in Australian universities: Findings from 1994 to 2009. Centre for the Study of Higher Education, Melbourne: University of Melbourne. Retrieved May 10, 2015 from http://www.cshe.unimelb.edu.au/research/experience/docs/FYE_Report_1994_to_2009.pdf
Kennedy, G., Corrin, L., Lockyer, L., Dawson, S., Williams, D., Mulder, R., Khamis, S. & Copeland, S. (2014). Completing the loop: returning learning analytics to teachers. Proceedings Rhetoric and Reality: Critical perspectives on educational technology, 31st ascilite Conference, Dunedin, New Zealand. Retrieved May 12, 2015 from http://www.academia.edu/12074024/Completing_the_loop_returning_learning_analytics_to_teachers
Kift, S. (2009). Articulating a transition pedagogy to scaffold and to enhance the first year student learning experience in Australian higher education: Final report. ALTC Senior Fellowship. Retrieved April 14, 2015 from http://www.altc.edu.au/resource-transition-pedagogy-report-qut-2009
Learning Analytics: Assisting Universities with Student Retention 8
Kift, S., Nelson, K., & Clarke, J. (2010). Transition pedagogy: A third generation approach to FYE - A case study of policy and practice for the higher education sector. The International Journal of the First Year in Higher Education, 1(1), 1-20.
Koshy, P. (2014). Student equity performance in Australian higher education: 2007 to 2012. National Centre for Student Equity in Higher Education (NCSEHE), Perth: Curtin University.
Macfadyen, L., & Dawson, S. (2010). Mining LMS data to develop an ‘early warning system’ for educators: A proof of concept. Computers & Education, 54(2), 588-599.
Mirriahi, N., & Dawson, S. (2013). The pairing of lecture recording data with assessment scores: A method of discovering pedagogical impact. Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), Leuven, Belgium. doi:10.1145/2460296.2460331
Mirriahi, N., Gašević, D., Long, P., & Dawson, S. (2014). Scientometrics as an important tool for the growth of the field of learning analytics. Journal of Learning Analytics, 1(2), 1 4.‐
Nelson, K., Clarke, J., Stoodley, I., & Creagh, T. (2014). Establishing a framework for transforming student engagement, success and retention in higher education institutions. Final Report 2014, Canberra, Australia: Australian Government Office for Learning & Teaching.
Nelson, K., Kift, S., & Clarke, J. (2012). A transition pedagogy for student engagement and first-year learning, success and retention. In I. Solomonides, A. Reid & P. Petocz (Eds.) Engaging with learning in higher education. Faringdon, UK: Libri Publishing.
Norris, D., Baer, L. & Offerman, M. (2009). A national agenda for action analytics. National Symposium on Action Analytics, St. Paul, MN. Retrieved May 12, 2015 from http://lindabaer.efoliomn.com/uploads/settinganationalagendaforactionanalytics101509.pdf
Norris, D., & Baer, L. (2012). A toolkit for building organizational capacity for analytics. Washington, DC: Strategic Initiatives, Inc. Retrieved 12 May, 2015 from https://docs.google.com/a/student.rmit.edu.au/file/d/0B0CqeHHcp-YzMEZPY2hCRHpyLVU/edit
Norris, D., & Baer, L. (2013). Building organizational capacity for analytics. Boulder, CO: Educause. Retrieved 10 May, 2015 from https://net.educause.edu/ir/library/pdf/PUB9012.pdf
Ochoa, X., Suthers, D., Verbert, K., & Duval, E. (2014). Analysis and reflections on the third Learning Analytics and Knowledge Conference (LAK 2013). Journal of Learning Analytics, 1(2), 5 22.‐Pechenkina, E & Anderson, I (2011) Background paper on Indigenous Australian higher education: trends, initiatives and policy implications. Retrieved May 12, 2015 from
Learning Analytics: Assisting Universities with Student Retention 9
www.innovation.gov.au/HigherEducation/IndigenousHigherEducation/ReviewOfIndigenousHigherEducation/Pages/Research.aspx.
Raca, M., Tormey, R., & Dillenbourg, P. (2014). Sleeper’s lag: A study on motion and attention. Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ‘14), Indianapolis, IN, USA. doi:10.1145/2567574.2567581
Rivera, J., and Rice, M. (2002). A comparison of student outcomes & satisfaction between traditional & web based course offerings. Online Journal of Distance Learning Administration, 5(3).
Romero, C., & Ventura, S. (2010). Educational data mining: A review of the state-of-the-art. IEEE Transaction on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 40(6), 601-618.
Siemens G. (2012). Learning analytics: Envisioning a research discipline and a domain of practice, Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ’12), Vancouver, British Columbia, Canada. http://learninganalytics.net/LAK_12_keynote_Siemens.pdf
Siemens, G., Dawson, S., & Lynch, G. (2013). Improving the productivity of the higher education sector: policy and strategy for systems-level deployment of learning analytics. Society for Learning Analytics Research/Australian Government Office for Learning and Teaching. Retrieved 14 April, 2014 from http://www.itl.usyd.edu.au/projects/SoLAR_Report_2014.pdf
Siemens, G. & Long, P. (2011) Penetrating the Fog: Analytics in learning and education. EDUCAUSE Review, 46(4) July/August. Retrieved 20 May, 2013 from http://www.educause.edu/ero/article/penetrating-fog-analytics-learning-and-education
Suthers, D., & Verbert, K. (2013). Learning analytics as a “middle space.” Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), Leuven, Belgium. doi:10.1145/2460296.2460298
Thomas, L. (2002).Widening participation in post compulsory education.‐ Education & Training, 44(4/5), 241 – 242. doi:10.1108/et.2002.44.4_5.241.2
Tinto, V. (2009) Taking student retention seriously: rethinking the first year of university. ALTC FYE Curriculum Design Symposium, Queensland University of Technology, Brisbane, Australia. Retrieved May 8, 2015 from http://www.fyecd2009.qut.edu.au/resources/SPE_VincentTinto_5Feb09.pdf
Tsui, J., Lau, J., & Shieh, L. (2014). Septris and SICKO: Implementing and using learning analytics and gamification in medical education. Educause Learning Initiative. Retrieved May 10, 2015 from https://net.educause.edu/ir/library/pdf/ELIB1401.pdf
Learning Analytics: Assisting Universities with Student Retention 10
Universities Australia (2013). A smarter Australia: An agenda for Australian higher education 2013-2016. Canberra, ACT: Universities Australia. Retrieved April 21, 2015 from https://www.universitiesaustralia.edu.au/news/policy-papers/Universities-Australia-Policy-Statement-2013---2016#.VVFLwPmqpBc
van Barneveld, A., Arnold, K., & Campbell, J. (2012). Analytics in higher education: Establishing a common language. Educause Learning Initiative. Retrieved May 10, 2015 from http://net.educause.edu/ir/library/pdf/eli3026.pdf
van Harmelen, M., & Workman, D. (2012). Analytics for learning and teaching. JISC CETIS Analytics Series, Analytics for Learning and Teaching, 1(3). Retrieved May 12, 2015 from http://publications.cetis.ac.uk/wp-content/uploads/2012/11/Analytics-for-Learning-and-Teaching-Vol1-No3.pdf
Verbert, K., Govaerts, S., Duval, E., Santos, J., Assche, F., Parra, G., & Klerkx, J. (2013). Learning dashboards: an overview and future research opportunities. Personal and Ubiquitous Computing, 18(6), 1499 – 1514. West, D., Lizzio, A. Miles, Toohey, D., C. Huijser, H., Bronniman, J. & Searle, B. (2013) Learning Analytics: How it can assist Universities with Student Retention. OLT Strategic Commissioned Project Application. Willcoxson, L., Manning, M., Hibbins, R., Joy, S., Thomas, J., Girardi, A., Leask, B., Sidoryn, T., Cotter, J., Kavanagh, M., Troedson, D., & Lynch, B. (2011). The whole of university experience: Retention, attrition, learning and personal support interventions during undergraduate business studies. Surrey Hills, NSW: Australian Learning and Teaching Council. Retrieved May 10, 2015 from http://eprints.usq.edu.au/20138/
Wolff, A., Zdrahal, Z., Nikolov, A., & Pantucek, M. (2013). Improving retention: Predicting at-risk students by analysing clicking behaviour in a virtual learning environment. Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), Leuven, Belgium. doi:10.1145/2460296.2460324
Zaïane, O. (2001). Web usage mining for a better web-based learning environment. Proceedings 4th IASTED International Conference on Advanced Technology for Education (CATE ‘01), Banff, Canada. Retrieved 12 May, 2015 from http://webdocs.cs.ualberta.ca/~zaiane/postscript/CATE2001.pdf
Learning Analytics: Assisting Universities with Student Retention 11
Appendix C – Demographic Frequency Data of Academic Level Survey Participants
Variable Category Absolute Frequency
Relative Frequency
Location (n = 351)
Australia 341 97%New Zealand 10 3%
Primary Work Role (n = 353)
Teaching Students 188 53%Learning Support 47 13%Management/Administration 37 11%Other 32 9%Research 24 7%Academic Development 18 5%Student Support 7 2%
LMS at Institution (n = 353)
Blackboard 203 58%Moodle 124 35%Brightspace (D2L) 13 4%Sakai 2 1%Other 11 3%
Employment Basis (n = 351)
Full Time 284 81%Part Time 44 13%Casual 18 5%Other 5 1%
Academic Level (n = 351)
Lecturer 124 35%Senior Lecturer 88 25%Other 59 17%Associate Professor 30 9%Associate Lecturer/Tutor 30 9%Professor 20 6%
Length of employment in current institution(n = 324)
Less than 1.5 years 40 12%1.5 – 5 years 68 21%5 – 10 years 96 30%10- 20 years 86 27%More than 20 years 34 11%
Length of employment in Higher Education Sector(n = 345)
Less than 1.5 years 11 3%1.5 – 5 years 42 12%5 – 10 years 85 25%10- 20 years 130 38%More than 20 years 77 22%
Involvement in teaching students (n = 353)
Teaches students 276 78%Does not teach students 77 22%
Learning Analytics: Assisting Universities with Student Retention 12
Appendix D - Relevant OLT Projects
The following is a list of relevant OLT projects which have been considered during the project and, which are linked to in the broader project literature review.
Project title Lead institution
Summary
Enabling connected learning via open source analytics in the wild: learning analytics beyond the LMS (2014)
Queensland University of Technology
This project extends the field of Learning Analytics (LA) by developing an open source toolkit for performing sophisticated analysis of learners' engagement in connected learning environments.
Innovative perspectives and approaches for enhancing the student experience (2014)
University of Melbourne
This project builds capacity of higher education institutions by developing new perspectives and approaches for enhancing the student experience.
Engaging postgraduate students and supporting higher education to enhance the 21st century student experience (2014)
Bond University
The project engages postgraduate students and higher education personnel to contribute their experiences, insights and strategies to enhance the 21st century student experience, with an enquiring lens on the growing postgraduate coursework experience.
Shaping the 21st century student experience at regional universities (2014)
University of the Sunshine Coast
This project seeks to understand the 21st century experiences of the diverse undergraduate student populations of regional universities to identify how equitable outcomes for these students can be achieved.
Open for learning: using open data tools and techniques to support student learning (2013)
Monash University
This project explored the extent to which providing open access to data through digital technology allows students to re-configure and re-use data in order to address real-world problems relating to their learning behaviours and learning decisions.
Developing a culturally appropriate data quality framework for Aboriginal and Torres Strait Islander higher education statistics (2013)
University of Notre Dame
This project addressed the persistent and problematic nature of data and statistics pertaining to Aboriginal and Torres Strait Islander peoples, in particular in relation to their participation in higher education.
Learning Analytics: Assisting Universities with Student Retention 13
Project title Lead institution
Summary
Evidenced-based approach to the design and redevelopment of inclusive technology enhanced learning environments (2013)
University of South Australia
This project aimed to demonstrate the benefits of an evidence-based approach to the design of inclusive technology enhanced learning (TEL) environments.
Data mining of learning behaviours and interactions for improved sentiment and performance (2013)
University of Technology Sydney
This project aimed to develop innovative and workable data mining technology for an in-depth understanding of what makes good academic sentiment and performance and how poor-performing students can receive early intervention in a customized way in order to advance to high-performing status.
Student retention and learning analytics: a snapshot of current Australian practices and a framework for advancement (2013)
University of South Australia
This project aimed to undertake a review of learning analytics across Australian universities within an international context. The project focuses on how learning analytics are informing teaching practice, personalised learning, and applications for improving retention and identification of students at-risk.
Completing the loop: returning meaningful learning analytic data to teachers (2013)
University of Melbourne
The project has developed a web-based analytics tool that will support teachers to more easily interpret learning analytics to help them improve teaching and learning practices.
Improving educational opportunities for Aboriginal education workers through technology-based pedagogy (2013)
Murdoch University
Design-based research (DBR) informed by Indigenous research theories were used to analyse the potential educational roles for AEWs enabled by E-Learning and new technologies.
Exploring the role of technology in fostering sense of belonging in students studying by distance (2013)
Charles Sturt University
This project examined not only the impact of specific programs designed to enhance student belonging, but moreover how institution-wide change in learning technologies and policies affects practice.
Learning Analytics: Assisting Universities with Student Retention 14
Appendix E – Impact Statement
Anticipated Changes at:
Impact on: Project Completion
6 months post-completion
12 months post-completion
24 months post-completion1
1 Team Members
Project members all have influence within their institution around the development of learning analytics infrastructure, strategies and/or implementation
Some team members have gained additional input to the development of learning analytics in their institution
Team members have influenced vendors roadmap and development of proprietary analytics packages
University of Newcastle to scope a student-focused analytics solution targeting retention outside of the larger MIS and from within Blackboard.
University of Newcastle implementation of student-focused analytics solution (for students)
2 Project team members immediate Students2
3 Spreading the Word
Presentations have been requested and completed in Malaysia and Singapore
Article for special edition of journal requested by editors
One external
Follow up ‘use case’ workshop run at one conference to provide examples of reports and gather further feedback Identified
applicability of
1 Learning analytics is at a very early stage and a fast moving field so it is difficult to project how this work would be useful in this time frame beyond being something that others would build on.2 only one of the project team members has a direct teaching role; most of us are located at a central level so
are likely to influence students in our institution more broadly
Learning Analytics: Assisting Universities with Student Retention 15
Anticipated Changes at:
Impact on: Project Completion
6 months post-completion
12 months post-completion
24 months post-completion
institution has engaged in a workshop to consider use cases to inform the development of learning analytics
2 additional institutions have requested workshops
Request to work with 2 Malaysian universities to extend this study in Malaysia
Several papers are scheduled at various conferences in the coming months
2 use case workshops are run in outside institutions
framework in Malaysia
4 Narrow opportunistic adoption in partner institutions (impacting on some students)
Learning analytics reports on retention have been trialled by some academics in 1 partner institution although they are available to all academics
Framework is applied robustly in 3 institutions to progress learning analytics initiatives
5 Narrow systemic adoption (in partner institutions impacting on all students)
Use case workshops have been run in two partner institutions
Use cases have been used to inform the development of reports at 1 partner institution
Framework has been used to inform development (or move forward on learning analytics
Advancement of the use of learning analytics for retention in partner institutions
The use of learning analytics reports is included in policy procedure in at least one partner institution.
Learning Analytics: Assisting Universities with Student Retention 16
Anticipated Changes at:
Impact on: Project Completion
6 months post-completion
12 months post-completion
24 months post-completion
in 3 partner institutions
6 Broad opportunistic adoption (in outside institutions impacting on some students)
Increase in the number of academics using learning analytics reports to assist with student retention
7 Broad systemic adoption (in outside institutions impacting on all students)
One LMS vendor has done initial mapping of academic use cases to available reports in their product
LMS vendor has adopted technical implementation approach by one institution as a standard
Use cases have been used as model for implementation in 3 institutions
Framework utilised by a number of outside institutions to inform the ongoing development of learning analytics for retention
Potential addition of reports to 1 vendors product
Additional projects to extend the work of the project in the sector (e.g. next iteration of another version of a framework)
Additional projects to extend the work on the project
Learning analytics becomes a core element of retention plan reporting and/or informing of retention strategies
Application of learning analytics work to various disciplines
Learning analytics emerges as a ubiquitous
Learning Analytics: Assisting Universities with Student Retention 17
Appendix F – Completed Dissemination Activities
Event Date Event Title, Location
Brief Description of Event Number of participants
10 June 2014 OLT Conference Sydney
Workshop on Learning Analytics with Prof Phil Long and Prof Gregor Kennedy
>80
8 July 2014 First Year Experience Conference, Darwin
Joint paper with Ms Cassandra Colvin (UniSA team)
>100
21 August Learning Analytics presentation, CDU
Presentation to introduce learning analytics to the academic community
>50
26-28 August 2014
Blackboard Learning and Teaching Conference, Gold Coast
Workshop on Learning Analytics Use Cases
>80
1 October Nanyang Technological University, Singapore (webinar)
Presentation on learning analytics for staff PD in learning and teaching
>200
8 October 2014 Flinders University Use case workshop with Flinders staff >20
11 November IRU, MRUN workshop, Universiti of Malaya, KL
Presentation on project to IRU/MRUN members
>80
18 November 2014
Charles Darwin University
Use case workshop with CDU staff >20
20 November 2014
Australian Learning Analytics Summer Institute
Panel session >80
23-26 November 2014
ASCILITE Conference, Dunedin NZ
Joint Poster Presentation on Project with UniSA team
13 February 2015 Newcastle University Use case workshop with Newcastle staff >60
16-20 March 2015
Learning Analytics Knowledge Conference, USA
Joint poster presentation on Project with UniSA team
9 April 2015 Let’s Talk Learning Analytics National Forum
Project Forum >150
Learning Analytics: Assisting Universities with Student Retention 18
Appendix G – Scheduled Dissemination Activities
Event Date Event Title, Location Brief Description of Event
16-19 June 2015 35th Annual Conference of the Society for Teaching and Learning in Higher Education (STLHE), Vancouver, Canada
Conference presentation on project findings
17-19 June 2015 10th eLearning Forum, Singapore Conference paper presentation on project findings
22-24 June, 2015 AACE EdMedia 2015: World Conference on Educational Media and Technology, Montreal Canada
Best Practice session on project findings
6-9 July 2015 HERDSA Conference, Melbourne Conference paper presentation on project findings
25-27 August 2015
Blackboard Teaching and Learning Conference Follow up workshop on use cases
27 -30 October 2015
ISSOTL conference Melbourne Paper presentation (if accepted)
Learning Analytics: Assisting Universities with Student Retention 19
Appendix H – High Level Framework Summary
See Next Page
The framework can also be downloaded as a stand-alone pdf from here.
Learning Analytics: Assisting Universities with Student Retention 20
Learning Analytics: Assisting Universities with Student Retention 21
Appendix I – Discussion Questions
See Next Three Pages
The discussion questions can be downloaded as a stand-alone pdf from here.
Learning Analytics: Assisting Universities with Student Retention 22
Learning Analytics: Assisting Universities with Student Retention 23
Learning Analytics: Assisting Universities with Student Retention 24
Learning Analytics: Assisting Universities with Student Retention 25
Appendix J – Evaluation Report
Evaluation Reflections
Charles Darwin University – SP13-3268
Learning Analytics: assisting universities with student retention
Background
The overall aim of this project was to develop a framework which cross references the tools available through learning management systems and learning analytics with variables and actions to improve retention of at risk students. The project addressed this aim through an extended literature review, two national surveys, in-depth interviews and case studies.
The focus of this project is on the use of learning analytics by higher education institutions in Australia, for student retention. The data gathered was related to academic and executive views and uses of learning analytics.
The guiding focus of the evaluation was to determine if the project’s aims were achieved, and outcomes delivered, within budget and on time.
The intended outcomes were to:
- Gain a better understanding of the maturity of the learning analytics implementation and application within Australia
- Explore and identify the range of ways analytics are being used in the sector
- Increase awareness of the role, function and potential uses of analytics for student retention
- Increase awareness of the elements that need to be considered for learning analytics implementation
These outcomes were intended to be accomplished via the following demonstrable outputs:
Overview of use/potential use and limitations of learning analytics
Framework for evaluating use of analytics for student retention
Case studies on the use of the framework to evaluate analytics
National forum
Learning Analytics: Assisting Universities with Student Retention 26
Evidence
The first interactions between the Project and Evaluation Teams were at the OLT workshop for the 2013 Strategic Commissioned Projects, conducted in April 2014. Within the Learning Analytics Cluster, there were two projects, with this project being led by Associate Professor Deborah West from Charles Darwin University, with team members Henk Huijser, David Heath, Alf Lizzio, Carol Miles, Danny Toohey, Bill Searle and Jürg Bronnimann.
In order to identify that the project’s aims were achieved and outcomes delivered both formative and summative evaluation strategies were utilised. The Evaluation team was provided with access to the key documentation from the project team and were included in significant project team communications. In addition, a member of the evaluation team was a participant in virtual and face to face project team and reference group meetings. Specifically, an evaluation team member participated in the 3 day project team meeting in September 2014.
Throughout the lifecycle of the project the evaluation team provided input and advice.
The Evaluator found several key factors that contributed to the successful achievement of the project aim and goals. These factors include:
● Regular meetings of the project team with the Evaluator and Reference Group from the beginning of the project, which were well supported by project plan updates and reports on activities. This ensured that the team were provided formative feedback to further enhance the proposed project outcomes.
● Active and sustained communications between project team members from the partner institutions.
● Strong project management, as demonstrated by extensive and appropriate documentation and insightful input to the project from the project manager.
● Appropriate knowledge of institutional structures and priorities, ensuring that the activities undertaken related to institutional strategies and requirements in this emerging field.
● Establishment and maintenance of a genuine team approach to the project. This was exemplified in the breadth of contribution made by project team members at national forum.
● Diversity of skill set in the project team, which ensured a range of perspectives, breadth of analytical skills, and variation of insight into the project communication requirements.
Learning Analytics: Assisting Universities with Student Retention 27
Project Management
It has been documented that effective project management has the following elements:
● Identifying requirements,● Establishing clear and achievable outcomes,● Balancing the competing demands for quality, scope, time and cost,● Managing the expectations of various stakeholders, and● Adapting plans to overcome challenges.
From a Project Management perspective, the project was well managed and all stakeholder groups were involved. There was effective and significant communication with all members of the project team and involvement of the reference group assisted with project execution and promulgation of project outputs.
From the outset it was evident that this was an active, enthusiastic and well-led project with clear project goals and strategies.
The leadership from Associate Professor Deborah West along with the capable and conscientious oversight of David Heath as Project Manager, were key factors in the success of this project. This facilitated a collaborative team environment with a clear and ongoing focus on the project deliverables. Inclusion of a member of the Evaluation Team in key project discussion provided formative evaluation and input throughout the project and was facilitated by the project leader in a positive and generative manner.
Achievement of Outcomes
The key summative evaluation questions centred on whether the project was able to identify the usage of learning analytics tools, generate a meaningful framework, establish documented case studies and undertake the national forum.
There were over 353 valid survey responses from academic staff, and the data within these surveys were able to be analysed and mapped to identify the context and tensions that are currently present in the use of learning analytics to address student retention.
Institutional data was derived from 23 interviews across 15 institutions, and the completion of 24 institutional level surveys (22 from Australia and 2 from New Zealand). It should be noted that this extent of response at sector level is a positive reflection on the execution of the institutional level survey by the project team.
This was supplemented with five case studies. Survey inform also provided insight into the current state of LMS use in the sector.
Learning Analytics: Assisting Universities with Student Retention 28
The final phase of the project featured the national forum, which included input from the University of South Australia Learning Analytics OLT Strategic Grant project team. The national forum was a well attended (148 attendees) event, that generated informed discussion on the Learning Analytics field in general, and the project outputs specifically.
The national forum in particular was used to elicit the final input to the institutional Learning Analytics framework.
The forum and overall project is supported by the website at: http://www.letstalklearninganalytics.edu.au/
To date (June 2015) there have been 13 workshops/presentations undertaken, addressing and disseminating the project and its outcomes. Over 900 attendees have participated at these events. The project team have reported that a further 8 presentations are scheduled by the end of 2015.
Summary
The project activities, and in particular the national forum, ensured that a large number of stakeholders (institutions, academic managers and educators) were not only party to the collection of the sector data, but also engaged with the rapidly changing agenda of Learning Analytics, and its specific application to student retention.
The form of the surveys and the associated data provide important insights into the tension between institutional level managerial use of learning analytics data and the explicitly educational use of the data. The variation in perspectives identified by the project, and the means by which a whole-of-institution capability is advanced, is a key insight to the effective maturation of learning analytics in the sector. The institutional framework is now a tested tool for extending and informing the necessary dialogue, at multiple levels, within institutions, and it can be expected that the framework will be actively used across the sector.
Overall the evaluators appreciated the opportunity to work with the capable and enthusiastic project team. The extent of interest in the forum, the active nature of the forum, and the ongoing positive inter-institutional dialogue in the weeks following its conclusion is reflective of the value of this project.
Learning Analytics: Assisting Universities with Student Retention 29
top related