An Investigation of the Impact of Learner Support Initiatives on Retention in ICT Programmes

Dr. Michael English, Dept. of Computer Science & Information

Systems, University of Limerick, Tel: 061-202772

Email: Michael.English@ul.ie

Dr. Hussain Mahdi, Dept. of Electronic & Computer Engineering,University of LimerickTel: 061-213492Email: Hussain.Mahdi@ul.ie

To conduct a literature review on the most commonly adopted retention interventions in ICT undergraduate programmes at 3rd level institutions both nationally and internationally.

To evaluate the relationship and, hence, the impact of ICT learner support services on the progression of students on ICT programmes, as a major indicator of retention in the field.

To create and conduct a new student survey into gain an insight into current ICT students choices/expectations of their study programmes/career, whether they contemplated leaving, and factors that helped their continuing, as well as their perception of the ICT learner support services offered.

Aims & Objectives

Data analysis of attendance at Supplemental Academic Learning Supports (SALS) & their correlation with 1st Year ICT student progression for the period 2010-2014.

Data analysis of attendance at SALS & their correlation with 1st Year ICT student performance for the period 2010-2014.

Analysis of responses of 1st year ICT students of the class of 2015 to survey on their perceptions of retention issues and the SALS services provided by the Learning Centres.

Methodology

Factors Influencing Student Retention◦ Preparation for HE – Prior Attainment◦ Institutional and Course Expectations & Commitments◦ Academic & Learning Experience◦ Academic and Social Integration - Belonging◦ Quality of Institutional Support Services◦ Family Support & Personal Commitments◦ Financial Issues

In most cases, students may have more than one reason for leaving their course

Factors Influencing Student Retention

Transition/Orientation Programmes

Academic Mentoring and Advisory Programmes

Curriculum Development

Active Collaborative Learning Programmes

Pastoral Programmes

Traditional Approaches to Enhancing Retention

A lack of student awareness of what the study of ICT/CS courses entails often resulting in students having misconceptions as to the type of subjects that they must master in order to succeed within these courses. -Institutional & Course Expectations & Commitments

There is also evidence suggesting that the order in which the core ICT subjects are taught to students could be important. -Academic & Learning Experience

Students entering ICT courses with limited maths skills and problem solving abilities. -Prior Attainment

Most of the core ICT subjects involve planning and logic learning and hence learning these subjects requires a significant amount of time to practice. However, most ICT courses may not provide sufficient practicing time for students. -Prior Attainment, Academic & Learning Experience

Factors Influencing Retention - ICT Specific

ICT projects require students to perform analysis, design, coding, testing, as well as effective communication and working in teams from the early stage of ICT courses. This is hindered by entry level ICT students lacking these skills at this stage. -Academic & Social Integration, Academic & Learning Experience, Prior Attainment

Many early stage ICT students report a lack of community identity and that ICT courses can lack socialisation aspects which would foster such community relationships. -Academic & Social Integration

In many cases the attrition rate of female students in ICT courses is higher than their male counterparts; this is attributed to a number of departmental characteristics which maybe causing such imbalance. -Academic & Social Integration

Learning core ICT subjects, such as computer programming, can be an intimidating experience for some students and detrimental to their self-efficacy. -Academic & Learning Experience

Factors influencing Retention – ICT Specific

Creating engaging pedagogical environments, via the adoption of innovative student-centred ICT subject delivery approaches, designed to facilitate subject instruction in an easy to follow & enjoyable manner and nurture self-learning & social integration. Examples: use of gamification, visual programming tools & flipped classrooms.

Active collaborative learning schemes which foster a sense of belonging & ICT community building through the use of peer groups to facilitate learning and project work in a student-led manner. Examples: Peer-Learning & Paired Programming.

Foundation courses to ease the transition of students into ICT programmes of study by ensuring students acquire the necessary foundation knowledge, core ICT skills & other skills such as problem solving. Example: CS-0 courses in programming.

Orientation programmes designed specifically for ICT students, with the aim of fostering a sense of community within the department and introduce new students to campus life as ICT majors. Example: the Scavenger Hunt at the University of Illinois.

Common ICT Specific Retention Approaches

The ICT Learning Centre (ICTLC) is designed to support and enhance student active-learning within ICT disciplines across UL, with a particular focus on programming and software related skills.

The Regional Peer-Supported Learning Centre (PSLC) promotes pro-active peer-supported learning schemes, particularly the Peer-Supported Learning Groups (PSLG); a group-based, student-led collaborative learner support scheme that targets difficult subjects/modules.

The ICTLC & PSLC provide Supplementary Academic Learner Support (SALS) services which seeks to engage students through learner focused support & tutoring, augmenting existing teaching and learning activities in collaboration with faculty members.

Overview of UL Learner Support Centres

736 first year students in four ICT programmes over five academic years.

Only students who progressed to and registered in the 2nd year of their original programme of enrolment, having successfully completed all required modules by the end of the 1st year, were counted as retained.

Engaged student: one who availed of 4 or more support sessions over the entire academic year; Not Engaged: one who availed of less than 4 support sessions over the entire academic year.

At Risk student: one who attained a final grade of C or lower in their core ICT module in the 1st Semester; Not At Risk student: one who attained a higher than C final grade.

Effect of SALS Services on Student RetentionQuantitative Data Analysis Process

Students’ Participation In SALS Services

Academic Year

Number of Students

Students Who Attended at Least One Support Session

2009-2010 128 66%

2010-2011 120 48%

2011-2012 143 59%

2012-2013 174 70%

2013-2014 171 53%

Progression Rates Of All Engaged And Not Engaged Students

Progression Rates Of Engaged And Not Engaged Students Who Were At Risk

2009-2010 2010-2011 2011-2012 2012-2013 2013-2014 Average

77% 73%81%

70%

87%77%

44%54% 57%

63%57% 58%

Engaged & Progressed to 2nd Year Not Engaged & Progressed to 2nd Year

2009-2010 2010-2011 2011-2012 2012-2013 2013-2014 Average

59% 60% 60%53%

73%

61%

26%33% 31%

42%34% 35%

Engagement Levels Of At Risk vs. Not At Risk

Average engagement levels of At Risk students is 8% lower than those who are Not At Risk.

2009-2010 2010-2011 2011-2012 2012-2013 2013-2014 Average

42%

33%

26%

39%

14%

31%

55%

34%

43%40%

22%

39%

At Risk & Engaged Not At Risk & Engaged

Students’ Performance For Core Programming Modules in 1st & 2nd Semesters – Comparison of Engaged and Not-Engaged Students

 

Academic Year2009-2010

2010-2011

2011-2012

2012-2013

2013-2014

Average

Semester 1

Pass Rate (C Grade and

above)

+15.7%

+1.3%+16.1

%+6.3% +19% +11.6%

A Grade +7.9% -3.8% +8.1% +4.6% +18% +7.0%

Semester 2

Pass Rate (C Grade and

above)

+29.7%

-1.3%+21.3

%+11.8

%+36% +19.5%

A Grade+21.8

%-2.5%

+16.5%

+5.5% +7% +9.7%

Students’ CAO Data Analysis

2009-2010 2010-2011 2011-2012 2012-2013 2013-2014 Average

378 381

393

410

386 390

416425

445 448 451

437

Average CAO Points of Students who did not Progress to 2nd YearAverage CAO Points of Students who Progressed to 2nd Year

Of the bottom 25% of students (ranked by CAO points), between 72% and 86% were identified as At Risk and that between 49% and 70% of this cohort did not progress to the 2nd year of their programme.

• The survey comprises 34 multi-choice, closed and open-ended questions.

• 78 responses were received.

• 42% of respondents stated that they have availed of the support services at least once. Also, 47% of all respondents fall into the At Risk category.

• 28% of respondents were very certain about their choice of career before enrolling on their current programme of study; 10% very uncertain of their choice. However, no correlation was found between these figures and the students At Risk status.

Effect of SALS Services on Student Retention Qualitative Data – Student Survey

78%

12%

10%

Country of Birth

Ireland

Other (non-EU)

Other (EU)87%

13%

Language Spoken at Home

English

Other

88%

12%

Qualification Prior to Entry

Leaving Certificate

Other

Programme Choice and At Risk Status & Engagement

At Risk Availed of the Services

43% 40%

80%

60%

1st choice 2nd or lower choice

• Students whose programme of study fully matched their prior expectations were least likely to be At Risk.

• Also, students who chose their programme of study for reasons such as “earning good money” and “The best I could get with my entry score” are more likely to be At Risk.

Computer Programming

Mathematical Skills

Writing Skills Communication Skills

57%

50%

21%18%

40%

34%

61%

50%

Weak and At Risk Strong and At Risk

Prior Skills & At Risk Status

Prior Skills & Engagement

Computer Programming

Mathematical Skills

Writing Skills Communication Skills

43%

50% 50%45%

20%

37%42% 40%

Weak and Availed of the Services Strong and Availed of the Services

Students Reasons For Considering Leaving Their Programme Of Study Or Staying On

33% of respondents have considered, at this stage, leaving their current programme of study with 65% of these falling into the At Risk category. For the 67% who had not considered leaving only 38% fell into the At Risk category.

23% of those students who considered leaving their programme availed of our SALS services, whereas 27% of those who did not consider leaving availed of the services.

Subject difficulty of the course (I am falling behind in some subjects) 4.08Course was not as I expected 2.81Unsure of my career objectives 2.68Difficulty in managing my time (e.g. part-time job, family commitments etc.) 2.46

Students’ reasons for considering leaving their programme of study

74% of respondents indicated that they have had difficulties with one or more modules on their current programme.

44% of these, had difficulty with an ICT core module, 33% with a maths module, and the remaining 23% with other modules (e.g., accounting, chemistry, engineering mechanics).

41% of those who had considered leaving due to module difficulty had availed of SALS services, compared to 59% of those who had not considered leaving.

Students’ Difficulties with Modules

Responses suggest that 57% of respondents allocate 3 hours or less per week to module study

Students who allocate more hours per week for their studies being more likely to avail of the SALS services and less likely to be At Risk.

Study Time, At Risk Status & SALS Uptake

<1 1-3 3-6 >6

60%64%

50%

11%

30%

41%

63%67%

At Risk Availed of Support Services

Number of Hours Spent Studying

Reasons For Availing Of SALS Services

42% of surveyed students have availed of one or more of our services with only 40% of this group being At Risk. For those who did not avail, 51% were At Risk.

Answer Options 1 - Not Important 2 3 4 5 - Very

ImportantTo get higher grades 0% 5% 1% 28% 63%To pass my exams 3% 3% 7% 23% 61%To consolidate my grasp of programming/electronic design concepts 3% 4% 16% 28% 44%

To meet people from the same course and make new friends by working and learning in smaller groups 32% 25% 21% 7% 3%

To develop my specific skills in time management, communication, study, presentation and critical thinking

21% 19% 19% 16% 13%

Lecturer/TA suggested that I use these supports 16% 11% 28% 15% 13%Pre-exam revision support 4% 8% 15% 27% 35%

60.5% of respondents felt that the services have helped them achieve higher grades and/or pass their exams.

About 31% indicated that the services they availed of have helped them integrate into their academic programme and/or make new friends from the same course.

Student Perceived Utility of SALS Services in Achieving Their Goals

Answer Options 1 - Strongly Disagree 2 3 4 5 - Strongly

AgreeI achieved higher grades than I would have without the support. 3% 5% 21% 28% 33%I passed my exams. 4% 8% 20% 31% 29%The support has helped me to understand the more difficult content of my modules. 5% 8% 24% 28% 29%I enjoyed meeting and working with people from the same course and have made new friends. 12% 15% 33% 19% 9%The support helped my integration into my course. 13% 15% 25% 21% 12%

Perceived Utility of the SALS Services 1

Answer Options 1 - Not At All 2 3 4 5 - Very

MuchUnderstanding difficult subject content 5% 12% 32% 31% 20%Problem solving 5% 15% 38% 30% 12%Interpersonal Communication Skills 17% 25% 42% 10% 7%Learning & Study techniques 20% 27% 40% 10% 3%Presentation 27% 25% 35% 8% 5%Team work 19% 28% 36% 12% 5%Critical Thinking 14% 12% 42% 25% 7%Making new friends 25% 29% 25% 14% 7%Time-management & Productivity 20% 31% 31% 10% 8%Enjoyment 12% 26% 33% 22% 7%Attendance of classes 15% 20% 37% 14% 14%

Drop-In sessions

• For students who availed of the SALS services, the percentage who found the service helpful or very helpful were 60% for the Drop-In Services, 68% for the PSLG Sessions and 55% for the Targeted Sessions.

Perceived Utility of the SALS Services 2

Answer Options 1 - Not At All 2 3 4 5 - Very Much

Understanding difficult subject content 7% 13% 31% 24% 24%Problem solving 6% 18% 34% 30% 12%Interpersonal Communication Skills 11% 23% 38% 21% 8%Learning & Study techniques 12% 25% 38% 17% 8%Presentation 22% 28% 31% 12% 6%Team work 20% 23% 36% 11% 11%Critical Thinking 13% 20% 39% 20% 8%Making new friends 27% 27% 28% 10% 7%Time-management & Productivity 23% 24% 29% 14% 11%Enjoyment 15% 27% 28% 21% 9%Attendance of classes 18% 16% 31% 15% 19%

PSLG sessions

Answer Options 1 - Not At All 2 3 4 5 - Very Much

Understanding difficult subject content 19% 9% 37% 21% 14%Problem solving 16% 7% 42% 21% 14%Interpersonal Communication Skills 21% 12% 50% 7% 10%Learning & Study techniques 21% 7% 42% 21% 9%Presentation 23% 12% 49% 7% 9%Team work 26% 14% 45% 5% 10%Critical Thinking 22% 10% 41% 17% 10%Making new friends 31% 17% 33% 12% 7%Time-management & Productivity 24% 12% 43% 7% 14%Enjoyment 19% 14% 50% 7% 10%Attendance of classes 24% 10% 33% 14% 19%

Targeted sessions

“Everyone that helped me had great knowledge and experience with the areas they specialize in and everyone was very approachable.”

“There have been useful sessions for modules such as Databases and Software Testing and the Peer Supported Learning Group covers Java for Mobile apps.”

“I'd like it if even more competitions were pushed by the centre, internal competitions (similar to collegiate) and support for external competitions (Dare to be digital, etc.).”

“It is a great environment to do work and collaborate; everyone there is very helpful and very friendly. Good atmosphere.”

“The ICT centre is an excellent facility. The books and supporting tutors are of great help for IT subjects.”

“The facility is well publicised and has an excellent reputation amongst students.”

“The ICT is excellent and everyone involved should be very proud. It's a great help.”

“Would really not have managed my Java module without the help and support from ICT Learning Centre.”

“All I want to say is the centre should be running as it provides invaluable free support in all aspects of course materials.”

Students’ Comments on SALS Services

Our SALS services seem to align closely with best practice/characteristics of successful retention approaches.

SALS services offered at UL had a significant impact on the continuation and progression of ICT students (19% higher progression rate for Engaged over Not Engaged students).

The impact is more significant in the case of the students identified as At Risk (26% higher progression).

Engaged students achieve significantly higher rates of passing their core programming modules in both semesters, attaining at least a C3 grade (≈16% higher on average).

The engagement level of At Risk students is 8% lower than those who are Not At Risk.

We have also identified other potential markers for early identification of At Risk students, e.g. course match, CAO points, perceived prior knowledge in core subjects.

Conclusions – Historical Data Analysis

Students whose current programme of study has not been their 1st choice are twice as likely to fall into our At Risk category.

Students who consider themselves weak at maths and programming are significantly more likely to be At Risk.

Students who had considered leaving their current programme of study were nearly twice as likely to fall into our At Risk category.

Module difficulty seems to have been a significant factor in making students consider leaving their programme with 65% of these falling into the At Risk category.

60.5% of respondents felt that the SALS services have helped them achieve higher grades and/or pass their exams.

Conclusions – Student Survey

Services similar to SALS would be highly beneficial in any HE institution offering ICT programmes.

There is a clear need to mainstream these services at institution level with pro-active partnerships between all stakeholders to reach all targeted students.

A more pro-active policy and mechanism for identifying and targeting At Risk students needs to be agreed and implemented in a timely, systematic and continuous manner.

Expansion of the SALS services with a specific focus on collaborative and student-centred learning approaches in order to further nurture independent and engaged learners and social inclusion.

Recommendations 1

It is essential for any successful retention initiative to be inclusive of all stakeholders. No single entity can effectively achieve all retention objectives on its own. As such, retention is the responsibility of all staff in an institution.

SALS units should expand their out-reach activities to target pre-entry students in order to excite and inform them about computing and ICT-related study programmes and careers.

SALS units need the support and opportunities to further develop their research activities in this context in partnership with relevant academic departments and staff in order to undertake further in-depth investigations and to facilitate more longitudinal research studies.

There is a need to expand this research to incorporate more HEIs so that an in-depth insight into progression and retention issues in ICT programmes in Ireland can be obtained.

Recommendations 2

Any Questions?