Engaging Sociology Students in the Numbers Game: The Use of Active

Learning Strategies in Undergraduate Methods Teaching

Dr Ciaran Acton and Dr Bernadette McCreight

School of Sociology and Applied Social Studies University of Ulster

HEA Workshop and Seminar Series 2014

Making Undergraduate Social Science Count

Outline Summary of the HEA TRM Project (Completed 2013)

Context

Overview of research methods strategy in Sociology

Student population

Assessment & Feedback

Formative Assessment (UU Principles)

Examples

Findings

Next steps

Follow-up HEA Project

Outline of the HEA TRM Project

Aims of the Project • To obtain student feedback on AfL strategies

employed on a quantitative methods module

• To use this feedback to develop the assessment strategy and enhance the student learning experience

• To work towards a model of best practice

Research Design • Stage 1 - Questionnaire on students’ perceptions and

experiences of research methods

• Stage 2 – Focus group interviews

Developing Research Skills

Research-Related Modules Year 1 – Introduction to Qualitative Research Methods

Year 2 – Quantitative Methods; Research Methodology

Year 3 – Sociological Placement; Dissertation

Explicit links to other modules Year 1 – Introduction to Sociology; Sociology of Health

Year 2 – Information Society; Modern Ireland

Year 3 – Sociology of Work; Sociology of Education

Widening participation UU has a large proportion of students historically

under-represented in HE

• 45.8% of full-time degree entrants in UU in 2011/12 from NS-SEC Group 4-7 (Benchmark = 34; UK average = 31.2)

• Impacts on certain disciplines more than others and creates challenges for the teaching of particular parts of the sociology curriculum

Quantitative Research Methods Module

80 to 100 students

Diverse group (socially and academically)

• 67% of students’ parents did not attend HE

• 46% Grammar; 42% Secondary; 12% Other

Background to the Module Aims of the module

To give students the opportunity to develop the research skills necessary to devise, administer and analyse the results of a social survey

To equip students with the ability to interpret and critically assess quantitative research findings published within the field of sociology

Content and Delivery

Twelve 2-hour lectures

Twelve 1-hour seminars

Twelve 90 minute computer-based workshops

Assessment and Feedback Summative Assessment

Research Report (2,500 words)

End of module exam (3 hours)

Formative Assessment

Tutor and peer feedback

1. Seminar Activities (group work)

(a) Questionnaire Design

• Choose a topic; design 6 questions; pilot the questions

(b) ‘Reading’ Quantitative Data

• Interpreting quantitative data in published research (e.g. Ark Research Updates; Sociology articles)

ARK Research Updates

Tutor and Peer Feedback (cont) 2. Computer-based Workshops

Informal ‘no stakes’ assessment on SPSS

Student learning in the workshops is assessed on two occasions (descriptive statistics; inferential statistics)

Collaborative learning

3. Interactive Lectures

The use of personal response system (PRS) technology

(a) Assessing students’ understanding of the lecture material

• Allows the lecturer to identify problems at an early stage

• Anonymity makes it easier for students to participate

(b) Promoting active learning

• Encourages reflection on statistical problems

• Facilitates collaborative learning

Correlation Matrix (SPSS Output) Correlations

1 .614** .063* -.022 .003 -.302**

. .000 .027 .528 .886 .000

1796 1319 1251 816 1796 1534

.614** 1 .099** .056 -.022 -.202**

.000 . .003 .149 .418 .000

1319 1321 907 669 1321 1142

.063* .099** 1 .286** .029 -.085**

.027 .003 . .000 .303 .005

1251 907 1252 593 1252 1082

-.022 .056 .286** 1 .035 -.043

.528 .149 .000 . .319 .254

816 669 593 817 817 708

.003 -.022 .029 .035 1 -.003

.886 .418 .303 .319 . .909

1796 1321 1252 817 1800 1538

-.302** -.202** -.085** -.043 -.003 1

.000 .000 .005 .254 .909 .

1534 1142 1082 708 1538 1538

Pearson Correlation

Sig. (2-tailed)

N

Pearson Correlation

Sig. (2-tailed)

N

Pearson Correlation

Sig. (2-tailed)

N

Pearson Correlation

Sig. (2-tailed)

N

Pearson Correlation

Sig. (2-tailed)

N

Pearson Correlation

Sig. (2-tailed)

N

RAGE S7Q2b - Age -

Respondent

P2AGE S7Q2b - Age -

Person 2

PERCAP1 Income per

person in household

RHOURSWK S7Q6a -

Hours worked per week

VIEWS11P Views on

11+, high = disaf f ected

TEENBABY Att itudes to

teen pregnancy , low =

against teen pregnancy

RAGE S7Q2b

- Age -

Respondent

P2AGE

S7Q2b - Age

- Person 2

PERCAP1

Income per

person in

household

RHOURSWK

S7Q6a -

Hours worked

per week

VIEWS11P

Views on

11+, high =

disaf f ec ted

TEENBABY

Attitudes to

teen

pregnancy ,

low =

against teen

pregnancy

Correlat ion is signif icant at the 0.01 lev el (2-tailed).**.

Correlat ion is signif icant at the 0.05 lev el (2-tailed).*.

Discuss the correlation matrix with the person next to you and consider the following question: How would you describe the relationship between age and attitudes to teenage pregnancy?

1. Sig., +ve & strong

2. Sig., -ve & strong

3. Sig., +ve & weak

4. Sig., -ve & weak

5. Not significant

Sig

., +ve

& s

trong

Sig

., -v

e & s

tron

g

Sig

., +ve

& w

eak

Sig

., -v

e & w

eak

Not

sig

nifica

nt

6%12%

0%

82%

0%

Selection of Results from Survey & Focus Groups

Questionnaire adapted from Williams (2007)

• Student perceptions and experiences of quantitative methods

• Disseminated to 2nd and 3rd year students (Sociology & Sociology with Criminology)

• Key themes:

Attitudes to quantitative methods

Students’ experience on the module

Assessment and feedback

Three Focus Groups with 2nd & 3rd year students

• Addressed questionnaire themes and other issues (barriers to learning; developing student confidence; applying research skills)

Fears and anxieties Questionnaire Results

• All have GCSE maths (or equivalent) but 28% had ‘a bad experience’ of maths at school

• 34% said ‘the idea of learning statistics makes me feel anxious’

Focus Groups ‘I was petrified. It scared me a lot. When I heard computers I was really frightened. I am more confident now that I could actually sit down and use SPSS.’

‘When I used to see tables I never knew how they got there. I didn’t know sociologists had to do this sort of work.’

‘Whenever I heard it was going to involve numbers I thought I was going to fail it.’

Assessment and Feedback 1. Seminar Activities

Questionnaire Results

• 86% ‘agreed’ that giving feedback on other students’ survey questions ‘helped me to develop my own understanding of questionnaire design’

• 73% felt ‘confident’ or ‘very confident’ in their ability to ‘read journal articles which discuss the findings of quantitative research’

• However 35% of students ‘preferred to work on my own rather than within a group environment’

Assessment and Feedback 1. Seminar Activities (Questionnaire Design)

Focus Groups

‘Designing the questions was really difficult.’

‘You have to get your hands dirty. We thought [this] would be so easy but it wasn’t. How wrong were we?’

‘We thought our questions were brilliant, but the pilot proved they weren’t. If we had to do the questionnaire ourselves it would be really scary.’

‘Questionnaire design was difficult if other people did not turn up.’

Assessment and Feedback 1. Seminar Activities (‘Reading’ Quantitative Data)

Focus Groups

‘I can now criticise other research papers.’

‘Quantitative statistics are in a lot of articles I would read for most modules.’

‘Especially those journal articles. You are more confident to have a flick through it instead of turning the page.’

‘There was lots of statistics in the criminology fear of crime module. It gives us an advantage because we can now look at the statistics and are familiar with them.’

Assessment and Feedback 2. Computer-based Workshop Assessments

Questionnaire Results

• 87% ‘agreed’ that the workshop assessments ‘enabled me to assess my ability to analyse data using SPSS’

• 91% ‘agreed’ that the workshop assessments were ‘good preparation for the research project’

• However, 53% felt that ‘marks should be awarded for the workshop assessments’

Assessment and Feedback 2. Computer-based Workshop Assessments

Focus Groups

‘When it came to carrying out the research project you could do it because you had had the practice beforehand. The workshops allowed you to try it out.’

‘The computer test on the Life and Times Survey was helpful.’

‘It give you a chance to play with it and work out what you had to do.’

Assessment and Feedback 3. Interactive Lectures (PRS)

Questionnaire Results

• 67% ‘agreed’ that PRS ‘encouraged interaction with other students during the lecture’ (31% ‘undecided’)

• 70% ‘agreed’ that PRS provided them with the opportunity ‘to gauge my understanding of the lecture material (26% ‘undecided)

• 59% ‘enjoyed’ using PRS (35% undecided)

Assessment and Feedback 3. Interactive Lectures (PRS)

Focus Groups

‘The questions were good, instead of just a 2 hour lecture.’

‘It gave you the opportunity to know what to do. If you didn’t know the answer you knew you were behind and you had to push yourself a bit more.’

‘It gave you time to let the information absorb in, rather than just a 2 hour lecture where you are being fed information. You can test yourself.’

‘It’s good the way it is anonymous, you know, if you get the answer wrong it’s not like you’re ashamed.’

Key Lessons

Giving students the freedom to make mistakes

Embedding research skills throughout the degree

Developing students’ understanding of assessment and feedback

Building confidence

• ‘It was the one module I did best on, yet it was the module I thought I was going to fail.’

Next Steps

HEA TRM Project (Stage 2):

Embedding Quantitative Research Skills within the Undergraduate Curriculum

Builds upon the success of two previous projects:

• Engaging Students in Quantitative Methods (UU)

• Studying Northern Ireland (QUB)

Aims to develop a model for embedding quantitative research skills throughout the curriculum with a particular emphasis on the use of existing datasets and resources

l

HEA TRM Project (Stage 2)

Utilise the new NILT Teaching Datasets & Resources

Foster closer links with ARK

Introduce guest speakers (e.g. those centrally involved in large scale survey research)

Increase students familiarity with quantitative data throughout their degree and raise confidence

Ensure progression by focusing on all three years of the degree

Progression across the degree

Level 4 (1st Year)

Introduction to Sociology

Sociology of Health and Illness

Level 5 (2nd Year)

The Information Society

Modern Ireland

Level 6 (Final Year)

The Sociology of Work

The Sociology of Education

Dissertation

Moving forward HEA report

Ongoing development based on student feedback & evaluation

Closer links with employers (part of new employability strategy)

Closer links with QUB (Q-Step Programme)

Feedback and advice from colleagues in cognate disciplines and other universities

Networking events HEA Workshop