teaching research methods in the social sciences ... · 1 teaching research methods in the social...
TRANSCRIPT
1
Teaching research methods in the Social Sciences
Facilitator: Natasha Taylor
Leeds, 21 May 2015
Joan Smith and Phil Wood, University of Leicester
Abstract
The presentation outlines an action research project in which we trialled strategies to
develop EdD students’ critical writing and peer reviewing skills, by engaging them in a
process of peer assessment. The project began with an intensive, residential critical
writing weekend and culminated in the establishment of a student-led, peer reviewed
journal. We report on students’ perceptions of their involvement in the project as a
transformational experience. Students were interviewed before and after the
intervention, and reported shifts in their understanding of criticality, peer review and
critical writing. Perhaps more importantly, there were significant shifts in their
perceptions of themselves as capable of and confident in their criticality. Learning was
enhanced through critical friendship, and students derived a sense of belonging to a
research community in which they were playing a meaningful role. Critical friendships
were established that would thrive beyond the life of the project. We consider the
implications of the project for teaching and supervisory pedagogies at postgraduate level.
2
Developing doctoral students’ critical writing and
reviewing skills through peer assessment
Developing doctoral students’ critical writing skills through peer
assessment Joan Smith, Phil Wood, Gareth Lewis & Hilary Burgess
(University of Leicester School of Education) Project funded by HEA Social Sciences Strategic
Priorities Teaching Research Methods in the Social Sciences
Issues
• Challenges of developing critical writing (students and supervisors)
• Affective barriers to accepting feedback
• Isolation of school-based researchers
• Need to develop pedagogies to develop criticality & resilience, & induct students into peer review
The project: ‘writing as social practice’ (Kamler & Thomson, 2006: 5)
Students were offered opportunities to work with others to:
• Develop an awareness of what critical writing is
• Practise and develop their own critical writing
• Provide constructive critical feedback via a peer reviewing system to other students;
• Become integrated into a research community in a meaningful way.
The project
• Intensive residential critical writing weekend
• Forming of editorial board
• The Bridge https://journaleducationalresearchinformedpractice.wordpress.com
• Summer School workshop led by students
Data collection
• Pre- and post-weekend interviews
• Notes taken during roundtable
• Students’ written feedback on drafts
• Observation notes from editorial board meeting
• The Bridge
Initial findings • Understanding of critical writing considerably enhanced through
engagement with the process
• Shift in understanding of criticality as negative and destructive to supportive and developmental (caring?)
• Parallels drawn between giving feedback & teaching (caring?);
• Shifting identity from initial alienation and feelings of inadequacy to growing sense of positive researcher identity: key difference & keyword marking the ‘before’ and ‘after’ is ‘confidence’
• Apparent that this is less to do with technical skills; much more bound up with affect, emotion, anxiety and belonging
• Evidence that learning is enhanced via social aspects of the peer assessment and review experience, and engagement in a community
• Critical friendships established/strengthened through peer assessment and review
• Support group moved from camaraderie to critical friendship
Emotion, affect, anxiety, insecurity ‘I think I’m going to find [peer reviewing] quite challenging. .. .Partly, because they’re writing in an area that’s not my area. So I don’t really know much about it. ..So I end up saying really pathetic things like ‘That's really interesting’ and then I think ‘what use is that to anybody?’ …[It’s difficult] partly because I don’t know [the other students] well enough yet. ..So …we’re having a congenial conversation instead of a deep conversation… we’re all being terribly polite and nice to each other…We don’t know enough about what each other is doing in order to be critical…somebody gave me one of their scripts to read and my first reaction was ‘this is too descriptive’. There were no references in it. It was very narrative. But it was just her first, you know, initial thoughts. She’d got it down on paper and was going to go away and work on it. So I think it’s just…it’s easier not to say anything, because you’re just being destructive, not helpful… I didn’t feel sure of my grounding…’ (Eloise, Interview 1)
Emotion, affect, anxiety, insecurity
‘I’ve read some of the other students’ stuff … but the problem is … you don’t want to upset anyone. You look at their work…but you just say, ‘Oh, it’s good’, and…we’re not really doing anyone any favours there’ (Peter, interview one).
Shifting identity & growing confidence
‘I felt much more confident by the end of the weekend. I understand what critical writing looked and felt like. I think beforehand, we’ve had a couple of meetings at the library, my little, sort of, support group, and I’ve not really felt able to give anybody any meaningful feedback about their writing, and we didn’t know where to start. And it’s just having that structure, in a pro-forma that was provided to sort of give the feedback in the beginning and emailed out before the course started that just helped to…I understood what I was being asked to comment on and how.’ (Eloise, interview 2)
Shifting identity & growing confidence ‘When I sit down at the table with our peers, really, I’m probably the least confident of anyone there, you know, because I’m not always that confident in my ability… In the past I’ve worked on group stuff before and I haven’t really contributed that well, because I haven’t felt…everyone’s seemed to understand what’s going on except me… it’s always in the…back of my head really… when I read my work and read…others’ in the group…peers’, sometimes I feel mine is just too simplistic…I tend to use straightforward words, let’s say non-academic words... that’s something that I know I need to get away from’ (Peter, interview one). ‘You know, when I look back on (the critical writing weekend) now, I thoroughly enjoyed it and I got a lot of things out of it, and the thing is, like I mentioned, … before … to me it’s a confidence thing, and towards the end of it I certainly felt a bit more confident in me, about my own ability’ (Peter, interview two).
Shifting identity & growing confidence
‘Before it was a case of reading it and you might say, ‘Oh yeah that’s good… maybe a couple of grammatical mistakes’ – you wouldn’t go too deep. Whereas I think now we don’t mind stepping that little bit further, you know what I mean? And I think … two or three meetings’ time, I think we’ll be quite OK with it. It is just being prepared to…let’s say there’s something bad in what you’ve done, and they’re really going to point these things out then, you know, you want to have that little bit of a bond…you want a reasonably good bond with them’ (Peter, interview two).
Sense of belonging to a community of researchers
‘I think it was really great [that there were students from all stages of the doctorate involved]. Really enjoyed that, really enjoyed meeting them all, they’re all just such lovely people and it makes you feel, I think it helps that we’re all distance learners, even people like me who live nearby. Everyone’s all scattered to the four winds that it, it just gives you a feeling of belonging to somebody, that you don’t get as the kind of student we are’ (Eloise, interview 2)
Final thoughts
• Residential weekend transformational and empowering: students taking work forward
• Importance of critical friendships
• Too important to leave to chance!
References
Kamler, B. & Thomson, P. (2006) Helping doctoral students write: pedagogies for supervision. Abingdon: Routledge
Liam Foster & Tom Clark, University of Sheffield
Abstract
Whilst recent funding initiatives are a welcome development, there is continuing concern
about the level of quantitative skills possessed by UK social science students specifically
and their levels of employability more generally. Indeed, the difficulties of developing such
experience is likely to remain particularly acute for those students, institutions and
departments who find themselves outside the reach of those initiatives. This paper
explores the inquiry-based strategies we have been developing in the Department of
Sociological Studies at the University of Sheffield. Designed to enhance students’
experience of doing sociological research, we will discuss how, with the assistance of the
HEA, we developed of a series of student-led quantitative workbooks, worksheets, and
associated teaching strategies and subsequently embedded them within a ‘research
methods spine’. The paper considers some of the difficulties of employing an inquiry-
based framework and outlines some considerations for those wishing to adopt a ‘student-
focused’ and ‘research-led’ approach to the learning and teaching of research methods.
17
Embedding inquiry-based approaches to quantitative
research within a 'research methods spine'
Embedding inquiry-based approaches to quantitative research within a ‘methods spine’
Tom Clark and Liam Foster [email protected] and [email protected]
28/05/2015 © The University of Sheffield
Contents
• The wider context of teaching quantitative
research methods
• Inquiry-based learning
• Developing ‘a methods spine’
• Doing quantitative sociological research
• An integrated approach
Wider context – top down
• Lots of ‘top-down’ movement toward quantitative social research
• Continuing drives toward promoting statistical skills among social
scientists (Nuffield/ESRC/HEFCE funding)
• Employment
• Post-Graduate Research (and beyond)
• Big data?
• High quality data – and lots of it
• Research mindedness in a research saturated society
28/05/2015 © The University of Sheffield
Wider context - bottom up
• From the perspective of UG students, there are any
numbers of constraints – and opportunities
• ‘my level of ability is…in a formative stage…I’m not a natural
mathematician’
• ‘literature surrounding doing quantitative research is often quite
inaccessible and dry’
• But…
• ‘I would like to learn more about quantitative research as quite often
interesting and/or relevant facts within sociology that support points both
in my own essays and within general reading are of a quantitative
statistical nature’. Flo
28/05/2015 © The University of Sheffield
Esther says…
• ‘I think I’m weaker on analysis because we haven’t ever
had to actually do it, and the whole maths thing puts me
off a little bit. I think it would be good to learn more about
quantitative research ... I think that most students who
do sociology that I know struggle with thinking
mathematically, which puts them off wanting to do
anything with statistics because it seems a bit
complicated’
28/05/2015 © The University of Sheffield
Elina says...
• ‘I feel that my biggest weakness at the moment in
quantitative social research is statistical techniques and I
find that it makes quantitative research seem more
complex. In addition, as it is an important part of the
research process I feel that in order to consider
conducting a quantitative social research it would be
essential for students to learn the basics in the simplest
form’
28/05/2015 © The University of Sheffield
A methods spine
• 5 (core) modules across three levels
• Introduction to social research (L1: 10 credits)
• Doing social research (L1: 10 credits)
• Doing quantitative sociological research (L2: 20
credits)
• Doing qualitative sociological research (L2: 20
credits)
• Dissertation (L3: 40 credits)
28/05/2015 © The University of Sheffield
Doing quantitative sociological research
• Aims to provide a theoretical and practical foundation for conducting
independent quantitative sociological research
• Design and sampling
• Data collection (primary and secondary)
• Descriptive and inferential analysis (bivariate)
• Assessment
• Poster presentation (survey-based ‘research commission’ –
group-led)
• Research report (using ESDS data-sets - individual)
28/05/2015 © The University of Sheffield
28/05/2015 © The University of Sheffield
How do we teach it?
• Blended learning
• 16 lectures, 2 lab-based workshops, 7 group-focussed seminars, 5 inquiry-
based workbooks
• HEA funding enabled us to continue to develop workbooks used within the
previous courses in our home department (see Foster and Clark, 2013).
• ‘Research Rationales, Research Questions and Research Hypothesis’
• ‘Designing Variables and Understanding Levels of Measurement’,
• ‘Describing Data’,
• ‘Using Chi-Square, Phi & Cramer’s V’
• A guide to analysing data using PASW (IBM SPSS)
28/05/2015 © The University of Sheffield
Quantitative Workbooks
28/05/2015 © The University of Sheffield
Why do we teach in this way?
• Inquiry-based learning, student-centred, inclusive (peer support)
• Quantitative analysis is not just a series of techniques: it is part of a process of
research – so we start right at the beginning and integrate themes throughout
• Our students think in narratives, not in numbers
• They can, however, understand:
Basic numerical operations
Basic issues in research design
• Alignment between learning and teaching and assessment is vital, but pace of
learning is often variable – secure resources are necessary for students to connect
their L&T with assessment tasks in their own time
• Understanding the ideas and techniques at work behind non-parametric tests (now)
provide a secure and accessible platform for more complex analysis (later)
• Grounded in examples from the UK Data Service, and student assessments
28/05/2015 © The University of Sheffield
What do our students think?
• ‘The information is rich and detailed without being over complicated.
Examples that are relative to students current life experience are
used which creates interest and also gives inspiration for topics of
research relevant to students lives. In terms of experience of
quantitative methods within sociological research these workbooks
are effective as they provide simple exercises in order to practise
methods that are previously only abstract to the student. After
completing them I feel more confident in my own statistical
knowledge. Overall I feel these workbooks would be a very useful
resource to me as a student who both struggles with maths as a
subject and wants to learn as much as possible about this
methodology’. Flo.
28/05/2015 © The University of Sheffield
What do our students think? (2)
• ‘The books definitely enhanced my knowledge of
quantitative social research and demonstrated new
aspects of it and how it can be used in practice ... I was
very pleasantly surprised with how student friendly these
workbooks were and they definitely changed my attitude
towards quantitative research and statistics. Moreover I
feel that I gained a lot from participating in this project
and hopefully will have a chance to use in the future’.
Elina
28/05/2015 © The University of Sheffield
What do our students think? (3)
• ‘I found it interesting because it was put
into a sociological context, so there was
some sort of relevance to it, and the sort of
topics I am interested in were integrated
into the statistics’. Adam
28/05/2015 © The University of Sheffield
What do our students think? (4)
• ‘The work books definitely enhanced knowledge
and experience. They didn’t really give me a
huge interest in quantitative methods (I think I’ll
always be put off a little by the maths side of it),
but everything was explained clearly and easily
and I feel as though I know a lot more about it’.
Esther.
28/05/2015 © The University of Sheffield
Difficulties and challenges
• Integrated curriculum design
• Level of ambition has to match capacity - and be understood in relation
to the programme it sits within
• Embedding method
• More substantive modules need to take some strain
• Contradictory messages from within
• Department (and Faculty) are supportive, but developing open-access
resource is not a ‘research priority’
• Contradictions from outside
• ‘[at A Level] we spent most of the time ripping into quantitative stuff
rather than looking at why it might be useful’ (Adam)
28/05/2015 © The University of Sheffield
But…
• Our IBL research methods modules that include the workbooks are
more popular amongst students than they were when they were
primarily didactic
• Provided a solid foundation for a more advanced L3 (F6) module
‘How to Lie with Statistics’
• Student feedback has not been problematic – which is actually very
positive - students are far from resistant
• Any negative comments are primarily directed to group work and
the use of IT
• More students do go onto conduct quantitative dissertations and some
are going on to PGR study (and employment)
28/05/2015 © The University of Sheffield
Karen Burrows, Amy Pearson and Nick Wragg, University
Centre, Grimsby College
Abstract
Teaching research methods is problematic; research suggests that the terminology is
challenging and is often referred to as inaccessible and ‘complicating common sense’.
This session aims to address these problems by showcasing an innovative and creative
resource in teaching research methods; and the emphasis is on the use of games in the
classroom. Developed over a 2 year period, the resource consists of a board game in
the traditional sense called Methodology: building examples and meaning within higher
education. Players are grouped into teams of 4 and work their way around the board
collecting cards related to research methods. The game is designed to encourage the
students to analyse and evaluate plus consider the appropriateness of research design
and to match their chosen methods to a particular question and aims and objectives.
The game is highly adaptable and can be tailored to the needs of both the tutor and the
student group.
36
Methodology: building examples and meaning
within higher education
HEA Conference, Leeds, May 21st 2015
‘METHODOLOGY Building examples and meaning within Higher
Education’
Karen Burrows, Amy Pearson and Nick Wragg
Background/History
• Originally aimed at Chinese learners
• Encouraging engagement with research methods
• Developed in 2013 from a 3 stage process:
– Problem solving worksheet
– Concept mapping group activity
– Initial board game
Testing and Feedback
• Feedback from HEA Summer Conference 2013
• Internal testing (non-traditional, Chinese learners)
• Quickly became apparent the game engaged a full range of students and staff
• HEA Workshop October 2013
• External testing (selected universities, traditional pathway students)
• Ongoing Peer review leading to current version
Philosophy and Approach
Strategy/Time
Horizons
Data Collection/Data Type
Ethics Research Design Terminology
Research (1)__________ refer to the appropriateness of your (2)__________ in relation to the rights of those who become the subject of your work or affected by the work. They also relate to yourself and ensuring no harm comes to you and other researchers. Ethical concerns can occur at all stages of your (3)__________ project: when seeking access: during data (4)__________: as you analyse (5)__________ and when you report your findings.
behaviour collection data
ethics research
Questionnaire
Sample
Probability
Interview
Quantitative
Qualitative
d _ d _ c _ _ v _
_ n _ u _ t _ v _
_ n t _ _ p r _ _ _ v _ _ _
_ o _ i _ i v _ _ _
_ r _ g _ _ t _ _ m
_ _ a _ i _ m
RESEARCH METHODOLOGY
Board Game
Original
• Chinese Focused
• Emphasis on three stage build
• Methodology Foundations
• Framework – Philosophy
– Approach
– Strategy
– Data Collection Methods
– Time Horizons
Final Version
• Aim to engage all students (differentiation via card booster packs)
• Focus on standalone game – Additional squares, resources
• Methodology Foundations and wider considerations
• Framework
-Philosophy -Strategy -Data Collection Methods
-Sampling -Ethics -Questions and Considerations
Game Playing in HE
With the use of technology an everyday occurrence, students using social media and playing games in these forums, the question posed is, why not use games as a teaching and learning method? This would also provide a useful method for the international learners as arguably game playing (as in the social forums) is universal, perhaps on the same basis as numbers. So game playing would go far in overcoming the language and cultural barriers that international students and UK teachers may find.
Our Game Rationale
The objective is to build a logical research methodology and be able to define and justify methodological choices. The first to build a logical methodology and justify it to the rest of the group successfully is the winner. The methodology is built via card collection with cards you deem to be appropriate kept and recorded on the methodology sheet; warning other players may steal a card you need.
D
ata
co
llectio
n
me
tho
ds
Inte
rview
acce
ss grante
d
by rich
data
sou
rce-Take
a d
ata colle
ction
card
Mo
ve
fo
rwa
rds
3
sp
ace
s...
Qu
estio
nn
aire
resp
on
se rate
s slo
w-m
iss A G
o
Choose your
time horizon
Your tutor has concerns that your research methods do not match your
research question – miss a go!!!
SWOP A CARD
Lap top stolen-
Discard one card of your
choice
Philosophy
Data collection
ETHICS
Move forwards
3 spaces... Philosophy
You critique your
colleagues research
question- STEAL A
CARD from your
opponent
SWITCH
ETH
IC
S
Yo
ur
tuto
r h
as
co
nc
ern
s o
ve
r
pla
gia
ris
m -
go
ba
ck
to
sta
rt
an
d ju
sti
fy y
ou
r
se
lec
tio
n!!
!
Yo
u c
riti
qu
e
yo
ur
co
lle
ag
ue
s
res
ea
rch
qu
es
tio
n-
ST
EA
L A
CA
RD
fro
m y
ou
r
op
po
ne
nt
Da
ta
co
lle
cti
on
m
eth
od
s
Mo
ve
fo
rwa
rds
3
pa
ce
s..
.
‘METHODOLOGY Building examples and meaning within
Higher Education’
• An engaging method for teaching methodology, and judging learning through formative assessment
• Ideal for seminar activities and independent student learning
• Currently in development for manufacture (Available for pre-order – see us for details)
• Booster Packs in the development stage
Impact of Research 1
The HE landscape is changing:-
• QAA Auditing Requirements
• Student as Producer
• Student Voice and Expectations
• Continuing issues over funding
• Continuing dynamism in areas of distance/blended learning
Impact of Research 2
• Paper One - It’s All in the Game – Embedding Research through Game Playing
• Book One – Publishing Contact, The Challenges of teaching and learning – Chinese Students
• PhD Research Project – Karen Burrows
• Continuing research based on progression of boardgame
Thank You
• We welcome you to examine the game in the following session.
Contact us at: