quantitative and qualitative methods sebastian m. rasinger research methods in linguistics. an...

Quantitative and Qualitative Methods

Sebastian M. RasingerResearch Methods in Linguistics. An Introduction.

Second EditionLondon: Bloomsbury

S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

Agenda

• Pros and cons of questionnaire use• Design • Coding


Questionnaires - Basics

• Theoretically suitable for both quantitative and qualitative data– But practically a lot more useful for quantitative

approaches

• large amounts of data with relatively little effort?– feedback/return rate– reliability– extensive piloting of usually more than 2 versions before

final run


Basics: know what you want…

• once the questionnaire is completed, there’s no way back needs to be ‘perfect’

• clear aim/hypothesis• is questionnaire the best methodology for

your research? It very often isn’t!• Questionnaire versus (semi-)structured

interviews


Questions: Phrasing• think about what exactly you want to know (again!)

– age or year of birth? i.e. age or cohort effects?• Avoids coding-nightmare afterwards

– definite point in time or period of time?• ‘since when’ vs. ‘for how long’

– sex or gender?• Biological ‘setup’ vs cultural construct?

– children: young? pre-school? Post-comp.ed? general? daughters? sons?

• Unique anonymous identifier


Question phrasing: avoid…

• Ambiguous terms and questions: be clear what you want. You know what you mean, the respondent doesn’t

• Long, syntactically complex questions• Double-barrelled questions: Do you like

your job and the working conditions? 2 questions – ask 2 questions.

• Very general questions: Do you think X is a good idea? usually tell us very little


Question phrasing: avoid… (2)

• Leading questions: Don’t you agree that Mitt Romney would have made an awful president?

• Questions which include negatives – Most people skim read

• Technical terms: you’re an expert, your respondents most likely not

• Does the respondent have the requisite knowledge? Is your respondent literate in the language of the questionnaire?


Open vs closed questions

• Closed:– Quick to respond to and easy to code/process– But: limited scope of info; precise phrasing

• Open– Larger scope with regard to info obtained– But: tedious to complete; better interviews

instead? (NB: no feedback mechanism with questionnaires!)

S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e.

Bloomsbury.

Closed questions – answer options

Should be clear, precise and not overlap

E.g., age group:18-20 21-2424 2626-3018-2020 2020-24 25-30

Have you learnt English before or after puberty?

Yes – No


Closed questions: multiple items• Should include most/all possible answers

– Closely linked to the question – Thorough research at design stage!– Open ended option for ‘emergencies’ only

‘What are your thought about this workshop? – tick one box only’

complete waste of time generally provides useful information is fascinating I couldn't live without it other (please specify) _________________


Multiple items – layout • Layout – which box belongs to which answer?

Strongly agree agree undecided disagree strongly disagree

vs.

Strongly agree Agree Undecided Disagree Strongly agree


Bloomsbury.

Scales and semantic differentials

• E.g. Likert scales• Measuring attitudes/opinions/agreement/

disagreement• Even or odd number of items?

– Neutral option or not?• 3, 5 or 7 point?

– Depends on degree of accuracy/detail required: to few not detailed enough; too many: tedious


Bloomsbury.

Scales (cont’d)‘Research methods is the most important module of my

MA’AgreeAgree Neither Neither

agree nor agree nor disagreedisagree

disagreedisagree

33 22 11

Strongly Strongly agreeagree

AgreeAgree Neither Neither agree nor agree nor disagreedisagree

disagreedisagree Strongly Strongly disagreedisagree

55 44 33 22 11

Very Very strongly strongly agreeagree



DisagreeDisagree Strongly Strongly disagreedisagree

Very Very strongly strongly disagreedisagree

77 66 55 44 33 22 11S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e.

Bloomsbury.

Number and sequencing

• As many questions as you need… but not more– E.g., age and DOB together often unnecessary

• Simple, easy and uncontroversial qs first• Thematic blocks (but beware of response

sets!)• Give clear instructions throughout:

– ‘If “yes”, go to q.14. If “no”, continue with q.3’


Bloomsbury.

Problems…• Response sets:

– Respondent’s tendency to agree of disagree

• Acquiescence responses/social desirability responses– Do not reflect respondent’s actual opinion but what

respondent thinks it the ‘right’ or desirable answer

Reliability and validity issues


Bloomsbury.

… and ways to solve them

• Bi-directional questions:– Measure opinion one way– And then the other way round

‘Research methods modules are completely unnecessary’ (yes/no)

‘Research methods modules are important to develop key skills.’ (yes/no)


Bloomsbury.

Coding

• =translation of data into numerical values so it can be used for quantitative analysis

• Open questions: either qualitative approach or development of detailed coding scheme

• Closed questions: – Data that is already numerical (e.g. DOB, age)

can be used straightaway– Data that is not numerical


Bloomsbury.

Coding (cont’d)

• Assign particular numerical value to particular answer optione.g. gender:male=1 female=2 or vice versa

e.g. age groups (not age!)18-20 =1 21-24 =2 25+ =3


Bloomsbury.

Coding (cont’d)

• E.g., multiple items: complete waste of time =1 generally provides useful information =2 is fascinating =3 I couldn't live without =4 other (please specify) __________ (additional

coding)


Bloomsbury.

Coding – scales

• ‘self coding’• if you want to combine questions/variables into an

overall index, make sure all coding runs into the same direction, e.g., all ‘agrees’ are 5 and all disagrees are 1



disagreedisagree Strongly Strongly disagreedisagree

55 44 33 22 11


Bloomsbury.

Coding of scales (cont’d)

• 5 point scales‘Research methods are completely unnecessary’

5=strongly agree

‘Research methods are importantto develop key skills.’5=strongly DISagree


Bloomsbury.

Piloting

• …and debugging• For MA, full pilot (e.g. trial run with smaller sample)

usually unnecessary• BUT: always a good idea to test whether they work

– Phrasing (and spelling)– All answer options included?– Etc…

• Use friends, partners, children, supervisors…


Bloomsbury.

quantitative and qualitative methods sebastian m. rasinger research methods in linguistics. an...

Documents