13/12/2013

1

Research Seminar for

Educational Sciences

Prof. Dr. Chang Zhu

Department of Educational Sciences

Overview

• Quantitative research

• Qualitative research

• Questionnaires

• Interviews and observations

• Sampling

2

13/12/2013

2

• Make questions clear: word questions as clearly

as possible

• Keep questions short

• Collect demographic information based on the

need of the study

3

Questionnaire

Questionnaire

• Include only items that are relevant

• Define or explain them if some terms are

not familiar for the participants.

• Use examples if item format is unusual.

4

13/12/2013

3

Questionnaire

• Avoid leading questions: A leading

question is one that suggests a certain

response (either literally or by

implication) or contains a hidden

premise.

• Avoid ambiguous, and define or explain

them if some terms are not familiar for

the participants.

5

Questionnaire

• Cover letter or brief introductory or

directory words

� Why this questionnaire

� How the data will help the respondent and

the field in general

� How to respond

� Endorsement or approval by certain

admin/organization if applicable

6

13/12/2013

4

Questionnaire

I am happy to work in this school.

I am happy in my job as a teacher.

There is no better job than to be a teacher.

If I could choose again, then I would change the teacher

profession for another job. (R)

I like to be in my class.

I don't like to be in my class. (R)

I want to remain in the education for my job.

7

An example: well-being of teacher

Reliability

• E.g. measuring your weight 4 times a day

(is it reliable?)

• Testing your statistics knowledge 3 times a

week…

• Asking the well-being of a student with 5

different questions….

8

13/12/2013

5

Questionnaire

• Pilot testing

� A smaller sample

� Understanding of the questions

� Identify problems if any

� Provide suggestions for modification

9

Questionnaire

• Questionnaires (written)o Larger sample

�By mail

�Via Email

� online

• In-person question-and-answer (oral)o Smaller sample

� Telephone

� In person10

13/12/2013

6

Questionnaire

• Questionnaires

� Deadlines

� Follow up if possible (a reminder)

� Consider the convenience of respondents (e.g.

return envelope)

• Ethics

� Anonymity, confidentiality

� No harm, not threatening questions

11

Online survey

1

2

13/12/2013

7

Online survey

1

3

Online survey

1

4

13/12/2013

8

Online survey tools

• LimeSurvey

• https://www.vub.ac.be/survey/index.php?lang

=en

• Survey Monkey

• http://www.surveymonkey.com

• Qualtrics

• https://www.qualtrics.com/

• VUB/ULB Rekencentrum (rekencentrum@vub.ac.be)

• OSuCre1

5

1

6

13/12/2013

9

Interview

� Individual

� Group

� Focus group interview

– several individuals

– ensure all have their say

– individuals can interact, respond

– possibly lead to a shared understanding..17

Interview

• Understand themes of the world from the

subject’s own perspectives

• Capture the experiences and lived meanings of

the subjects’ everyday world (from their own

perspectives and in their own words)

18

13/12/2013

10

Interview

• Can obtain data that cannot acquire

from observation alone; gather in-depth

data about the experiences, views,

feelings of participants; explore reasons

• Can be structured, unstructured or semi-

structured

19

Interview questions

• Don’t ask two things in one question. E.g.

do you find online communication a

problem and the teacher should give more

guidance?

• Don’t ask Confusing or wordy questions

• Avoid unrelated questions

• Avoid Yes-No Answers.

20

13/12/2013

11

Interview questions

Interview questions

o E.g. ‘Can you tell me about…’

‘what do you think of…’

‘can you describe…’

‘can you say something more about…’

‘do you mean that….’

21

Interview

Interviewing

• Setting the stage, briefing

• Encourage the interviewee to describe their

point of views

• Follow up questions, clarifications

• If needed, summarize or round off (and ask

confirmation)

22

13/12/2013

12

�Interviewing

– listen more

– don’t interrupt

– avoid leading questions; ask open-ended

questions

– keep participants focused; follow up on

what they say

– do not judge or debate

23

Interview

� Collecting data from interviews

– take notes during the interview

– audio- or video-taping the interview

– write notes after the interview

– transcription of recording; write down

the data, subjects, and participant name

or codes

24

Interview

13/12/2013

13

• Some data can be observed in natural setting

• More objective information than self-reports

• Recording observations

• Field notes (description and understanding of the

research setting and participants, as detail as

possible)

• Recorded data and filed notes can be organized,

categorized and analyzed.

25

Observation

Observation

� Observational notes

• descriptive notes: detailed notes about

what occurred, what the observer sees,

hears, the actual setting, etc.

• reflective notes: about the observer’s

thoughts, impressions, personal reactions,

experiences

26

13/12/2013

14

Validity

� Validity in qualitative research

• descriptive: factual accuracy

• interpretative: are the meanings or words of

participants correctly interpreted?

• theoretical: how is the studied phenomenon

relate to a broader theory?

• evaluative: does the researcher report in an

objective, unbiased way?

27

Validity

� Strategies for ensuring the validity

– talk little, listen a lot

– record accurately

– begin writing early

– report fully

– write accurately

28

13/12/2013

15

A short discussion

• What kind of research method you would use for

your master thesis research?

• How are you going to collect your data?

• What would be your target sample?

• How many sample participants to reach?

2

9

Sampling

A. Defining a population

B. Selecting a sample

30

13/12/2013

16

Population

• What is your population of interest?

• To whom do you want to generalize your

results?

– All doctors

– All university students in Flanders

– All secondary school pupils

– Women aged 25-45 years

– Other

• Can you sample the entire population?

3

Sampling

• A sample is “a smaller (but hopefully representative) collection of units from a population used to determine truths about that population” (Field, 2005)

• Why sample?

– Resources (time, money) and workload

– Gives results with known accuracy that can be calculated mathematically

• The sampling frame is the list from which the potential respondents are drawn

– Registrar’s office

– Class rosters

– …..

3

13/12/2013

17

Sampling

• 3 factors that influence sample representativeness

• Sampling procedure

• Sample size

• Participation (response)

• When might you sample the entire population?• When your population is very small

• When you have extensive resources

• When you don’t expect a very high response

3

Sampling

34

13/12/2013

18

Sampling

3

5

TARGET POPULATION

STUDY POPULATION

SAMPLE

The sampling process…

POPULATION

SAMPLE

INFERENCE

13/12/2013

19

Sampling

Selecting a random sample

• Simple random sampling

• Systematic sampling

• Stratified sampling

• Cluster sampling

37

• Simple random sampling

38

13/12/2013

20

Sampling

• Systematic sampling

3

9

Sampling

4

0

• Stratified sampling

13/12/2013

21

Sampling

• Cluster sampling

41

Sampling

C. Selecting a non-random sample

• Convenience sampling

• Purposive sampling

• Quota sampling

D. Qualitative sampling

• Representative

• Often purposive

• Criterion sampling

42

13/12/2013

22

Sampling

Important to bear in mind:

• Is your sample representative of your target

population?

• Are the research results generalizable to your

target population?

• The degree to which the selected sample

represents the population is the degree to

which the research results are generalizable

to the population.43

Sampling

Sample size:

• In quantitative studies, samples should be as

large as possible; the larger the sample, the

more generalizable the results will be…

• In qualitative studies, samples usually

smaller, depending on the type of research

44

13/12/2013

23

Rules of thumb for determining

the sample size...

For smaller samples (N ‹ 100): Survey the entire

population.

The larger the population size, the smaller the

percentage of the population required to get a

representative sample

If the population size is around 1500: 20%

should be sampled.

If the population size is around 500: 50%

should be sampled.

Beyond a certain point (N = 5000): a sample

size of min. 400

Rules of thumb for determining

the sample size...

13/12/2013

24

Sample size

Sample size:

• No universally minimum sample sizes

• Some suggestions for quantitative studies:

if Population=<100, n=<100

if P=500, n~250

if P=1500, n~300

if P>5000, n~400-500 or more

47

Sample size

Sample size:

• No hard rules

• Some suggestions for Qualitative studies:

Can be 1- 60/70 participants

interviews: 15-30 or more, depending on population

size, should be representative of the sub-groups/

multiple contexts…

Case studies: 1-several, depending research needs

48

13/12/2013

25

Sample size

Sample size:

• Two general indicators to determine

whether a sample size is sufficient:

� Representativeness

� Redundancy of information (data

saturation) (in Qualitative research)

49

Sample size

Sample size:

• Avoid sampling error (occurs in the random

selection procedure)

• Be aware of sampling bias (eg. use of non-

random sampling..) (should be noted and

described in the research report)

50

13/12/2013

26

Task 1st semester

• Submit your research paper or the literature

review paper by 31 Jan. 5pm in Pointcarre

Assignment

5

1