Data Collection and Analysis

Empiricism

Language research is an empirical activity i.e. It involves the collection and analysis of data which can be observed, collected and analysed

Some sources of data

1. Obtaining data from teacher designed language production tasks e.g. a language teaching activity

2. Obtaining data from surveys using interviews or questionnaires

3. Obtaining data from real-life casual conversation

Obtaining data from language production tasks

Aim – to obtain ‘naturalistic’ language data by means of a stimulus e.g. picture or diagram

1. Open ended – analyzing the language used by the student

2. Closed – analyzing the answers to teacher questions about the picture

- What closed questions might the researcher ask about the picture or diagram?

Open and closed responses to a picture prompt

• 1. (Open response) In pairs, look at this picture and take turns to

say what you think is happening in the picture.

2.(Closed response) Compose some closed questions to ask about

the picture

How will the data be recorded?

• Tape record or video the student’s language responses

• You could possibly write it down, if the student speech is quite slow

How will the data be analysed?

Some possibilities:1. Categorise the grammar errors e.g. tense,

missing articles2. Categorise the pronunciation errors e.g.

missing sounds3. List the content items e.g. picture context4. Note the student’s organisation of the

response e.g. how logical is it?

Obtaining data from surveys

Place the following steps in designing a survey in the correct order*:

Decide on the type of data displayIdentify the target groupDecide on a method of analysing dataIdentify survey tools e.g. questionnaire, interviewDefine the survey purpose and goalsDesign when, where etcChoose a sample group

The survey process

Define the survey purpose and goalsIdentify survey tools e.g. questionnaire,

interviewIdentify the target groupChoose a sample groupDecide when, where etc Decide on the type of data displayDecide how to analyse the data

Obtaining data from questionnaires

1. Closed questions – some examples:- What are your qualifications?- What age group are you in 20-40, 41-60- Do you like research – yes or no?2. Ranked response questions e.g. Likert scale3. Open questions e.g. What makes a good language teacher?

What are the advantages and disadvantages of open and closed questions ?

Advantages and disadvantages

• Open questions give a better indication of what the subject thinks and feels, but they can be harder to analyze.

• Closed questions are easier to display and analyze, but will only tell you about the specific questions you have asked.

Designing survey itemsComment on these survey items:1. The best location for the new university would be: A- Vietnam B -

Saigon High School C - Ho Chi Minh City2. Vietnam should spend more on education and less on the army – yes or

no3. The best language to study is obviously English – yes/no4. It is not possible for students not to learn something in class –

agree/disagree5. All Australians are kind – agree/disagree6. Multi focus research projects are desirable but not usual at master’s level

generally speaking – agree/strongly agree/ disagree/strongly disagree/

Interpreting responses

Nunan conducted a survey of teachers’ beliefs about the nature of language and learning. He asked for written responses to the following question: “State three beliefs you have about language development that determine the way you teach”. There were 372 responses.

How would you organise this data?*

• Nunan divided the 372 responses into 3 broad categories, and divided these 3 categories into a number of sub-categories.**

A possible statistical analysis of Nunan’s data

1. He could have organised the data according to nominal groups e.g. males/females, more experienced/less experienced

2. He could have looked for response differences between the nominal groups above.

Obtaining data from interviews

Types of interview questions:

Structured – aims and questions are pre-determined

Semi-structured – the general idea/aim of the interview is pre-determined

Unstructured – the interview’s ideas/aims/ questions are determined by the interviewee’s response from a general topic

Interview Preparation

1. Prepare the interview schedule- aims, type of interview, questions

2. Pilot (practise) the interview3. Select subjects4. Mechanics – time, place, seating, recording

How effective is note taking compared with tape recording?

Obtaining data from conversational analysis

Structured Interview exercise

• Make 5 questions to ask me about my education.

Semi-structured interview

• If it were a semi-structured interview about my education what would the questions be?

What would be your 2 main questions?

Open interview

• You could begin by saying: • What would you like to tell me about your

education?

Interview exercise

• One student interviews another about her/his educational experiences?

Casual conversation

e.g. a group of friends chatting over coffee

- do casual conversations follow a pattern?

- are these patterns culture specific?

How can casual conversations be recorded and analyzed?

Discuss this in groups

The recording and analyzing process

1. Video/tape record the conversation2. Transcribe the conversation3. Select one or more of these features:

- structure, content, genre, grammatical form- turn-taking, domination, interruptions, silences, overlapping, topic switches, breakdown, gender roles

4. Analyze the features you selected e.g. interruptions -types of/who interrupts/ when/how/why/what is the result?

Transactional encounters

Conversation in the exchange of goods and services

e.g. doctors/patients, teachers/students, service

person/customer, police/citizen

Exercise in conversational analysis

In groups of four, - two students role play a conversation between a

teacher and student- two students record the data, - all four analyze the data.

- The observers should decide which aspect of the conversation they will observe and record.

• Bell, Questionnaires• • Question types• • verbal or open• choose from a list• choose form a category e.g. age group• rank orders• quantity• • Q wording• • ambiguity/precision• assumptions• memory, how reliable• knowledge, is it available• avoid double wording e.g. do you attend research and statistics courses?• leading Qs• presuming Qs• hypothetical’s• • How do you identify the sample?• • Do you pilot the questionnaire?

• Patton, Qualitative Interviewing• • Informal conversational interview• unstructured, not predetermined• enables foci and issues to emerge• • Interview guide• same general topics with each interviewee but with scope to pursue related lines of inquiry• • Standarized open-ended interview• - same exact wording and sequence of questions• - guide available for inspection• - variation can be minimised• - efficient use of time• - responses can be compared• • Fixed response interview• • p. 320 summary of interview types• •

• Yin, Case Study Research/Data Collection• • Use multiple sources to achieve triangulation to produce ‘converging lines of inquiry’, 304• • triangulation of data• investigator triangulation• theory triangulation• methods triangulation• • Create a case study data base• • Maintain a chain of evidence to enable third party to trace sources• • •

Discourse Analysis

1. Studies how spoken or written texts make sense e.g. how do the references work in the text.

2. Studies how language functions such as ‘inviting’, ‘apologising’, ‘explaining’ etc. work.

In goups of four role play and analyse an apology.