tqm in statistics finland’s interview data collection process matti simpanen, kai vikki and eero...

TQM in Statistics Finland’s Interview Data Collection ProcessMatti Simpanen, Kai Vikki and Eero Tanskanen

11th of July 2008 2M.Simpanen, K.Vikki, E.Tanskanen

Data collection!Data collection!

Weighting, corrections

Analyse, reporting

Quality output Good reputation and glory!

Survey plan, sampling


Topics

The employment system The recruitment process Interviewer training The standardised interviewing method Setting the objectives Quality monitoring


Survey process is a coherent process

A survey is a process in which every individual stage is influenced by the other stage

Every part of the survey process has to have adequate understanding of the survey process, its backgrounds, goals and operating principles

Even interviewers Needs long-term, continuous quality monitoring (various perspectives)

Interviewers employment system What kind of employment system helps to keep quality high?

Remuneration From what interviewers are paid? Efficienty or quality? System has to be encouraging and fair


Quality starts from the recruitment process Systematic recruitment process

to measure applicant’s ability to understand and follow the standardised interview method

to measure interaction and style of interview (positiveness) to measure ability to adopt backgrounds and goals of surveys

quickly (general understanding of surveys and interview work) to measure general commitment abilities and a tolerance of routines Interviewers represent survey organisation, are impartials

In Statistics Finland recruitment process divides into three phases: Phone contact Structured job application form Job interviews


The quality demands training and orientation To orietate interviewers to survey process

basic training, project training, further training Basic training

The goal is to orientate new interviewers to interviewer’s work to familiarise interviewers to standardised interviewing method to orientate new interviewers to the survey process as a whole to orientate new interviewers to the organisation (commitment)

In SF one week basic training + repetition and follow-up after 1-2 months


Interviewer skills need continuing training Project training

to give adequate understanding of the individual survey project and its backgrounds

Further training to develop and to maintain interviewers skills to motivate interviewers to commit interviewers to the survey process and to the quality work to repeat items of standardised interviewing method

Goal of training is also to get interviewers understand their role in the survey process and the value of their work


Questionnaire standards Interviewing techniques Interviewing style Interview interaction

Standarised interviewing method


Questionnaire standards

Structured way to make questions, answer alternatives, introductions, descriptions to the questionnaires

Helps the interviewer realise how to ask different kind of questions gives time to interviewer to use his/her skills of interaction to get the

interviews interviewer´s work is to do interviews, not to think how different

questions should be asked to reduce interviewer effect


Interviewing techniques

the manner in which interviewer presents questions and answer alternatives, introductions, descriptions

the manner interviewer presents the question modules and introductions and

clarifies questions if the respondent does not understand the question (probing)

Interview techniques allow more time to use to the interaction Using interviewing techniques the interviewer ”teaches” the respondent

to understand our way to do interviewing work (the ”right” answer atmosphere)

To keep quality high (to ask questions similarily leads to less interviewer effect and more comparable and reliable data)


Interviewer: Always read the following text as well:

Did you return in April, May or June from a domestic trip with overnight stay, with a destination of at least 30 km from your home.

Included are all trips with overnight stay in Finland; also trips to own free-time residence, visits to friends or relatives, and business and professional trips?

Amount of "No"-answers (no trips) in year 2001 and 2002

Travel Survey

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec All year

Month

0

10

20

30

40

50

60

70

80

90

100%

'02 '01

"Always read the following text as well"


Interviewing style

Articulation Pace Use of pauses and emphasis Speech manners Neutrality of language Voice in general


Interview interaction

We put effort to interaction: The basic precondition is to get the respondent to agree to the

interview Interviewers behave to convince the respondent (persuation)

To create right kind of atmosphere by applying the interviewing technique and to use the correct interviewing style

In SF we measure by recordings the interviewers´ interviewing techniques, interviewing style and interaction


Objectives, follow-up

To maintain quality high needs the objectives, the goal settings efficiency and quality to the projects and to the interviewers schedule, response rate, quality

Quality control by monitoring (recordings, supervisor monitoring) Objectives to commit interviewer

to the project to the survey to the (survey)organisation

Cost control system


Feedback system

Appraisals Interviewer feedback surveys Respondent feedback surveys


Appraisal discussions

Feedback of the individual survey project is not enough to keep quality of the interviews high

Needs broader discussions with interviewers how the employee has succeeded how to improve co-operation in the work society

motivation professional development

and again to commit him-/herself to the survey and to interview work In SF each employee has annual development and performance

discussions (appraisals) with their immediate or own supervisor personal performance -> flexible personal pay component


Interviewer feedback surveys

Systematic, comparable feedback surveys from each projects to develope questionnaires and survey processes systematic feedback

Also systematic feedback on the interviewers’ working conditions, work methods and job orientation

to motivate to commit the interviewers part of the survey processes

Feedback surveys show also that researchers/organisation respects interviewers’ attitudes and their work


Respondent feedback surveys

Systematic, comparable feedback surveys from each projects to develope questionnaires and survey processes systematic feedback response burden validation


At the end...

The main point is NOT to say every organisation should have its own interviewing organisation but ...

survey is the coherent process - including several subprojects every party of the whole process has to have similar values of the

process every party has to have similar understanding of the process every party needs to commit themselves to the survey process - not

only an individual project Processes should be systematic and systematically evaluated To maintain quality of data collection demands continuing follow-up All these mentioned before have a great influence to the quality Poor data collection cannot be corrected with mathematical tricks!


Grazie! Thank you!

[email protected](back in the office 4.8.2008)

mailto:[email protected]

tqm in statistics finland’s interview data collection process matti simpanen, kai vikki and eero...

Documents

tanskanen survey process

survey process basic

tanskanen quality

new interviewers

training basic training

interviewers skills

coherent process

orietate interviewers