Download - Quality Control for Field Operations
![Page 1: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/1.jpg)
1
Quality Control for Field Operations
![Page 2: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/2.jpg)
2
Overview
Goal• To ensure the quality of survey field
work
Purpose• To detect and deter interviewer errors
and falsification
![Page 3: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/3.jpg)
3
Overview (cont.)
What do we check in QC for field work?• Interviewer procedures• Forms• Interviewer productivity• Interviews• Address listing
![Page 4: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/4.jpg)
4
Overview (cont.)
What are the tools for performing QC on field work?• Interviewer observations• Formal / Informal reviews• Monitoring reports• Reinterview• Address listing QC
![Page 5: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/5.jpg)
5
Interviewer Observations
What are the goals of interviewer observations?• To assess effectiveness of interviewer
training
• To ensure interviewers understand and follow prescribed interviewing procedures
• To provide interviewers with feedback to encourage continuous improvement
![Page 6: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/6.jpg)
6
Interviewer Observations (cont.)
What are interviewer observations?• Supervisor “shadows” interviewer during
initial assignments• Observe interviewer’s work to ensure
appropriate procedures are followed• Evaluate interviewer’s work and provide
positive and negative feedback to interviewer
• See Handout on Field Control Form Information
![Page 7: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/7.jpg)
7
Interviewer Observations (cont.)
What are the results of observations?• Supervisor decides if interviewer’s
performance is acceptable
• If so, allow interviewer to continue work independently
• If not, re-train interviewer on procedures or release interviewer from the operation
![Page 8: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/8.jpg)
8
Formal / Informal Reviews
What are the goals of reviews?
• To ensure the quality of the data collected on forms and questionnaires
• To ensure interviewers and clerks understand and follow procedures for completing forms and questionnaires
![Page 9: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/9.jpg)
9
Formal / Informal Reviews (cont.)
How are informal reviews performed?• Supervisor meets with employees to
review forms and/or questionnaires
• Review for legibility, completeness, and accuracy
• Provide positive and negative feedback to employee for continuous improvement
• Repair errors where necessary
![Page 10: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/10.jpg)
10
Formal / Informal Reviews (cont.)
How are formal reviews performed?• Usually performed in a field office by
office staff
• Review questionnaires or listings for legibility, completeness, and accuracy
• Repair errors or return materials to the field when necessary
![Page 11: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/11.jpg)
11
Formal / Informal Reviews (cont.)
What work products are subject to reviews?
• Any form or questionnaire that requires a quality check (e.g., payroll forms, questionnaires, address listing forms, etc.)
![Page 12: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/12.jpg)
12
Formal / Informal Reviews (cont.)
What are the results of informal reviews?• Supervisor determines if the form or
questionnaire meets quality standards
• If so, the form/questionnaire is accepted and submitted
• If not, repair the form/questionnaire prior to submission
![Page 13: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/13.jpg)
13
Monitoring Reports
What are the goals of monitoring reports?
• To track status of operations
• To inform management of progress
• To identify problems early for timely rectification
![Page 14: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/14.jpg)
14
Monitoring Reports (cont.)
How do we know what reports to monitor for QC purposes?• Define quality goals for the operation (e.g.,
response rates, productivity rates, cost, etc.)
• Identify variables that provide information on quality goals
• Design report(s) that present the important variables for quick review and assessment
![Page 15: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/15.jpg)
15
Monitoring Reports (cont.)
How do we monitor the reports for QC purposes?
• Determine necessary frequency for monitoring (usually daily)
• Look for outlier values (e.g., low response rates, high costs, etc.)
• Determine reason(s) for outlier values
• Take appropriate action
![Page 16: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/16.jpg)
16
Reinterview
What is reinterview?
• Quality check on interviewing activities
• Primary tool for ensuring quality of interviewers’ field work
• Independent, objective review of interviewing work
![Page 17: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/17.jpg)
17
Reinterview (cont.)
What types of reinterview can be used?
• Random reinterview
• Administrative reinterview
• Supplemental reinterview
![Page 18: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/18.jpg)
18
Reinterview (cont.)
What decisions are made in reinterview?• Falsification (release interviewer)
• Interviewer error (retraining/feedback)
• Respondent error (no action)
• Rework may be necessary for falsification or other serious errors
![Page 19: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/19.jpg)
19
Random Reinterview
What is the purpose of the random reinterview component?
• Provides broad protection over entire workload and all interviewers
• Serves as an effective deterrent for interviewer falsification
![Page 20: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/20.jpg)
20
Random Reinterview (cont.)
What is the sampling design for the random reinterview component?• Select random sample of cases within
every enumerator’s workload, or
• Two-stage design: take a sample of interviewers, then a sample of cases within those interviewers’ workloads
![Page 21: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/21.jpg)
21
Administrative Reinterview
What is the purpose of the administrative reinterview component?
• Provides “targeted” approach to identifying suspect work
• Provides valuable performance data to field supervisors
![Page 22: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/22.jpg)
22
Administrative Reinterview (cont.)
How does the administrative model work?• Accumulates data on work
characteristics for all interviewers• Vacant housing units• Partial interviews• Single-person households• Deleted housing units• Average population per housing unit
![Page 23: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/23.jpg)
23
Administrative Reinterview (cont.)
How does the administrative model work? (cont.)
• Compare accumulated data for each interviewer to the data for all other interviewers in their assignment area
• Identify and flag interviewers who are significantly different than their area average
![Page 24: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/24.jpg)
24
Administrative Reinterview (cont.)
How does the administrative model work? (cont.)
• Provide field supervisors with a list of “outlier” interviewers and what variables caused the flag
• Field supervisors determine if the interviewers’ “outlier” status is explicable
• More of suspect interviewer’s cases put into reinterview
![Page 25: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/25.jpg)
25
Supplemental Reinterview
What is the purpose of the supplemental reinterview?
• Provides field supervisors with the capability of checking work for selected interviewers
• Tool for use during investigation of suspected falsification
![Page 26: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/26.jpg)
26
Supplemental Reinterview (cont.)
What is the sampling design for the supplemental reinterview?
• Select a random sample of cases from the interviewer’s workload
• Field supervisor can specify how many cases to check
![Page 27: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/27.jpg)
27
Address Listing QC
What is the goal of address listing QC? To ensure the quality of address
listing work
• To protect against undercoverage and overcoverage
![Page 28: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/28.jpg)
28
Address Listing QC (cont.)
What are the tools for performing address listing QC?
Dependent QC
• Advance listing
• Suppression
![Page 29: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/29.jpg)
29
Address Listing QC (cont.)
What decisions are made in address listing QC?
• If errors exceed acceptable limit, recanvass assignment area
• Repair minor mistakes
• Listers may be released or retrained depending on severity of errors
![Page 30: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/30.jpg)
30
Dependent QC
What is dependent QC?
• Random, independent check of addresses listed by survey listers
• Field supervisors verify existence of housing units or validity of housing unit deletes
![Page 31: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/31.jpg)
31
Dependent QC (cont.)
What is the sampling design for dependent QC?
• Select a random sample of housing units to verify from the listers’ workload
• Sample should include all types of listings (e.g., verifications, adds, deletes)
![Page 32: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/32.jpg)
32
Advance Listing
What is advance listing?
• Field supervisor does initial listing of assignment areas prior to making assignments to listers
• Gives supervisor knowledge of the area so they are able to detect faulty or inadequate work
![Page 33: Quality Control for Field Operations](https://reader030.vdocuments.us/reader030/viewer/2022032709/56813019550346895d9593f3/html5/thumbnails/33.jpg)
33
Suppression
What is suppression?
• Selected addresses are purposely left off listings
• Listers’ ability to find and add the suppressed addresses is assessed
• Provides confidence in the listers’ ability to perform adequate canvassing