Re-designing the German job vacancy survey ─ assessing the impact of high non-response rates

Hans Kiesl, Susanne Rässler

Institute for Employment Research (IAB), Germany

International Conference on Establishment Surveys III

Montreal • June 18-21, 2007

2

Background

Information on job vacancies in Germany

Business units might report job vacancies to the Federal Employment

Agency

Federal Employment Agency publishes monthly statistics on number

of registered job vacancies (by industry class and NACE-sector)

IAB conducts a yearly (4th quarter) mail sample survey among

business units to estimate number of job vacancies (registered or

not) and to get additional information (e.g. about recruiting strategies)

Mail questionnaire (8 pages in length)

In the future: quarterly survey (CATI interviews in quarters 1-3)

3

Basic sampling design

stratified SRS; 16 sectors 7 size classes West/East

sampling rates and sample sizes in different size classes:

  West East

  rate sample size rate sample size

< 10 0.9% 11067 3.3% 10963

10 - 19 5.9% 8533 26.1% 9546

20 - 49 7.3% 6745 33.6% 8282

50 - 199 8.6% 4541 13.3% 1815

200 - 499 32.9% 3249 50.3% 1111

500 - 999 68.3% 1806 73.0% 395

>= 1000 75.1% 984 84.8% 193

       

total 2.4% 36925 8.0% 32305

4

Problem: extremely low response rates

0%

5%

10%

15%

20%

25%

30%

35%

1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

5

Reasons for non-response

After 2004 survey, CATI subsample of non-respondents to

find out main reasons for non-response

sample of 1700 business units

26% no/wrong telephone number

16% not willing to respond

58% respondents; their reasons for non-response in job vac. survey:

75%: no time; too much work (88% for largest units)

27%: no job vacancies (41% for smallest units)

25%: no relevant topic (44% for smallest units)

17%: take part in surveys only if mandatory

9%: never take part in surveys

6

Impact of length of questionnaire (1)

During the 4th quarter of 2006 (at the same time of the regular survey

with 8 pages) a separate survey was conducted (1 page, basic infos,

e.g. number of job vacancies).

Questions:

Has length of questionnaire significant impact on response

rates? (Prediction: yes)

If so, do different response rates lead to different estimates of

number of job vacancies? (Prediction: yes)

7

Impact of length of questionnaire (2)

Response rates by size of business units:

  8 pages 1 page abs. diff. rel. diff.

< 10 21.0% 27.3% + 6.3% + 29.9%

10 - 19 17.0% 30.7% + 13.7% + 80.1%

20 - 49 18.9% 28.2% + 9.3% + 49.1%

50 - 199 23.2% 28.6% + 5.4% + 23.2%

200 - 499 18.1% 27.1% + 9.0% + 49.9%

500 - 999 18.4% 28.5% + 10.1% + 54.9%

>= 1000 26.8%      

8

Impact of length of questionnaire (3)

Response rates by sector:

  8 pages 1 page abs. diff. rel. diff.

Agriculture, farming, forestry, fishing 24.8% 36.5% + 11.6% + 46.9%

Manufacturing 19.6% 29.8% + 10.2% + 52.3%

Construction 21.5% 28.8% + 7.3% + 34.1%

Sale, trade, hotels and restaurants, transport, communications 14.9% 23.3% + 8.4% + 56.1%

Financial intermediation, insurance, renting and business activities 17.3% 24.9% + 7.6% + 44.0%

Public administration, education, health, social work, other service activities 23.0%      

9

Weighting and estimation

Calibrate Horvitz-Thompson-estimator to totals from auxiliary data:

sampling frame out of date (> 1 year)

up-to-date estimates on number of units by size and by sector (no

cross-classification)

up-to-date estimates on number of employees by size and sector (no

cross-classification)

registered number of job vacancies by sector

Previously: iterative proportional fitting with additional restriction

two different weighting factors within each stratum (units with

and without job vacancies)

no variance estimation

Now: linear GREG

10

Effect on distribution of weighting factors

11

Impact of length of questionnaire on estimated number of job vacancies (1)

  8 pages Standard error CV

  1 page    

West 775,000 29,000 3.7%

  848,000 51,000 6.0%

East 212,000 10,000 4.7%

  229,000 32,000 14.0%

Germany 987,000 31,000 3.1%

  1,077,000 60,000 5.6%

12

Estimated number of job vacancies

0

100000

200000

300000

400000

500000

600000

700000

800000

900000

< 10 10 - 19 20 - 49 50 - 199 200 - 499 500 - 999 >= 1000

8 pages

1 page

Impact of length of questionnaire on estimated number of job vacancies (2)

13

Impact of length of questionnaire on estimated number of job vacancies (3)

Estimated number of job vacancies

0

50000

100000

150000

200000

250000

300000

350000

400000

450000

Agriculture,farming, forestry,

fishing

Manufacturing Construction Sale, trade,hotels and

restaurants,transport,

communications

Financialintermediation,

insurance,renting andbusinessactivities

Publicadministration,

education,health, socialwork, other

service activities

8 pages

1 page

14

Conclusions

Length of questionnaire has considerable effect on response rates

(as expected).

Change in response rates seem to have only little effect on main

survey estimates (job vacancies)

(not as expected).

Nevertheless split questionnaire design will be adopted in future.

Small-scale non-respondent CATI follow-up survey will be conducted

every year.

Thank you very much for listening!

International Conference on Establishment Surveys III

Montreal • June 18-21, 2007