2011 census
DESCRIPTION
2011 Census. 2007 Census Test – emerging findings Garnett Compton, ONS. BSPS – 12 September 2007. Updated 4 September 2007 . Session Aim. Aim: To share emerging findings from the 2007 Census Test Outline plan for publishing 2007 Census Test evaluation - PowerPoint PPT PresentationTRANSCRIPT
2011 Census2007 Census Test – emerging findingsGarnett Compton, ONS
Updated 4 September 2007
BSPS – 12 September 2007
Session Aim
Aim: • To share emerging findings from the 2007
Census Test• Outline plan for publishing 2007 Census Test
evaluation• Present a brief update on other 2011 Census
developments
Contents• Test background• Results:
• Address register development• Post-out/hand delivery• Income question
• Key findings of 2007 Test• Further evaluation and publication• Did you know ….
2007 Census Test - Objectives
Test objectives:• Assess the effect on response of:
– Inclusion of an income question; and– the use of post-out to deliver questionnaires.
• Assess the feasibility of major innovations in proposed 2011 Census operational procedures:
– Outsourcing of field staff recruitment, pay and training;– development of an operational intelligence system to
enable individual questionnaires to be tracked; and,– development of an address list and address checking
procedures.
2007 Census Test - Sample sizes by LA and ETC
05000
10000150002000025000300003500040000
Camden
L'pool
Stoke
Bath & NE S
Car'shire
54321
ETC
2007 Census Test – High-level Design• Address checking
– Conducted in all Test areas during Sept and October– Split discretionary and full contact methods– Used controlled errors
• Delivery – 50% Post-out, 50% hand delivery– 50% questionnaires income, 50% no income – For hand delivery 3 attempts at contact over 2 week period
• Collection/Follow-up– Central post-back– 23 May – 22 June– 3 attempts everywhere– reminder letter to all outstanding addresses as at 31 May
2007 Census Test – Key constraints
Key constraints affecting evaluation of the Test:
• Voluntary– Relied on public’s good will to complete a return
• Publicity• Sample – skewed to harder to enumerate areas• Follow-up
– Fixed number of follow-up attempts everywhere
Results
Household* return rates by LA
0%
10%
20%
30%
40%
50%
60%
70%
Camden
L'pool
Stoke
Bath & NE S
Car'shire
All areas
* As at 15 July
Household* return rates by ETC
0%
10%
20%
30%
40%
50%
60%
70%
ETC* As at 15 July
Address register development
Results
Key results• Address checking (AC) found about 12% new
addresses– Time consuming to validate and update AR
• 4.4% (4,400) of questionnaires undelivered– Of which 17% were added by address checkers– Reasons include vacant properties, non-existing
addresses, duplicate addresses.• 1,200 new addresses found during enumeration
– Of which, over 50% found at follow-up.– Nearly 70% of new addresses were sub-premise
addresses – suggest existed at time of AC.
Early conclusions• Suggests we need to plan to do a 100% address
check in 2011 no matter what the delivery design.
• Plan is for a rolling address check over a longer period of time (4-6 months).
– enables better quality address checking; and,– enables more time to update the final address register
with Address check findings.
• Consider a re-address check shortly before the Census in a small %’age of areas
Early conclusions cont …• Main enumeration and controlled errors didn't
identify all missing addresses – need to improve methods and guidance
• Issues around whether information with address suppliers can be shared;
• Criteria for deciding which address products to use as a base under consideration.
Delivery Method
Results
Household return rates* by delivery method by ETC
0%10%20%30%40%50%60%70%80%
1 2 3 4 5 All areasETC
Post-out Hand delivery * As at 15 July
Success rates* at follow-up by delivery method by ETC
* As at 15 July
ETC Post-out Hand delivery
1 38.7% 39.9%
2 29.1% 29.4%
3 24.2% 26.1%
4 19.1% 18.4%
5 14.9% 16.3%
Overall 23.9% 24.3%
Delivery method – estimated cost savings
Estimated cost savings between 100% post-out and 100% hand deliveryInitial return* rate difference
(%’age points)Estimated savings
5 £28m - £35m
6 £25m – £33m
10 £6m - £21m
15 -£18m - £1m
* At the start of follow-up – 23 May
Address register coverage
New addresses found during the 2007 Census Test by delivery method
During delivery During follow-up Total
Delivery method
No. %’age No. %’age No. %’age
Post-out 9 0.02% 478 0.94% 487 0.95%
Hand delivery 540 1.06% 181 0.36% 721 1.42%
Total 549 0.54% 659 0.65% 1,208 1.19%
Delivery method - conclusions• Post-out has an impact on return rates, minimal
impact which can be addressed through additional follow-up.
• A post-out methodology will provide significant savings (£25-£33m) to invest in targeted follow-up and community liaison.
• Improvements identified for the address register and follow-up procedures suggest that the levels of AR undercoverage will be small and manageable.
Decision: In E&W, post-out will be the primary means (at least 85%) of delivering questionnaires in 2011.
Income – Setting the sceneIncluding income depends on:• results from the Test; and,• consultation on other topics and relative priority of
income in relation to other demands.
Further analysis required as follows:• Quality and accuracy of responses to income
question• Item imputation rates• Public perception• Other data sources
Household return rates* by income/no income question by ETC
0%
10%
20%
30%
40%
50%
60%
70%
1 2 3 4 5 All areas
Income No Income * As at 15 July
2007 Census Test - Key findings
Delivery method• In E&W, post-out will be the primary method (at
least 85%) of delivering questionnaires in 2011.Address register development• Address checking will be required for 2011,
currently planning 100% for E&W.Outsourcing recruitment, training and pay:• Worked well in the Test and are considering
outsourcing for 2011;
Other key findings:Recruitment:• More difficult than expected in some areas, mostly delivery
enumerators.• Good MI to identify and manage problems early.Training:• Some development issues with e-learning but overall well
received/effective.Pay:• Much simpler system defined thereby avoiding some of the
problems experienced in 2001.• Hourly pay worked well, provided good control and
flexibility; travel expenses still considered cumbersome. Refinement for 2011 required.
Other key findings:LA Liaison:• Principles and benefits of LA Liaison proven.• Variable engagement across the LAs – some more engaged than
others • Going forward we need to consider:
– methods for achieving more consistency across LAs; – Making it simpler for ONS and LAs– Achieving Chief Exec buy-in.
Follow-up:• Transfer of information held centrally to field staff worked well within the
constraints of the Test.• A good start to follow-up is imperative – need to review start dates.• Organisation and management of field staff worked well, but more
development required on doorstep interaction to convince respondent to respond.
2007 Census Test Evaluation– planned publications
Publication
Statistical Evaluation of the 2007 Census Test
2007 Census Test - Evaluation of the delivery method
2007 Census Test - Evaluation of the Income question
Evaluation of 2007 Census Test – Summary Report
Did you know?
Some other key Census developments:
• Rehearsal - Spring 2009• Route A – contract to be let in January 2008• Route C – start procurement in January 2008• Finalising questionnaire – Spring 2008• White Paper – Autumn 2008
Questions ????
Household return rates* by delivery method by LA
0%10%20%30%40%50%60%70%
Post-out Hand delivery * As at 15 July
Household return rates* by income/no income question by LA
0%10%20%30%40%50%60%70%
Camden
L'pool
Stoke
Bath
Carmarthen
All areas
Income No Income * As at 15 July