www.ipums.org/international1 using a restricted-access web-site of anonymized, integrated census...

www.ipums.org/international 1

Using Using www.ipums.org/international www.ipums.org/international a restricted-access web-site of a restricted-access web-site of

anonymized, integrated anonymized, integrated census microdata census microdata

(for 1, 2, 3, 4, 5, 6, 7, 8 … 50+ (for 1, 2, 3, 4, 5, 6, 7, 8 … 50+ countries) countries)

* * ** * *Robert McCaa, Steven Ruggles, & Matt Robert McCaa, Steven Ruggles, & Matt

SobekSobekUniversity of Minnesota Population CenterUniversity of Minnesota Population Center

[email protected]@umn.edu


OutlineOutline

» 1. What are census microdata?1. What are census microdata?» 2. What is IPUMS-International?2. What is IPUMS-International?

» Partners: > 50 official statistical agenciesPartners: > 50 official statistical agencies» Goals: inventory, preserve, anonymize, harmonize, Goals: inventory, preserve, anonymize, harmonize,

disseminate (free of cost: disseminate (free of cost: www.ipums.org/international)www.ipums.org/international)

» Principles: respect, retain, harmonize, enhance, free Principles: respect, retain, harmonize, enhance, free research research

» 3. How do we integrate?3. How do we integrate?» Use international standardsUse international standards» Expert national assistanceExpert national assistance

» 4. Who are our users and what do they use it 4. Who are our users and what do they use it for?for?» Major research universities in the US & world-wideMajor research universities in the US & world-wide» Policy researchers: World Bank, WHO, etc.Policy researchers: World Bank, WHO, etc.


121001026007007200000112100001042220020260070072000001121000010432300100600700720000012123000000423002004007000000000000000000005230020020070000000000000000000062300200000700000000000000000000

What are “census microdata”?:What are “census microdata”?:anonymized, computerized anonymized, computerized

census records of individuals, census records of individuals, households & dwellingshouseholds & dwellings

Researchers can:Researchers can:

Easier to harmonize than tables.Easier to harmonize than tables.Study any desired set of characteristics.Study any desired set of characteristics.

Facilitates comparative research.Facilitates comparative research.


IPUMS-International partners IPUMS-International partners by stage (please see Table 1)by stage (please see Table 1)

darkest green = final stage (data in dissemination)darkest green = final stage (data in dissemination)lighter green = middle stage (agreement signed)lighter green = middle stage (agreement signed)lightest green = first stage (agreement pending)lightest green = first stage (agreement pending)


IINNTTEERRNNAATTIIOONNAALL

II

PP

UU

MM

SS

8 countries available now (Table 2)8 countries available now (Table 2)1998 – Colombia, National Institutes of Health, 1 1998 – Colombia, National Institutes of Health, 1 countrycountry1999 – International, National Science Foundation, 1999 – International, National Science Foundation, 7 countries7 countries

Brazil, China, France, Kenya, Mexico, USA, Brazil, China, France, Kenya, Mexico, USA, VietnamVietnam

30 country-projects underway30 country-projects underway 2003 –Latin America, NIH, 14 countries2003 –Latin America, NIH, 14 countries

Argentina, Argentina, **Bolivia, Chile, Costa Rica, Bolivia, Chile, Costa Rica, DominicanDominican

Republic, Ecuador, El Salvador, Guatemala, Republic, Ecuador, El Salvador, Guatemala, Honduras, Nicaragua, Panama, Paraguay, Honduras, Nicaragua, Panama, Paraguay,

Peru,Peru,Venezuela Venezuela

2004 –Europe, NIH, 16 countries2004 –Europe, NIH, 16 countriesAustria, Belarus, Bulgaria, Czech RepublicAustria, Belarus, Bulgaria, Czech RepublicGermany, Greece, Hungary, Netherlands, Germany, Greece, Hungary, Netherlands,

Poland,Poland,Portugal, Romania, Portugal, Romania, **Russia, Slovenia, Spain, Russia, Slovenia, Spain, **Turkey, UKTurkey, UK

17 country-projects to begin next year17 country-projects to begin next year 2005 – Global, NSF, 17 countries2005 – Global, NSF, 17 countries

Bangladesh, Canada, Egypt, Bangladesh, Canada, Egypt, **Iran, Iran, **Iraq, Iraq, Israel,Israel, **Indonesia, Madagascar, Malawi, Malaysia, Indonesia, Madagascar, Malawi, Malaysia, Mongolia,Mongolia, Pakistan, Palestinian Authority, Pakistan, Palestinian Authority, Philippines, Philippines, Tajikistan, Turkmenistan, Uganda Tajikistan, Turkmenistan, Uganda


IPUMS-International IPUMS-International a global collaboratory of Official Statistical a global collaboratory of Official Statistical

Institutes & Universities to: Institutes & Universities to: » 1. 1. InventoryInventory the world’s census microdata the world’s census microdata» 2. 2. PreservePreserve endangered microdata and endangered microdata and

documentationdocumentation* * ** * *

» 3. 3. IntegrateIntegrate census microdata census microdata» a. use standards of UNSD, Eurostat, ISCO, ISCED, etc.a. use standards of UNSD, Eurostat, ISCO, ISCED, etc.» b. facilitate comparative research in time and spaceb. facilitate comparative research in time and space

» 4. 4. AnonymizeAnonymize census microdata to preserve census microdata to preserve statistical confidentiality, using highest statistical confidentiality, using highest standardsstandards

» 5. 5. DisseminateDisseminate restricted access, custom restricted access, custom extractsextracts to approved researchers/research to approved researchers/research projects at no costprojects at no cost

Demonstration:Demonstration:


IPUMS-International principles:IPUMS-International principles:

» 1. 1. RespectRespect absolute anonymity and confidentiality absolute anonymity and confidentiality» 2. 2. RetainRetain all original data, except adjustments to all original data, except adjustments to

insure privacy (top codes, blurrings, masking, re-insure privacy (top codes, blurrings, masking, re-ordering, etc.)ordering, etc.)

» 3. 3. HarmonizeHarmonize codes using international standards codes using international standardsoccupation: ISCO, HISCO (detailed, general)occupation: ISCO, HISCO (detailed, general)education: ISCED education: ISCED “ “ “ “family: IPUMS, etc. family: IPUMS, etc. “ “ “ “

» 4. 4. EnhanceEnhance with constructed variables (momloc, with constructed variables (momloc, etc.)etc.)

» 5. 5. DisseminateDisseminate free of charge to reseachers who free of charge to reseachers who need the data and agree to abide by conditions of need the data and agree to abide by conditions of useuse


IINNTTEEGGRRAATTEESS

Photos from Colombia Photos from Colombia integration projectintegration project, February, February--

March, 2000:March, 2000:4 experts from DANE (census 4 experts from DANE (census

office)office)+7 academics (3 universities)+7 academics (3 universities)

IIPPUUMMSSii

Standard:UN/Standard:UN/Eurostat Eurostat Principles & Principles & Recs...Recs...

Census Census documentation documentation compiled for compiled for Colombian Colombian microdatamicrodata


IINNTTEEGGRRAATTEESS

II

PP

UU

MM

SSii

» » Integrate (harmonize), not Integrate (harmonize), not standardizestandardize

1. retain all original detail1. retain all original detail2. harmonize every digit2. harmonize every digit

» » How is this possible? How is this possible? Composite codes (multiple Composite codes (multiple

digits, 111)digits, 111)Not serial (1, 2, 3, ....) Not serial (1, 2, 3, ....) ((Please see Table 3Please see Table 3))

» » Why? Researcher confidentlyWhy? Researcher confidentlyunderstands the logicunderstands the logicuses as much detail as uses as much detail as

neededneeded


Composite coding scheme: Composite coding scheme: employment statusemployment status

IPUMSI IPUMSI Col Col Fra Fra Ken Mex Mex US Viet Viet

Code Label 1964 1993 1962 1975 1999 1970 2000 1960 1989 1999

0000 N/A *,5 B * B BB 0 BB 00 B B,1

ACTIVE (In Labor Force)

1000 EMPLOYED, not specified 1 1

1100 At work 4 1 1 01 1 10 10

1101 At work, and 'student' 14

1102 At work, and 'housework' 15

1103 At work, and 'seeking work' 13

1104 At work, and 'retired' 16

1105 At work, and 'no work' 18

1108 At work, family holding, not agricultural 03

1109 At work, familiy holding, agricultural 04

1200 Have job, not at work last week 3 02 20 12

1300 Armed forces 13

1303 Military trainee (France) 8 6

2000 UNEMPLOYED, not specified 2 3 05 2 30 20

2001 Unemployed (Vietnam) 4 5

2002 Worked less than 6 months, permanent job 2

2003 Worked less than 6 months, temporary job 6

2100 Unemployed, experience worker 1 21

2200 Unemployed, new worker 2 7 22

3000 INACTIVE (Not in Labor Force) 30

3100 Housework 3 6 10 3 50 31 6 2

3200 Unable to work/disabled 7 7 09 70 32 7 4

3300 In school 4 5 9 5 07 40 33 5 3

3400 Retirees and living on rent 8 60

3402 Retirees/pensioners 8 4 08

3500 Elderly 6

3600 No work available/discouraged 06

3700 Inactive, other reasons 9 0 0 0 11 4 80 34 6

9000 UNKNOWN/MISSING 9 00 9 99 9

Harmonized Codes and Labels Source Data Codes (selected samples)

integrated codesintegrated codesPlease see Please see Table 3Table 3Codes in original Codes in original

datadata

1000 Employed, not 1000 Employed, not specifiedspecified

Variable Description: Please see Variable Description: Please see Appendix BAppendix B


Example of Variable Availability:Example of Variable Availability: China, 1982-2000 (Table 4) China, 1982-2000 (Table 4)

19198282

19199090

19919955

20020000

Total comparable Total comparable variablesvariables

2222 ++55

+1+155

+1+100

Person variablesPerson variables 2222 ++55

+7+7

+2+2

Dwelling variablesDwelling variables 2222 ++00

+8+8

+8+8

Geography--Region, Geography--Region, Prov., CityProv., City

33 33

33

33

Social variables Social variables (education, ethnicity, (education, ethnicity, family, marriage)family, marriage)

88 ++44

+3+3

+1+1


DDIISSSSEEMMIINNAATTEESS

II

PP

UU

MM

SSii

Legally-binding license Legally-binding license agreement agreement

»protects privacy and protects privacy and confidentialityconfidentiality»assures proper use assures proper use

Access limited to: Access limited to: »Bona-fide researchers Bona-fide researchers (credentials) (credentials) »With a demonstrated scientific With a demonstrated scientific needneed»who agree to abide by license who agree to abide by license restrictionsrestrictions

»ConfidentialityConfidentiality»Educational, policy uses only (no Educational, policy uses only (no commercial use)commercial use)»No redistributionNo redistribution»Safely securedSafely secured»Proper citationProper citation

Restricted Access web-Restricted Access web-based extraction system based extraction system

(Appendix B)(Appendix B)



II

PP

UU

MM

SSii

Researcher selects Researcher selects »Countries, Countries, »Censuses,Censuses,»Cases/sub-populations, Cases/sub-populations, »Variables, and Variables, and »Sample densitiesSample densities

»Users:Users:»Of 518 applicants, 288 accepted; Of 518 applicants, 288 accepted; denial of access rate = 44%denial of access rate = 44%»Usage by country (% of projects): Usage by country (% of projects): Mexico (59%), USA (50%), Colombia Mexico (59%), USA (50%), Colombia (38%), Kenya (37%), Vietnam (37%), (38%), Kenya (37%), Vietnam (37%), France (34%), China (24%). France (34%), China (24%).

Restricted Access web-Restricted Access web-based extraction systembased extraction systemfacilitates chronological &/or facilitates chronological &/or

cross-national researchcross-national research



II

PP

UU

MM

SSii

»International Organizations International Organizations (35)(35)

»WHO (5) WHO (5) »World Bank (6)World Bank (6)

»National Statistical Institutes National Statistical Institutes (9)(9)»US Universities (195)US Universities (195)

»California (17)California (17)»Harvard (7)Harvard (7)

»Non-US Universities (49)Non-US Universities (49)»Africa (4)Africa (4)»Asia (10)Asia (10)»Europe (21)Europe (21)»Americas (14) Americas (14)

Affiliations Of 288 Users, Affiliations Of 288 Users,

May 2002-2004 May 2002-2004 (examples)(examples)



II

PP

UU

MM

SSii

»Living arrangements of the Living arrangements of the aged aged »Female labor-force Female labor-force participation and educational participation and educational attainment attainment »Regional inequality Regional inequality differentialsdifferentials»International migrationInternational migration»Divorce and family compositionDivorce and family composition»U.S. National Academy of U.S. National Academy of Sciences, Sciences, “Transitions to Adulthood in “Transitions to Adulthood in Developing Developing Countries” uses data from Countries” uses data from Colombia, Kenya, Colombia, Kenya, Mexico, and Vietnam to analyze Mexico, and Vietnam to analyze changing changing outcomes such as schooling, work, outcomes such as schooling, work, fertility, and fertility, and marriage as a function of age, marriage as a function of age, gender, etc.gender, etc.

Research Topics Research Topics (examples)(examples)


BBEENNEEFFIITTSS

II

PP

UU

MM

SSii

»Researchers & policy-makers:Researchers & policy-makers:High-quality, harmonized microdata—High-quality, harmonized microdata—free of chargefree of chargeComprehensive, integrated metadataComprehensive, integrated metadata

»» Official Statistics Institutes: Official Statistics Institutes: Fair, free, uniform solution to Fair, free, uniform solution to providing census providing census

microdata microdata Trust, transparency, more stakeholdersTrust, transparency, more stakeholdersIncreased usage, better science/policy Increased usage, better science/policy Enhanced cost-benefit ratioEnhanced cost-benefit ratioLittle or no marginal cost (project pays Little or no marginal cost (project pays license fee)license fee)»» Citizens, Society, and Governments:Citizens, Society, and Governments:Who we areWho we areWhat the future may bringWhat the future may bringHow policies might be improvedHow policies might be improved


On a millennial scale, census microdata On a millennial scale, census microdata survive survive

for only a short, but significant periodfor only a short, but significant periodMay we help preserve yours? May we help preserve yours?

... help promote good use?... help promote good use?


You are invited to visit the You are invited to visit the web-site:web-site:

Twww.ipums.org/internationalwww.ipums.org/international contact:contact:

[email protected]@umn.edu

* * * * * * * * * *

Thank you!!Thank you!!

www.ipums.org/international1 using a restricted-access web-site of anonymized, integrated census...

Documents

countries brazil

countries austria

countries available

countries bangladesh

countries argentina

countryprojects underway

dominican republic

national science foundation