The Human Mortality Database:Current status and

on-going developments

Magali Barbieri Dmitri JdanovUniv. California, Berkeley Max Planck Institute for

INED, Paris Demographic Research, Rostock

Acknowledgement: this presentation is based on the work conducted by manymembers of the HMD team over the years, at both the University of California,

Berkeley, and the Max Planck Institute for Demographic Research (MPIDR), Rostock.

House of Finance Days, Université Paris-Dauphine, HMD users meeting, March 8, 2019

Background

• Goal of the HMD:To provide detailed high-quality mortality and population data free of charge to all persons interested in the history of human longevity

• 50,000+ registered users :Academics (5,000 publications)StudentsActuariesPolicy analystsJournalistsOther corporations (pension funds, banks, etc…)

2

What is in the HMD?• Detailed historical data and supporting

documentation for 40 national populations:

– Death counts and estimated population exposures (person-years lived) at the finest detail possible

– Original estimates of age-specific death rates and life tables in various formats (age x time)

• Computed using various forms of input data:

– Death counts from national statistical offices

– Census counts

– Birth counts

– Official population estimates3

Additional information in

HMD funders and sponsors

Grants and donations:

Support provided by the U.S. National Institute on Aging (grants R01-AG011552 and R01-AG040245), the U.K. Institute and

Faculty of Actuaries, the Canadian Institute of Actuaries, the Dutch Royal Actuarial Association, AXA, Hannover-Re, Milliman-

France, RGA, SCOR and the Society of Actuaries.

Disclaimer: The author is solely responsible for the content of this presentation, which does not necessarily represent the official views of the

National Institutes of Health and other sponsors.

Who is responsible for the HMD?

Two teams of researchers:

• Max Plank Institute for Demographic Research (in Rostock, Germany)led by Vladimir Shkolnikov, Director

• UC Berkeley (Dept of Demography)led by Magali Barbieri, Associate Director(previously John Wimoth, Founding Director)

John R. WilmothFounding Director,

UCB in 2000, now UN

Vladimir M. Shkolnikov Director, MPIDR

Magali BarbieriAssociate Director,

Head of the UCB Team, UCB&INED

Dmitry Jdanov Head of the MPIDR

Team, MPIDR

Domantas Jasilionis

Sebastian Kluesener

Pavel Grigoriev

EvgenyAndreev

Eva KibeleSigrid Gellers

Rembrandt Scholz

Max Planck Team(members present and some former)

Berkeley Team(members present and some former)

Carl Boe

Dana Glei

Tim Riffe

Celeste Winant MonicaAlexanderLisa Yang

Gabriel Borges

Vladimir Canudas-Romo

Kirill Andreev

HMD Project Staff (March 2019)

• Directors (1 + 1 = 2)

• Country specialists (4 + 5 = 9)

• Administrative assistants (2)

• Others providing technical support (4 + 1 = 5)

Guiding principles• Comparability

– Over time (from 1751 to 2017)– Across countries (40 mostly high-income)

• Accessibility– free and easy access to data and metadata

• Flexibility – Data files in multiple formats

• Reproducibility– Access to all initial (input) data– Full documentation– HMD scripts are freely available

• Quality control– Standardized rigorous data quality checks for regular updates – Intensive data check procedures and research work for new countries– Work with external experts

Core activities:HMD country updates (1)

• Update of all HMD countries on rotating basis

• Priority countries:– The US

– The UK and components

– Germany

– Japan

– France

– Russia

– Sweden

– Canada

Core activities:HMD country updates (2)

• Steps involved in country updates:1. Collect data (births, deaths, and populations from NSOs

– publicly available or customized tables)2. Format as standard input files3. Prepare cocktail script (from a palette of standard HMD

computer routines)4. Run script and adjust if needs be5. Check internal and external consistency of output

(automatic diagnostic charts and other standard verifications) => exchange with in-country experts

6. Update all documentation files (internal and public)7. Submit to HMD Directors for verification8. Publish on HMD website=> 1 to 3 weeks per country(sometimes more if particular issues arise)

Core activities:Investigate new countries

• Recently added countries:– Greece– Croatia– South Korea

• Other countries investigated– Costa Rica– Moldova

• In progress / plans– Serbia– Romania– EU28– Hong Kong

Core activities:Improve the HMD methods

• Motivation: increase accuracy of mortalityestimates

• Current Methods Protocol = Version 6 (Dec. 2017 => all countries updated to this new version in 2018)

• Version 5: work in progress

– New inter-censal method

– Old age mortality

Why HMD?

How HMD is used by actuaries

Three major applications

1. Standard for relational models (to link client pools to national population)

2. Analyses of variability in risks over time and across populations

3. Mortality improvement models (mainly for model development and experimentation)

An example: life expectancy in MoldovaAll correct figures are highlighted in yellow

Numerator-denominator bias: an example of Moldova

Source: Penina, Jdanov, Grigoriev (2015)

* Since 1998 official population counts do not include Transnistriaregion

The problem: systematic bias (deaths and births refer to the de facto population, (.e. occurred within the country, while population estimates also include long-term emigrants - Moldavian citizens living abroad) leads to an under-estimation of mortality and fertility

The solution: population estimates were corrected using data on border crossing and additional data collected at the census 2004

Censuses and assessment of the population denominator

Data challenges

Bulgaria: correction of population data

The standard HMD inter-censal method is not applicable to the period 1985-1992 because of an irregular pattern of out-migration. In 1985-8, international migration was very restricted in Bulgaria. After the collapse of communism in 1989 - mass emigration (mostly of the Turkish minority) over the next several years.

HMD Solution: official population estimates were used for 1985-8, but new population estimates were calculated for the latter period. The year 1988 was treated as a “pseudo-census point” as the beginning of the inter-censal interval.

1985

(census year)

2001

census year

1984

2000

1992

(census year)

1991

3500000

3700000

3900000

4100000

4300000

4500000

4700000

1961

1963

1965

1967

1969

1971

1973

1975

1977

1979

1981

1983

1985

1987

1989

1991

1993

1995

1997

1999

2001

2003

MALES

FEMALES

3500000

3700000

3900000

4100000

4300000

4500000

4700000

1980 1985 1990 1995 2000

Females

Males

Trends in the total number of males and females. Bulgaria, 1961-2003. Official population estimates (left) and HMD data (right). Source: Jasilionis D., Jdanov D.A. Human Mortality Database: Background and Documentation for Bulgaria

The HMD inter-censal estimates for Germany

1) Using additional migration data and cubic spline interpolation for migration trends across cohorts we removed the population changes due to the earlier “cleaning” by the statistical offices.

2) We distributed the accumulated error (not the net migration!) uniformly over the adjustment period of 24 years (30 years for East German lands):

Changeable population definitions across time

Data challenges

Changes in the definition of population: Poland

Figure: Official and adjusted (Tymicki et al. , 2015) estimates of population of Poland

14,000,000

15,000,000

16,000,000

17,000,000

18,000,000

19,000,000

20,000,0001

96

01

96

21

96

41

96

61

96

81

97

01

97

21

97

41

97

61

97

81

98

01

98

21

98

41

98

61

98

81

99

01

99

21

99

41

99

61

99

82

00

02

00

22

00

42

00

62

00

82

01

02

01

22

01

4

Pre- and post-censal population estimates according to the 2002

Post-censal population estimates calculated according to the 1988

census

Post-censal population estimates calculated according

to the 1970 census

Post-censal population estimates calculated according to the 1960

census

FEMALES

MALES Post-censal population estimates according to the

2011 census

Unfofficial inter-censal estimates

based on the 2011 census

In the 2000s, Poland faced a massive out-migration that followed the EU enlargement of 2004. It was expected that the population counts will be corrected downward after the next population census of 2011. But Statistics Poland has unexpectedly decided to change the official definition of the population status from the permanently resident (acting in 2010 and earlier) to the usually resident (from 2011 onward). Statistics Poland did not re-estimate age-specific population counts back to previous census. Due to irregular migration pattern the standard HMD inter-censalmethod for reconstruction of annual population estimates is not applicable.

Mortality at advanced ages

Data challenges

Growing problems at advanced ages

2.5

2.7

2.9

3.1

3.3

3.5

3.7

3.9

4.1

4.3

4.5

1980 1985 1990 1995 2000 2005 2010 2015

Life

exp

ect

ancy

at

age

90

Year

Males (Standard HMD)

Females (Standard HMD)

males (SR80)

females (SR80)

Russia: life expectancy at age 90

Free & open access to all data

Open Data

Availability and Access: the data must be available as a

whole and at no more than a reasonable reproduction cost,

preferably by downloading over the internet. The data must

also be available in a convenient and modifiable form.

Re-use and Redistribution: the data must be provided under

terms that permit re-use and redistribution including the

intermixing with other datasets.

Universal Participation: everyone must be able to use, re-use

and redistribute - there should be no discrimination against

fields of endeavour or against persons or groups. For example,

‘non-commercial’ restrictions that would prevent ‘commercial’

use, or restrictions of use for certain purposes (e.g. only in

education), are not allowed.

Thorough documentation of data,

data sources, and

computational methods

Data Sources:- Computed from microdata- Computed from tabulated data from government- Computed from tabulated data from report - Computed from tabulated population and death data- Final report- Others - Preliminary report- Unknown

Challenge

How can high-quality data repositories compete with the “quick” data solutions covering the entire world and

easily available online?

Ongoing work and future plans

• Adding cause-of-death data• Sub-national data• Less (statistically) developed countries

• Latin America• China• India

Ongoing work and future plans:developing countries

Coverage of death registration (December 2014)

Source: UN Population Division (http://unstats.un.org/unsd/demographic/CRVS/CR_coverage.htm)

Data availability in the HMD (March, 2019)

HMD countries

Principal data sources on mortality in China

CENSUSES OR SURVEYS BY NATIONAL BUREAU OF STATISTICS (NBS) Population censuses: 1982, 1990, 2000, and 2010.

- enumeration of people who died in a household one year or 18 months beforethe census or survey. Inter-censal 1% sample surveys: 1987, 1995, and 2005

Annual Population Change Surveys. Smaller surveys for inter-censal years.

HOUSEHOLD REGISTRY (“HUKOU”) BY MINISTRY OF PUBLIC SECURITY- each resident is legally required to register in the household registration system,

registration to be cancelled within a month after death. Serves as basis for census.

VITAL REGISTRATION / SURVEILLANCE SYSTEMS BY HEALTH MINISTRY Nationwide Vital Registration System: 8 % of the national population,

ca. 110 million people (2005 est.), mostly urban, Eastern China (Rao et al. 2005). Disease Surveillance Points (DSP): 161 surveillance points, ~10 mill. people. National Child and Maternal Mortality Surveillance Points:

336 counties / urban districts covering 140 mill. people, child and maternal mortality.

China and Sweden (1950+), Male

China and Sweden (1950+), Male

UnderestimatedInfant Mortality

Overestimated mortality for ages 1-15

No accidental mortality hump

Death underregistration at the oldest-old ages

Source: Human Mortality Database and China Census Data

Life Expectancy for Chinese Males

Life Expectancy for Chinese Females

Infant mortality rates from NFHS III (2001-05) and SRS (2002-06) for 16 Indian states

Temporary life expectancy between exact ages 0 and 60 in India, 1970-75 to 2000-04.

Graphs from: Saikia, Jasilionis, Ram, Shkolnikov, 2011.

SRS - a nationwide system for collecting vital statisticsbased on a dual record system fora sample of villages/urban blocks.Key features: Set up in late 1960s, age-specific mortality

estimates /life tables from 1970-75 onwards;Covers major states by urban/rural

breakdown;Coverage: ~7.6 million pop. (2014).Problems: age heaping, over-estimation of old

age mortality….

Ongoing and future studies in cooperation with UshaRam (IIPS, Mumbai, India):- Data quality and coverage (NFHS, DLHS, SRS) focus on effects of age heaping.- Estimation of adult mortality by social statususing District Level Household Survey (DLHS) & National Family Health Survey (NFHS4).

- Examining validity of cause of death data.

Harmonized series of mortality estimates for India and its major states using Sample Registration System (SRS) and survey data

General goal of the AXA project

• To construct HMD-like life table series for Hong Kong and for Mexico and assess their accuracies for monitoring actuarial longevity risks

• Collaboration between – the University of California- Berkeley team of the

HMD and

– the AXA group and local entities (AXA China Region and AXA Mexico)

– with technical support from the Mortality Branch of the United Nations Population Division

Specific aims

1. Construct a time series of life tables at the national level for Hong Kong and for Mexico using the HMD approach and methods protocol

2. Establish a standard set of data quality indicators to evaluate the reliability of the life tables (building from the demographic literature)

3. Measure the impact of data quality issues on the assessment of variations in biometrical risks and future longevity trends

4. Propose adjustments to the series to improve accuracy, using indirect estimation techniques and/or statistical methods

Motivations• Joint interest in assessing mortality trends and their accuracy in a

growing number of countries• For Academics:

– Pressure to include additional countries into the HMD while preserving the high quality of the database mortality series

– Opportunity provided by increased investments by international organizations and private sponsors to improve national demographic data collection systems=> need to monitor the international Millenium/Sustainable Development Goals

• For AXA:– Regulatory need to constantly improve data quality in countries of

operationLongevity and mortality risks are of paramount importance at Life level, not only to calculate the Solvency Capital Required, but also to define the Best Estimates. Both are based on historical data, and the more relevant the data, the more accurate the mortality and longevity risks monitoring.=> need to better assess variations in biometrical risks and future longevity trends in historical mortality series

Result for Hong Kong promising

MexicoThe perfect case study: complete but clearly

imperfect demographic information

50

1990

2016

.2.4

.6.8

1

log(2

0q60)

.001 .01 .1 .2 .4 .6log(5q0)

Females

19902016

.2.4

.6.8

1

log(2

0q60)

.001 .01 .1 .2 .4 .6log(5q0)

Males

HMD Countries MEX

Ongoing work and future plans:CoD data

• For all HMD countries with cause-of-death data following the International Classification of Diseases (ICD)

• Back to 1950 or earliest year available• Respectful of privacy issues

– No access to input data for some countries– Five-year age group

• Three set of data series consistent with all-cause series:– Cause-of-death fractions – death counts– age-specific death rates

• Shortlist of <100 exclusive cause-of-death categories (mostly compatible with EUROSTAT and NCHS)

• Emphasis on disruptions arising from revisions of the ICD52

Adding cause-of-death data

Prepared COD series

• The United States (1959-2016)

• England and Wales (1950-2013)

• France (1958-2015)

• Canada (1950-2009)

• Sweden (1952-2012)

• Norway (1951-2012)

• Japan (1950-2013)

• The Czech Republic (1950-2013)

53

COD project stalled

• Competition issue with the Human Cause-of-Death Database (HCD)

• Same research teams (MPIDR, INED, and UC Berkeley)

• Conceptual differences: same idea but withadjustments for changes in the International Classification of Diseases + HCD includes non-HMD countries (with indirect estimation methods)

• On-going discussions to combine both databasesand host them on the HMD website

Ongoing work and future plans:sub-national data

Sub-national databases à la HMD

• Canadian Human Mortality Database (Université de Montréal, Canada=> UC Berkeley – thanks to the CIA)

• Japan Mortality Database (Institute for Population and Social Security Research, Tokyo – Former PhD student at UC Berkeley)

• United States Mortality DataBase (USMDB, usa.mortality.org) at UC Berkeley (state seriespublished, county series in the work)

• Germany Mortality Database (MPIDR)

• Australia Mortality Database (new project)

• France Mortality Database (looking for funding)

Monitoring mortality at sub-national level: data, methods, and evidence

Call for papers

A two-&-half-Day International workshop

To be held at ANU, Canberra, Australia, October 15-17, 2019.

Organized by: The Australian National University, the Human Mortality Database team.