How to avoid some of the pitfalls when deploying legacy targeting models

David Dipple

Introduction

Fellow of Royal Statistical Society Worked with Not For Profit and Charity Clients for over

25 years Recognised as an expert data modeller Trained numerous analysts and fundraisers in the use

of analysis in fundraising Worked with charities in UK and mainland Europe

David Dipple

Words from the wise

An approximate answer to the right question is worth a great deal more than the precise answer to the wrong question.

-The first golden rule to applied mathematics

The formulation of a problem is often more essential than its solution which may be merely a matter of mathematical or mental skill.

•A. Einstein

Processes

The analysis process?

Question

Answer

Gubbins

Process Flows

The only point where there is interaction is at the start – no time is allocated for re-visiting the question

The Analysis Processes

Results workshop

Answer

Marketing Brief

Question Initial Brief Analysis Brief

Analysis reqs

Initial Analysis

Initial results

Final analysis

Needs, wants and requirements

Marketing

Analysis

Forget complex relationships – simplicity is your friend Analysis follows the 80/20 rule◦ 80% of the analysis can be done in 20% of the time.◦ The last 20% takes 80% of the time

Looking for the perfect answer?

Who?

HealthNature

Humanity

Environment

Wildlife

Disability

Cancer & Medical Research

Religion

Inequality

3rd Word & Overseas

Animal Welfare

Binary Clustering: Charity Sector

Our Target?

Or

Traditionally many legacy campaign have been designed and devised around a message they are not shaped around supporters needs and requirements

To fully tap the legacy potential of the base a more supporter lead strategy would match supporter interests and propensity to legacy message

Campaign vrs Supporter Lead

Method◦ Mail◦ Phone◦ Event◦ Online

The halo effect

Getting the Message Across

Data

The Data Triangle

Questionnaires, Interests & Beliefs

Lifestage, Age, GenderGeodems

Segmentation

Recency, Frequency,Value, Forms of help.

AttitudinalDemographic

Behavioural

Supporter Information

Donor &Demographic

DetailsDatabaseDerived

Donations

Communications

Attitu

dinal

LTVsRFVsScores

Media codesResponsesMethod

InterestsLifestyleCause

NameAddressGenderAgeIncome

Payment TypeAmountDate

Donor Information

Donor & Demographic

Details

DatabaseDerivedDonations

CommunicationsAttitudinal

Geo-Dems are great for cold and certain aspects of warm targeting

For small population analysis they tend to be less useful◦ For one model that I created by using a geo-dem it added

0.5% to the power of the model Take care with including or excluding people based on

their geo-dem coding

Geo-Dems

Do they mean me? I think that they do!

Academic Centres, Students and Young Professionals

Retired - Low income - Aged in the City Suburbs

Acorn Description

PersonicxDescription

People tend to be interested in people◦ But why are they interested?◦What aspects of your cause excites them?◦What motivates them to give you money?

Interests

What data do we currently have?◦What is its quality

What data would we like to have?◦What barriers are there to getting it?

So what we do not not know?

Modelling & Analysis

But what type of model?◦ Legacy◦ Pledger◦ Legacy & Pledger◦ Residuary/Pecuniary

The past determines the future◦ Lifetime Model◦ Time Limited Model◦ Something Else

But We Need a Propensity Model!

SPSS Excel FastStats MapInfo & MapPoint My own software

My Analysis Toolkit

Modelling techniques◦Binary Logistic◦Discriminant◦Multinomial Logistic◦CHAID◦Proxy

Legacy Models

Type of Data◦Number of Relationships◦Supporter Lifetime◦Number of Gifts◦Age of Supporter◦Gift Aider

Time is not our friend!

Important Legacy Factors

Gender Response Age ResponseMale 8% Young 12%Female 10% Old 12%

PopulationResponse: 10%

Gender: MaleResponse: 8%

Gender: FemaleResponse: 12%

Age: YoungResponse: 15%

Age: OldResponse: 5%

Age: YoungResponse: 10%

Age: OldResponse: 16%

Beware of False Relationships

Understanding models: Basic outputClassification Tablec

776 134 85.3 908940 153597 85.5

173 725 80.7 83 272 76.6

83.0 85.5

Observed0

1

Legator

Overall Percentage

Step 10 1

Legator PercentageCorrect

Selected Casesa

0 1

Legator PercentageCorrect

Unselected Casesb

Predicted

Selected cases sel_var EQ 1a.

Unselected cases sel_var NE 1b.

The cut value is .500c.

Multiple ways of understanding if a model has worked. Most of the output can be ignored by non statisticians and the key – The key is finding what needs to be communicated to marketers and in what form. used to determine power.

Tom Smith

Testing Down the Model

High ScoreSelected

Supporters

Model Score

Even with a small population outcome models – test down the model to reduce the Tom Smith effect.

Building legacy models has so far been carried out by building statistical propensity models. These need previous results to determine what will happen.

But if there are no previous results you can’t build a model or can you?

No Legacy Info – don’t worry!

The factors that increase propensity to make a pledge or leave a legacy are fairly well know – as we saw earlier

Create binary flags for each of the data items given earlier and then add them all up. The higher the result, the more likely to make a pledge (and it works).

Proxy Models

Analysis of a legacy campaign tends to be point based, That is how many responded to being contacted

To truly understand the effect of legacy campaigning the relationship over time needs to be examined, including the effect on non legacy messages – that is the full supporter journey

Longitudinal Analysis

Going Forward

The Future?

Warehouse

Model

Message 1 Message 2 Message 3 Message 4

No Contact (at this point…)

Single model that determines both who should be contacted and with what message.

The biggest barrier to producing efficient models is lack of data – especially demographic and attitudinal data

Understand what the data is saying and then use an appropriate model - There is no one perfect solution

There is no certainty in modelling – models are built from past behaviour and if you change what you are doing it can take a while for the data to catch up

Examine the whole supporter journey to understand the full relationship

Define the question and the answer will be much easier – remember a model is not a panacea

Conclusions

Thank You For Listening

David.dipple@adroitinsight.com