Methods of researching Methods of researching things that haven’t happened things that haven’t happened

… especially good things and bad things… especially good things and bad things

Michael WoodMichael WoodPortsmouth Business SchoolPortsmouth Business School

michael.wood@port.ac.ukmichael.wood@port.ac.ukhttp://userweb.port.ac.uk/~woodm/NotHappened.ppthttp://userweb.port.ac.uk/~woodm/NotHappened.ppt

Slightly revised after comments from PBS Research Conference Slightly revised after comments from PBS Research Conference 8 June 20108 June 2010

AbstractAbstractMy aim in this talk is twofold. First, I will argue that most management

research gives too much status to facts and observations, focusing on what has happened in the past – on transcripts of people saying what they’ve done or what they think, or on subtle statistical distinctions between one aspect of past history and another. Such data may be important, but the danger is it encourages us to ignore the potentially much more interesting task of exploring possibilities – what could be or what should be – and predicting, or shaping, or making recommendations about, the future. Second, I will – tentatively – suggest some approaches for doing such research – these include methods involving experimentation, mathematics, modeling and simulation, works of fiction (including thought experiments), analysis of “common sense” data and hypothetical surveys. (The presentation will be on the web at http://userweb.port.ac.uk/~woodm/NotHappened.ppt .)

Where am I coming from?Where am I coming from?

• Vague feeling that I really don’t care about endless empirical data, but I’m more interested in new, exciting possibilities. – What might be instead of what is or was.– (Perhaps I just don’t like the real world?)

• I’m often asked how big should a sample be ... Expected answer is 30 or 50, so it’s rather fun to say 0!

• I suspect everything I want to say is entirely obvious, except possibly to those who know a little about research methods.

• Not about OR and economics, which already have a tradition of being flexible with the facts

• Not final word – in the spirit of anarchistic epistemology (Feyerabend, 1993 and Law, 2004)

I will …I will …

• Discuss what I mean and why it’s important

• Suggest some approaches for researching important things that haven’t happened

• Consider one or two case studies, with a class exercise

What hasn’t happened?• In one sense everything in the future hasn’t happened

yet – researching this is very important ... and obviously difficult!

• But if we can assume future is similar to past, then we can study past happenings to try to learn about the future. But ...– May be difficult to find past examples– Taleb’s black swans and the fate of the turkey (Taleb,

2008)• If future happenings are not likely to be similar to past

happenings then we have a problem!

Things that haven’t happened that Things that haven’t happened that are worth studying includeare worth studying include

• Things that may happen in the future that aren’t obviously similar to things that have happened in the past (and about which we can find get data)

Especially:– Good things, that we want to recommend or help to

happen– Bad things, that we would like to avoid (e.g. typical

risk analysis)• (Also counterfactuals to understand the past …?)

Typical MBA project: research into Typical MBA project: research into how to improve Xhow to improve X

1 Investigate the current problems with X.

2 Investigate how other organisations deal with X

3 Taking account of the findings from (1) and (2), and the literature and any ideas within the organisation, and any other sources of inspiration, produce some recommendations for improving X

4 Test the proposals to see how they work

The research bit of this project is The research bit of this project is normally seen as 1 and 2 onlynormally seen as 1 and 2 only

• The rhetoric of research methods (books, courses, etc) focuses on Steps 1 and 2 because these concern facts or observations about what has happened in the past – methods typically viewed as “qualitative” or “quantitative”

• Management tries to be “scientific” which – supposedly – means that we must focus on the facts about what has happened in the past. Hence the obsession with data, questionnaires (dismissed as “autoerotic fantasies by Salancik, 1979), interview transcripts, etc.

But …But …

• This ignores Steps 3 and 4 in which we want to formulate some recommendations for the future and test them

• This is surely the main problem with management research – how to investigate things which haven’t yet happened … answer in a minute, but first …

Taken-for-granted assumptions of Taken-for-granted assumptions of “research methods”“research methods”

• Empiricism: everything must be based on facts – ie on surveys, interviews, observations

• Methodism: approved methods only must be used

These assumptions seem to outlaw research on things that haven’t happened like Steps 3 and 4

Facts and empiricism in physicsFacts and empiricism in physics• Physics often seen as the exemplar for “science”, and

E=mc2 is probably the most famous result in physics. However …– unlike results in management, this is assumed to hold

everywhere and for all time– it summarises an infinite collection of possible events,

most which haven’t happened. E.g. my conversion to energy … (about 20,000 million million KwH)

– it was derived from thought experiments and only indirectly from observations, and certainly not on the basis of a statistical survey of matter in the universe.

Methods of researching things that Methods of researching things that haven’t happened in managementhaven’t happened in management

• Not easy for obvious reasons …• May be design or invention, rather than discovery (e.g.

Ackoff’s idealized design)• What follows is just some ideas. Not comprehensive or

fully worked out …

Methods might be based on ...Methods might be based on ...1. Experimentation – ranging from informal to randomized

controlled trials (see Ayres (2007) for examples)2. Mathematics (which is a way of exploring possibilities

without trying them – eg stock control formulae)“… the subject in which we never know what we are talking about, nor

whether what we are saying is true” (Russell, 1901)

3. Modelling and simulation (another way of exploring possibilities without trying them) – e.g. Simulating reactions to disasters, or sales situations

4. Scenario planning – work out details of possibilities5. Works of fiction (including thought experiments, hypothetical

examples, fables, utopias, dystopias)6. Analysis of “common sense” data (if simple called

philosophy, if more complicated called maths)7. Surveys asking “what would you do if ...?”

Any others methods of researching things that haven’t happened …?

Case study 1: Alternative vote Case study 1: Alternative vote (AV) vs first past the post (FPP)(AV) vs first past the post (FPP)

• This case is fictional – I’m making it up rather than looking at a real project, but does this matter?

Method of researching what Method of researching what hashas happenedhappened

• Interview key stakeholders to get their opinions about FPP and AV

• Survey voters’ views about FPP and AV• Look at other countries

Typical management research stops here!

But if people haven’t experienced AV will they have a useful opinion? This does not really give adequate information about the new system ...

Methods of researching what Methods of researching what hasn’t hasn’t happenedhappened

• Mathematical analysis of properties of different systems (Method Type 2: eg the Gibbard–Satterthwaite theorem)

• Simulate different voting patterns and compare AV with FPP (Method Type 3)

• Ask a sample of people how they would have voted under AV (get them to vote) – and compare results with actual results (Method Type 7)

Case study 2: Problems and Case study 2: Problems and opportunities of blackberries for opportunities of blackberries for

WLBWLB

• Actual MBA project

Methods of researching what Methods of researching what hashas happened happened

• Interviews with 12 blackberry users

But not much information on possible serious problems, or useful innovations. Small snapshot of typical present uses.

Perhaps research should aim to go past this ...

What would you do?

Methods of researching what Methods of researching what hasn’t hasn’t happenedhappened

• Experimentation: ask people to not use a blackberry, use it in a particular way etc (Type 1)

• Fiction – imagine extreme scenarios involving blackberries and WLB (Type 5)

• Ask people what they would do if ... to explore things that haven’t happened to them (Type 7)

• Mathematical modelling of WLB and blackberries ...? (Type 2) …

These allow research to explore future possibilities

Very tentative math approach to Very tentative math approach to WLB and Blackberry use …WLB and Blackberry use …

• For each email away from work devise a way of measuring (relative to work) – Disruption caused– Time off work gained– Response time improvement

• Look at patterns and thus model uses which haven’t happened …

In conclusionIn conclusion

• The most difficult and important thing for management research is investigating things that haven’t happened

• Obsessively cataloguing views of what has happened can only help to a limited extent

• So … need to be imaginative …

Any further thoughts tomichael.wood@port.ac.uk please.

ReferencesReferences• Ayres, I. (2007). Super crunchers: how anything can be

predicted. London: John Murray.• Feyerabend, P.K. (1993) Against method: an outline of

an anarchistic theory of knowledge (3rd edition) London: Verso.

• Law, J. (2004) After method: mess in social science research. Abingdon and New York: Routledge.

• Salancik, G. R. (1979) Field stimulations for organizational behavior research. Administrative Science Quarterly, 24, 638-649.

• Russell, B. (1901) 'Recent Work on the Principles of Mathematics', International Monthly, 4, 84.

• Taleb, N. N. (2008). The black swan: the impact of the highly improbable. London: Penguin.