Methods of researching things that haven’t happened … especially good things and bad things Michael Wood Portsmouth Business School Slightly revised after comments from PBS Research Conference 8 June 2010

Abstract My aim in this talk is twofold. First, I will argue that most management research gives too much status to facts and observations, focusing on what has happened in the past – on transcripts of people saying what they’ve done or what they think, or on subtle statistical distinctions between one aspect of past history and another. Such data may be important, but the danger is it encourages us to ignore the potentially much more interesting task of exploring possibilities – what could be or what should be – and predicting, or shaping, or making recommendations about, the future. Second, I will – tentatively – suggest some approaches for doing such research – these include methods involving experimentation, mathematics, modeling and simulation, works of fiction (including thought experiments), analysis of “common sense” data and hypothetical surveys. (The presentation will be on the web at

Where am I coming from? Vague feeling that I really don’t care about endless empirical data, but I’m more interested in new, exciting possibilities. –What might be instead of what is or was. –(Perhaps I just don’t like the real world?) I’m often asked how big should a sample be... Expected answer is 30 or 50, so it’s rather fun to say 0! I suspect everything I want to say is entirely obvious, except possibly to those who know a little about research methods. Not about OR and economics, which already have a tradition of being flexible with the facts Not final word – in the spirit of anarchistic epistemology (Feyerabend, 1993 and Law, 2004)

I will … Discuss what I mean and why it’s important Suggest some approaches for researching important things that haven’t happened Consider one or two case studies, with a class exercise

What hasn’t happened? In one sense everything in the future hasn’t happened yet – researching this is very important... and obviously difficult! But if we can assume future is similar to past, then we can study past happenings to try to learn about the future. But... –May be difficult to find past examples –Taleb’s black swans and the fate of the turkey (Taleb, 2008) If future happenings are not likely to be similar to past happenings then we have a problem!

Things that haven’t happened that are worth studying include Things that may happen in the future that aren’t obviously similar to things that have happened in the past (and about which we can find get data) Especially: –Good things, that we want to recommend or help to happen –Bad things, that we would like to avoid (e.g. typical risk analysis) (Also counterfactuals to understand the past …?)

Typical MBA project: research into how to improve X 1Investigate the current problems with X. 2Investigate how other organisations deal with X 3Taking account of the findings from (1) and (2), and the literature and any ideas within the organisation, and any other sources of inspiration, produce some recommendations for improving X 4Test the proposals to see how they work

The research bit of this project is normally seen as 1 and 2 only The rhetoric of research methods (books, courses, etc) focuses on Steps 1 and 2 because these concern facts or observations about what has happened in the past – methods typically viewed as “qualitative” or “quantitative” Management tries to be “scientific” which – supposedly – means that we must focus on the facts about what has happened in the past. Hence the obsession with data, questionnaires (dismissed as “autoerotic fantasies by Salancik, 1979), interview transcripts, etc.

But … This ignores Steps 3 and 4 in which we want to formulate some recommendations for the future and test them This is surely the main problem with management research – how to investigate things which haven’t yet happened … answer in a minute, but first …

Taken-for-granted assumptions of “research methods” Empiricism: everything must be based on facts – ie on surveys, interviews, observations Methodism: approved methods only must be used These assumptions seem to outlaw research on things that haven’t happened like Steps 3 and 4

Facts and empiricism in physics Physics often seen as the exemplar for “science”, and E=mc 2 is probably the most famous result in physics. However … –unlike results in management, this is assumed to hold everywhere and for all time –it summarises an infinite collection of possible events, most which haven’t happened. E.g. my conversion to energy … (about 20,000 million million KwH) –it was derived from thought experiments and only indirectly from observations, and certainly not on the basis of a statistical survey of matter in the universe.

Methods of researching things that haven’t happened in management Not easy for obvious reasons … May be design or invention, rather than discovery (e.g. Ackoff’s idealized design) What follows is just some ideas. Not comprehensive or fully worked out …

Methods might be based on... 1.Experimentation – ranging from informal to randomized controlled trials (see Ayres (2007) for examples) 2.Mathematics (which is a way of exploring possibilities without trying them – eg stock control formulae) “… the subject in which we never know what we are talking about, nor whether what we are saying is true” (Russell, 1901) 3.Modelling and simulation (another way of exploring possibilities without trying them) – e.g. Simulating reactions to disasters, or sales situations 4.Scenario planning – work out details of possibilities 5.Works of fiction (including thought experiments, hypothetical examples, fables, utopias, dystopias) 6.Analysis of “common sense” data (if simple called philosophy, if more complicated called maths) 7.Surveys asking “what would you do if...?”

Any others methods of researching things that haven’t happened …?

Case study 1: Alternative vote (AV) vs first past the post (FPP) This case is fictional – I’m making it up rather than looking at a real project, but does this matter?

Method of researching what has happened Interview key stakeholders to get their opinions about FPP and AV Survey voters’ views about FPP and AV Look at other countries Typical management research stops here! But if people haven’t experienced AV will they have a useful opinion? This does not really give adequate information about the new system...

Methods of researching what hasn’t happened Mathematical analysis of properties of different systems (Method Type 2: eg the Gibbard–Satterthwaite theorem)Gibbard–Satterthwaite theorem Simulate different voting patterns and compare AV with FPP (Method Type 3) Ask a sample of people how they would have voted under AV (get them to vote) – and compare results with actual results (Method Type 7)

Case study 2: Problems and opportunities of blackberries for WLB Actual MBA project

Methods of researching what has happened Interviews with 12 blackberry users But not much information on possible serious problems, or useful innovations. Small snapshot of typical present uses. Perhaps research should aim to go past this... What would you do?

Methods of researching what hasn’t happened Experimentation: ask people to not use a blackberry, use it in a particular way etc (Type 1) Fiction – imagine extreme scenarios involving blackberries and WLB (Type 5) Ask people what they would do if... to explore things that haven’t happened to them (Type 7) Mathematical modelling of WLB and blackberries...? (Type 2) … These allow research to explore future possibilities

Very tentative math approach to WLB and Blackberry use … For each away from work devise a way of measuring (relative to work) –Disruption caused –Time off work gained –Response time improvement Look at patterns and thus model uses which haven’t happened …

In conclusion The most difficult and important thing for management research is investigating things that haven’t happened Obsessively cataloguing views of what has happened can only help to a limited extent So … need to be imaginative … Any further thoughts

References Ayres, I. (2007). Super crunchers: how anything can be predicted. London: John Murray. Feyerabend, P.K. (1993) Against method: an outline of an anarchistic theory of knowledge (3rd edition) London: Verso. Law, J. (2004) After method: mess in social science research. Abingdon and New York: Routledge. Salancik, G. R. (1979) Field stimulations for organizational behavior research. Administrative Science Quarterly, 24, Russell, B. (1901) 'Recent Work on the Principles of Mathematics', International Monthly, 4, 84. Taleb, N. N. (2008). The black swan: the impact of the highly improbable. London: Penguin.